[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:10:21.257843 26959 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.83.254:42853
I20260709 13:10:21.259529 26959 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:10:21.260625 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:21.279166 26968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:21.279166 26966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:21.280339 26959 server_base.cc:1061] running on GCE node
W20260709 13:10:21.279173 26965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:22.553263 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:22.553521 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:22.553838 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602622553790 us; error 0 us; skew 500 ppm
I20260709 13:10:22.554738 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:22.561650 26959 webserver.cc:533] Webserver started at http://127.26.83.254:34567/ using document root <none> and password file <none>
I20260709 13:10:22.562597 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:22.562840 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:22.563336 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:22.569447 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/master-0-root/instance:
uuid: "c8a1d887873e49259848993e8211b778"
format_stamp: "Formatted at 2026-07-09 13:10:22 on dist-test-slave-329l"
I20260709 13:10:22.577961 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.003s	sys 0.005s
I20260709 13:10:22.584445 26974 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.585989 26959 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:10:22.586536 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/master-0-root
uuid: "c8a1d887873e49259848993e8211b778"
format_stamp: "Formatted at 2026-07-09 13:10:22 on dist-test-slave-329l"
I20260709 13:10:22.586926 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:22.641772 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:22.644114 26959 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:10:22.644622 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:22.724370 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.254:42853
I20260709 13:10:22.724444 27025 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.254:42853 every 8 connection(s)
I20260709 13:10:22.732793 27026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:22.755955 27026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778: Bootstrap starting.
I20260709 13:10:22.762388 27026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:22.764339 27026 log.cc:826] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778: Log is configured to *not* fsync() on all Append() calls
I20260709 13:10:22.769243 27026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778: No bootstrap required, opened a new log
I20260709 13:10:22.790479 27026 raft_consensus.cc:359] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a1d887873e49259848993e8211b778" member_type: VOTER }
I20260709 13:10:22.791133 27026 raft_consensus.cc:385] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:22.791395 27026 raft_consensus.cc:740] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a1d887873e49259848993e8211b778, State: Initialized, Role: FOLLOWER
I20260709 13:10:22.792400 27026 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a1d887873e49259848993e8211b778" member_type: VOTER }
I20260709 13:10:22.792945 27026 raft_consensus.cc:399] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:22.793267 27026 raft_consensus.cc:493] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:22.793629 27026 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:22.798667 27026 raft_consensus.cc:515] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a1d887873e49259848993e8211b778" member_type: VOTER }
I20260709 13:10:22.799625 27026 leader_election.cc:304] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8a1d887873e49259848993e8211b778; no voters: 
I20260709 13:10:22.801673 27026 leader_election.cc:290] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:22.802191 27029 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:22.805241 27029 raft_consensus.cc:697] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 1 LEADER]: Becoming Leader. State: Replica: c8a1d887873e49259848993e8211b778, State: Running, Role: LEADER
I20260709 13:10:22.806303 27029 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a1d887873e49259848993e8211b778" member_type: VOTER }
I20260709 13:10:22.807484 27026 sys_catalog.cc:565] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:22.821043 27031 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8a1d887873e49259848993e8211b778. Latest consensus state: current_term: 1 leader_uuid: "c8a1d887873e49259848993e8211b778" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a1d887873e49259848993e8211b778" member_type: VOTER } }
I20260709 13:10:22.822214 27031 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:22.821216 27030 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8a1d887873e49259848993e8211b778" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a1d887873e49259848993e8211b778" member_type: VOTER } }
I20260709 13:10:22.823087 27030 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:22.827207 27039 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:22.842474 27039 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:22.845288 26959 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:22.862228 27039 catalog_manager.cc:1383] Generated new cluster ID: bb534f603f294ce38f1bb00b367b8d8b
I20260709 13:10:22.862532 27039 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:22.883725 27039 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:22.885562 27039 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:22.903278 27039 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778: Generated new TSK 0
I20260709 13:10:22.904464 27039 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:22.912031 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:22.921238 27048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:22.923276 27049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:22.926927 26959 server_base.cc:1061] running on GCE node
W20260709 13:10:22.927623 27051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:22.928647 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:22.928916 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:22.929235 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602622929100 us; error 0 us; skew 500 ppm
I20260709 13:10:22.929855 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:22.934432 26959 webserver.cc:533] Webserver started at http://127.26.83.193:44815/ using document root <none> and password file <none>
I20260709 13:10:22.935017 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:22.935236 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:22.935531 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:22.936702 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-0-root/instance:
uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
format_stamp: "Formatted at 2026-07-09 13:10:22 on dist-test-slave-329l"
I20260709 13:10:22.942849 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:10:22.947808 27056 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.948711 26959 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:10:22.949113 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-0-root
uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
format_stamp: "Formatted at 2026-07-09 13:10:22 on dist-test-slave-329l"
I20260709 13:10:22.949438 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:22.988921 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:22.990708 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:22.993726 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:22.998697 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:22.998975 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:22.999346 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:22.999502 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:23.111789 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.193:44779
I20260709 13:10:23.111876 27118 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.193:44779 every 8 connection(s)
I20260709 13:10:23.125102 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:23.135708 27123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:23.142407 27124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:23.144205 27119 heartbeater.cc:344] Connected to a master server at 127.26.83.254:42853
I20260709 13:10:23.144760 27119 heartbeater.cc:461] Registering TS with master...
W20260709 13:10:23.145746 27126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:23.146214 27119 heartbeater.cc:507] Master 127.26.83.254:42853 requested a full tablet report, sending...
I20260709 13:10:23.146329 26959 server_base.cc:1061] running on GCE node
I20260709 13:10:23.147190 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:23.147432 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:23.147657 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602623147636 us; error 0 us; skew 500 ppm
I20260709 13:10:23.148468 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:23.150408 26991 ts_manager.cc:194] Registered new tserver with Master: af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779)
I20260709 13:10:23.152536 26959 webserver.cc:533] Webserver started at http://127.26.83.194:38647/ using document root <none> and password file <none>
I20260709 13:10:23.153208 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:23.153499 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:23.153666 26991 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59654
I20260709 13:10:23.153848 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:23.155284 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-1-root/instance:
uuid: "23b4242091514cf4887d6fdf87717de5"
format_stamp: "Formatted at 2026-07-09 13:10:23 on dist-test-slave-329l"
I20260709 13:10:23.163229 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:10:23.168085 27131 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:23.169130 26959 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:10:23.169495 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-1-root
uuid: "23b4242091514cf4887d6fdf87717de5"
format_stamp: "Formatted at 2026-07-09 13:10:23 on dist-test-slave-329l"
I20260709 13:10:23.169867 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:23.206626 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:23.208402 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:23.211088 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:23.214944 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:23.215252 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:23.215538 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:23.215744 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:23.307618 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.194:33335
I20260709 13:10:23.307885 27193 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.194:33335 every 8 connection(s)
I20260709 13:10:23.314913 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:23.328235 27198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:23.330760 27197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:23.330924 27200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:23.331643 27194 heartbeater.cc:344] Connected to a master server at 127.26.83.254:42853
I20260709 13:10:23.332301 27194 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:23.333225 27194 heartbeater.cc:507] Master 127.26.83.254:42853 requested a full tablet report, sending...
I20260709 13:10:23.333853 26959 server_base.cc:1061] running on GCE node
I20260709 13:10:23.334858 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:23.335105 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:23.335312 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602623335294 us; error 0 us; skew 500 ppm
I20260709 13:10:23.335924 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:23.335827 26991 ts_manager.cc:194] Registered new tserver with Master: 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:23.337822 26991 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59668
I20260709 13:10:23.339162 26959 webserver.cc:533] Webserver started at http://127.26.83.195:35053/ using document root <none> and password file <none>
I20260709 13:10:23.339789 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:23.340066 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:23.340443 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:23.341845 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-2-root/instance:
uuid: "78bbe9de354b47bb95b645fdc743bea9"
format_stamp: "Formatted at 2026-07-09 13:10:23 on dist-test-slave-329l"
I20260709 13:10:23.347152 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:10:23.351248 27205 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:23.352296 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:10:23.352689 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-2-root
uuid: "78bbe9de354b47bb95b645fdc743bea9"
format_stamp: "Formatted at 2026-07-09 13:10:23 on dist-test-slave-329l"
I20260709 13:10:23.353112 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602621086139-26959-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:23.372661 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:23.374109 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:23.377105 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:23.380676 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:23.380986 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:23.381346 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:23.381505 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:23.488850 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.195:35901
I20260709 13:10:23.488972 27267 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.195:35901 every 8 connection(s)
I20260709 13:10:23.508325 27268 heartbeater.cc:344] Connected to a master server at 127.26.83.254:42853
I20260709 13:10:23.508810 27268 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:23.509809 27268 heartbeater.cc:507] Master 127.26.83.254:42853 requested a full tablet report, sending...
I20260709 13:10:23.512082 26991 ts_manager.cc:194] Registered new tserver with Master: 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:23.513230 26959 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01871387s
I20260709 13:10:23.514189 26991 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59676
I20260709 13:10:23.570305 26991 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59688:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:23.574270 26991 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:23.690850 27233 tablet_service.cc:1511] Processing CreateTablet for tablet 57a3e45712a842e0b1d0fc7568486753 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:23.692703 27233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57a3e45712a842e0b1d0fc7568486753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.705034 27159 tablet_service.cc:1511] Processing CreateTablet for tablet 57a3e45712a842e0b1d0fc7568486753 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:23.706518 27159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57a3e45712a842e0b1d0fc7568486753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.706840 27231 tablet_service.cc:1511] Processing CreateTablet for tablet ee0994c9a9ad4420bc35c6f7a9e489c0 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:23.708379 27231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee0994c9a9ad4420bc35c6f7a9e489c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.710400 27232 tablet_service.cc:1511] Processing CreateTablet for tablet 5b7ee44bd2e643e5a64fbb8627620fc5 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:23.711884 27232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b7ee44bd2e643e5a64fbb8627620fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.714843 27230 tablet_service.cc:1511] Processing CreateTablet for tablet 8e1dadf7f86744d0b92a7ff1b03c896e (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:23.717748 27230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e1dadf7f86744d0b92a7ff1b03c896e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.718470 27158 tablet_service.cc:1511] Processing CreateTablet for tablet 5b7ee44bd2e643e5a64fbb8627620fc5 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:23.726820 27229 tablet_service.cc:1511] Processing CreateTablet for tablet 835fc7cd2a2641ab9269c1c172a07139 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:23.724900 27156 tablet_service.cc:1511] Processing CreateTablet for tablet 8e1dadf7f86744d0b92a7ff1b03c896e (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:23.740049 27158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b7ee44bd2e643e5a64fbb8627620fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.737020 27157 tablet_service.cc:1511] Processing CreateTablet for tablet ee0994c9a9ad4420bc35c6f7a9e489c0 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:23.741799 27154 tablet_service.cc:1511] Processing CreateTablet for tablet 625e7ef75e45444dbdfc17ba22057e53 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:23.736006 27228 tablet_service.cc:1511] Processing CreateTablet for tablet 625e7ef75e45444dbdfc17ba22057e53 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:23.750383 27229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 835fc7cd2a2641ab9269c1c172a07139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.741727 27155 tablet_service.cc:1511] Processing CreateTablet for tablet 835fc7cd2a2641ab9269c1c172a07139 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:23.765656 27080 tablet_service.cc:1511] Processing CreateTablet for tablet 835fc7cd2a2641ab9269c1c172a07139 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:10:23.767118 27080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 835fc7cd2a2641ab9269c1c172a07139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.763605 27156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e1dadf7f86744d0b92a7ff1b03c896e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.758536 27082 tablet_service.cc:1511] Processing CreateTablet for tablet ee0994c9a9ad4420bc35c6f7a9e489c0 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:10:23.762210 27084 tablet_service.cc:1511] Processing CreateTablet for tablet 57a3e45712a842e0b1d0fc7568486753 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:10:23.772909 27227 tablet_service.cc:1511] Processing CreateTablet for tablet 3155a7e09dc74a369b6d5694c395bea7 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:23.778007 27083 tablet_service.cc:1511] Processing CreateTablet for tablet 5b7ee44bd2e643e5a64fbb8627620fc5 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:10:23.777997 27152 tablet_service.cc:1511] Processing CreateTablet for tablet 889c1d5a24f7431f8b3008c5a61958e9 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:23.782307 27151 tablet_service.cc:1511] Processing CreateTablet for tablet d0c12f14d5944875ab26a08bc50e49b7 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:23.783481 27228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625e7ef75e45444dbdfc17ba22057e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.778921 27081 tablet_service.cc:1511] Processing CreateTablet for tablet 8e1dadf7f86744d0b92a7ff1b03c896e (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:10:23.781565 27226 tablet_service.cc:1511] Processing CreateTablet for tablet 889c1d5a24f7431f8b3008c5a61958e9 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:23.788089 27078 tablet_service.cc:1511] Processing CreateTablet for tablet 3155a7e09dc74a369b6d5694c395bea7 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:23.775038 27153 tablet_service.cc:1511] Processing CreateTablet for tablet 3155a7e09dc74a369b6d5694c395bea7 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:10:23.797683 27157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee0994c9a9ad4420bc35c6f7a9e489c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.782953 27225 tablet_service.cc:1511] Processing CreateTablet for tablet d0c12f14d5944875ab26a08bc50e49b7 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:23.801173 27077 tablet_service.cc:1511] Processing CreateTablet for tablet 889c1d5a24f7431f8b3008c5a61958e9 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:10:23.796800 27079 tablet_service.cc:1511] Processing CreateTablet for tablet 625e7ef75e45444dbdfc17ba22057e53 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:10:23.809247 27083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b7ee44bd2e643e5a64fbb8627620fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.810993 27077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 889c1d5a24f7431f8b3008c5a61958e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.813234 27079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625e7ef75e45444dbdfc17ba22057e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.813233 27227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3155a7e09dc74a369b6d5694c395bea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.818128 27076 tablet_service.cc:1511] Processing CreateTablet for tablet d0c12f14d5944875ab26a08bc50e49b7 (DEFAULT_TABLE table=test-workload [id=63cf404931df489abe8e420283f1d370]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:10:23.817574 27155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 835fc7cd2a2641ab9269c1c172a07139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.818625 27225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c12f14d5944875ab26a08bc50e49b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.824128 27076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c12f14d5944875ab26a08bc50e49b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.833250 27081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e1dadf7f86744d0b92a7ff1b03c896e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.836110 27226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 889c1d5a24f7431f8b3008c5a61958e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.836133 27078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3155a7e09dc74a369b6d5694c395bea7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:23.834765 27038 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:23.834194 27154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 625e7ef75e45444dbdfc17ba22057e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.868920 27153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3155a7e09dc74a369b6d5694c395bea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.879649 27084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57a3e45712a842e0b1d0fc7568486753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.879563 27151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c12f14d5944875ab26a08bc50e49b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.892982 27286 tablet_bootstrap.cc:492] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9: Bootstrap starting.
I20260709 13:10:23.898190 27082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee0994c9a9ad4420bc35c6f7a9e489c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:23.905658 27152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 889c1d5a24f7431f8b3008c5a61958e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:24.002671 27287 tablet_bootstrap.cc:492] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5: Bootstrap starting.
I20260709 13:10:24.018750 27286 tablet_bootstrap.cc:654] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.075129 27287 tablet_bootstrap.cc:654] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.100837 27288 tablet_bootstrap.cc:492] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c: Bootstrap starting.
I20260709 13:10:24.105222 27286 tablet_bootstrap.cc:492] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9: No bootstrap required, opened a new log
I20260709 13:10:24.105719 27286 ts_tablet_manager.cc:1403] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9: Time spent bootstrapping tablet: real 0.231s	user 0.016s	sys 0.011s
I20260709 13:10:24.110350 27286 raft_consensus.cc:359] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:24.111706 27286 raft_consensus.cc:385] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.115331 27286 raft_consensus.cc:740] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.111953 27287 tablet_bootstrap.cc:492] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5: No bootstrap required, opened a new log
I20260709 13:10:24.116129 27286 consensus_queue.cc:260] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:24.118191 27287 ts_tablet_manager.cc:1403] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5: Time spent bootstrapping tablet: real 0.120s	user 0.015s	sys 0.008s
I20260709 13:10:24.125808 27286 ts_tablet_manager.cc:1434] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.005s
I20260709 13:10:24.126716 27268 heartbeater.cc:499] Master 127.26.83.254:42853 was elected leader, sending a full tablet report...
I20260709 13:10:24.126684 27287 raft_consensus.cc:359] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:24.127419 27287 raft_consensus.cc:385] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.127687 27287 raft_consensus.cc:740] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.128348 27287 consensus_queue.cc:260] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:24.131449 27286 tablet_bootstrap.cc:492] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9: Bootstrap starting.
I20260709 13:10:24.131945 27288 tablet_bootstrap.cc:654] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.139585 27286 tablet_bootstrap.cc:654] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.145666 27194 heartbeater.cc:499] Master 127.26.83.254:42853 was elected leader, sending a full tablet report...
I20260709 13:10:24.146206 27287 ts_tablet_manager.cc:1434] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5: Time spent starting tablet: real 0.028s	user 0.003s	sys 0.008s
I20260709 13:10:24.147138 27287 tablet_bootstrap.cc:492] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5: Bootstrap starting.
I20260709 13:10:24.155704 27288 tablet_bootstrap.cc:492] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c: No bootstrap required, opened a new log
I20260709 13:10:24.156245 27288 ts_tablet_manager.cc:1403] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent bootstrapping tablet: real 0.094s	user 0.008s	sys 0.011s
I20260709 13:10:24.158115 27119 heartbeater.cc:499] Master 127.26.83.254:42853 was elected leader, sending a full tablet report...
I20260709 13:10:24.159420 27288 raft_consensus.cc:359] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.160246 27288 raft_consensus.cc:385] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.160578 27288 raft_consensus.cc:740] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.161348 27288 consensus_queue.cc:260] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.163856 27287 tablet_bootstrap.cc:654] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.168123 27286 tablet_bootstrap.cc:492] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9: No bootstrap required, opened a new log
I20260709 13:10:24.168630 27286 ts_tablet_manager.cc:1403] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9: Time spent bootstrapping tablet: real 0.037s	user 0.022s	sys 0.010s
I20260709 13:10:24.171584 27286 raft_consensus.cc:359] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.172147 27288 ts_tablet_manager.cc:1434] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260709 13:10:24.172428 27286 raft_consensus.cc:385] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.172792 27286 raft_consensus.cc:740] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.173295 27288 tablet_bootstrap.cc:492] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c: Bootstrap starting.
I20260709 13:10:24.173529 27286 consensus_queue.cc:260] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.175993 27286 ts_tablet_manager.cc:1434] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:10:24.177011 27286 tablet_bootstrap.cc:492] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9: Bootstrap starting.
I20260709 13:10:24.183821 27286 tablet_bootstrap.cc:654] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.184211 27287 tablet_bootstrap.cc:492] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5: No bootstrap required, opened a new log
I20260709 13:10:24.185549 27287 ts_tablet_manager.cc:1403] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5: Time spent bootstrapping tablet: real 0.039s	user 0.006s	sys 0.007s
I20260709 13:10:24.188771 27287 raft_consensus.cc:359] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.189581 27287 raft_consensus.cc:385] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.189904 27287 raft_consensus.cc:740] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.190723 27287 consensus_queue.cc:260] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.192953 27287 ts_tablet_manager.cc:1434] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:10:24.180709 27288 tablet_bootstrap.cc:654] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.197324 27287 tablet_bootstrap.cc:492] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5: Bootstrap starting.
I20260709 13:10:24.209659 27287 tablet_bootstrap.cc:654] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.239018 27286 tablet_bootstrap.cc:492] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9: No bootstrap required, opened a new log
I20260709 13:10:24.239524 27286 ts_tablet_manager.cc:1403] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.017s
I20260709 13:10:24.239684 27288 tablet_bootstrap.cc:492] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c: No bootstrap required, opened a new log
I20260709 13:10:24.240201 27288 ts_tablet_manager.cc:1403] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent bootstrapping tablet: real 0.067s	user 0.025s	sys 0.019s
I20260709 13:10:24.240682 27287 tablet_bootstrap.cc:492] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5: No bootstrap required, opened a new log
I20260709 13:10:24.241240 27287 ts_tablet_manager.cc:1403] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.019s
I20260709 13:10:24.242622 27286 raft_consensus.cc:359] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.243392 27286 raft_consensus.cc:385] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.243191 27288 raft_consensus.cc:359] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.243753 27286 raft_consensus.cc:740] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.243965 27288 raft_consensus.cc:385] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.244458 27288 raft_consensus.cc:740] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.244415 27287 raft_consensus.cc:359] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.244546 27286 consensus_queue.cc:260] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.245544 27287 raft_consensus.cc:385] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.245322 27288 consensus_queue.cc:260] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.247489 27286 ts_tablet_manager.cc:1434] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
W20260709 13:10:24.247725 27269 tablet.cc:2406] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:10:24.248514 27286 tablet_bootstrap.cc:492] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9: Bootstrap starting.
I20260709 13:10:24.249679 27288 ts_tablet_manager.cc:1434] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:10:24.250598 27288 tablet_bootstrap.cc:492] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c: Bootstrap starting.
I20260709 13:10:24.256232 27286 tablet_bootstrap.cc:654] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.257517 27288 tablet_bootstrap.cc:654] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.267946 27286 tablet_bootstrap.cc:492] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9: No bootstrap required, opened a new log
I20260709 13:10:24.268486 27286 ts_tablet_manager.cc:1403] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.007s
I20260709 13:10:24.271973 27287 raft_consensus.cc:740] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.273021 27286 raft_consensus.cc:359] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.273036 27287 consensus_queue.cc:260] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.274802 27288 tablet_bootstrap.cc:492] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c: No bootstrap required, opened a new log
I20260709 13:10:24.276721 27287 ts_tablet_manager.cc:1434] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.002s
I20260709 13:10:24.276839 27288 ts_tablet_manager.cc:1403] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.009s
I20260709 13:10:24.274646 27286 raft_consensus.cc:385] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.277576 27286 raft_consensus.cc:740] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.278352 27286 consensus_queue.cc:260] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.280891 27286 ts_tablet_manager.cc:1434] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:10:24.281888 27286 tablet_bootstrap.cc:492] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9: Bootstrap starting.
I20260709 13:10:24.282594 27288 raft_consensus.cc:359] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.283334 27288 raft_consensus.cc:385] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.283627 27288 raft_consensus.cc:740] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.284627 27287 tablet_bootstrap.cc:492] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5: Bootstrap starting.
I20260709 13:10:24.284389 27288 consensus_queue.cc:260] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.286911 27288 ts_tablet_manager.cc:1434] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:10:24.292445 27287 tablet_bootstrap.cc:654] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.292780 27288 tablet_bootstrap.cc:492] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c: Bootstrap starting.
I20260709 13:10:24.298871 27286 tablet_bootstrap.cc:654] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.309293 27288 tablet_bootstrap.cc:654] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.316121 27287 tablet_bootstrap.cc:492] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5: No bootstrap required, opened a new log
I20260709 13:10:24.316628 27287 ts_tablet_manager.cc:1403] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.013s
I20260709 13:10:24.316699 27286 tablet_bootstrap.cc:492] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9: No bootstrap required, opened a new log
I20260709 13:10:24.317255 27286 ts_tablet_manager.cc:1403] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.005s
I20260709 13:10:24.318595 27288 tablet_bootstrap.cc:492] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c: No bootstrap required, opened a new log
I20260709 13:10:24.319191 27288 ts_tablet_manager.cc:1403] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.003s
I20260709 13:10:24.319792 27287 raft_consensus.cc:359] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.320547 27287 raft_consensus.cc:385] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.320375 27286 raft_consensus.cc:359] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.320927 27287 raft_consensus.cc:740] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.321729 27287 consensus_queue.cc:260] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.322273 27288 raft_consensus.cc:359] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.322988 27288 raft_consensus.cc:385] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.323410 27288 raft_consensus.cc:740] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.323491 27295 raft_consensus.cc:493] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.324185 27288 consensus_queue.cc:260] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.324045 27295 raft_consensus.cc:515] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.326265 27287 ts_tablet_manager.cc:1434] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:10:24.327736 27287 tablet_bootstrap.cc:492] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5: Bootstrap starting.
I20260709 13:10:24.327623 27295 leader_election.cc:290] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.330343 27288 ts_tablet_manager.cc:1434] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:10:24.331485 27288 tablet_bootstrap.cc:492] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c: Bootstrap starting.
I20260709 13:10:24.334889 27287 tablet_bootstrap.cc:654] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.338383 27288 tablet_bootstrap.cc:654] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.344805 27286 raft_consensus.cc:385] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.345485 27286 raft_consensus.cc:740] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.346737 27286 consensus_queue.cc:260] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.349550 27286 ts_tablet_manager.cc:1434] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.002s
I20260709 13:10:24.352854 27286 tablet_bootstrap.cc:492] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9: Bootstrap starting.
I20260709 13:10:24.362437 27286 tablet_bootstrap.cc:654] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.368579 27288 tablet_bootstrap.cc:492] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c: No bootstrap required, opened a new log
I20260709 13:10:24.369127 27288 ts_tablet_manager.cc:1403] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.000s
I20260709 13:10:24.373317 27288 raft_consensus.cc:359] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.374169 27287 tablet_bootstrap.cc:492] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5: No bootstrap required, opened a new log
I20260709 13:10:24.374354 27288 raft_consensus.cc:385] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.374729 27287 ts_tablet_manager.cc:1403] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.010s
I20260709 13:10:24.374841 27288 raft_consensus.cc:740] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.376160 27288 consensus_queue.cc:260] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.379499 27287 raft_consensus.cc:359] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.380262 27287 raft_consensus.cc:385] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.381924 27288 ts_tablet_manager.cc:1434] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:10:24.382989 27288 tablet_bootstrap.cc:492] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c: Bootstrap starting.
I20260709 13:10:24.385495 27295 raft_consensus.cc:493] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.385967 27295 raft_consensus.cc:515] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.380553 27287 raft_consensus.cc:740] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.388199 27295 leader_election.cc:290] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.389792 27287 consensus_queue.cc:260] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.394820 27287 ts_tablet_manager.cc:1434] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:10:24.395908 27287 tablet_bootstrap.cc:492] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5: Bootstrap starting.
I20260709 13:10:24.398484 27168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5" is_pre_election: true
I20260709 13:10:24.399355 27168 raft_consensus.cc:2468] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 0.
I20260709 13:10:24.400875 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5" is_pre_election: true
I20260709 13:10:24.401609 27169 raft_consensus.cc:2468] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 0.
I20260709 13:10:24.402535 27292 raft_consensus.cc:493] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.403059 27292 raft_consensus.cc:515] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.406502 27292 leader_election.cc:290] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.407521 27059 leader_election.cc:304] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:24.408399 27295 raft_consensus.cc:2804] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.408787 27295 raft_consensus.cc:493] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.409140 27295 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.413599 27292 raft_consensus.cc:493] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.414139 27292 raft_consensus.cc:515] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:24.416231 27292 leader_election.cc:290] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901), af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779)
I20260709 13:10:24.420678 27295 raft_consensus.cc:515] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.394901 27288 tablet_bootstrap.cc:654] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.421278 27059 leader_election.cc:304] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:24.430997 27295 leader_election.cc:290] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 election: Requested vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.431648 27295 raft_consensus.cc:2804] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.432036 27295 raft_consensus.cc:493] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.432085 27288 tablet_bootstrap.cc:492] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c: No bootstrap required, opened a new log
I20260709 13:10:24.432914 27288 ts_tablet_manager.cc:1403] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.004s
I20260709 13:10:24.434484 27291 raft_consensus.cc:493] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.435101 27291 raft_consensus.cc:515] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.436383 27288 raft_consensus.cc:359] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.437186 27288 raft_consensus.cc:385] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.437511 27288 raft_consensus.cc:740] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.438035 27291 leader_election.cc:290] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:24.438225 27288 consensus_queue.cc:260] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.441183 27288 ts_tablet_manager.cc:1434] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:24.442368 27288 tablet_bootstrap.cc:492] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c: Bootstrap starting.
I20260709 13:10:24.444805 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:24.445634 27169 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.449753 27288 tablet_bootstrap.cc:654] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.456316 27169 raft_consensus.cc:2468] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 1.
I20260709 13:10:24.458496 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
I20260709 13:10:24.432525 27295 raft_consensus.cc:3060] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.461254 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
I20260709 13:10:24.465334 27242 raft_consensus.cc:2468] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 0.
I20260709 13:10:24.462371 27241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:24.463573 27288 tablet_bootstrap.cc:492] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c: No bootstrap required, opened a new log
I20260709 13:10:24.466361 27241 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.466750 27288 ts_tablet_manager.cc:1403] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.005s
I20260709 13:10:24.468827 27292 raft_consensus.cc:493] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.473162 27292 raft_consensus.cc:515] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.473757 27288 raft_consensus.cc:359] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.474749 27288 raft_consensus.cc:385] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.475165 27288 raft_consensus.cc:740] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.475281 27292 leader_election.cc:290] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
W20260709 13:10:24.466845 27059 leader_election.cc:343] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.476172 27241 raft_consensus.cc:2468] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 1.
I20260709 13:10:24.478741 27059 leader_election.cc:304] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:24.476193 27288 consensus_queue.cc:260] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.480878 27287 tablet_bootstrap.cc:654] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.499992 27286 tablet_bootstrap.cc:492] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9: No bootstrap required, opened a new log
I20260709 13:10:24.493240 27316 raft_consensus.cc:2804] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.500572 27286 ts_tablet_manager.cc:1403] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9: Time spent bootstrapping tablet: real 0.148s	user 0.034s	sys 0.040s
I20260709 13:10:24.515129 27241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
I20260709 13:10:24.515679 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
I20260709 13:10:24.516369 27243 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 0.
I20260709 13:10:24.516220 27241 raft_consensus.cc:2468] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 0.
I20260709 13:10:24.517158 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
I20260709 13:10:24.522184 27242 raft_consensus.cc:2468] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 0.
I20260709 13:10:24.519537 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" is_pre_election: true
I20260709 13:10:24.524612 27134 leader_election.cc:304] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: 
I20260709 13:10:24.546223 27092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" is_pre_election: true
I20260709 13:10:24.550973 27092 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 0.
I20260709 13:10:24.551206 27288 ts_tablet_manager.cc:1434] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent starting tablet: real 0.084s	user 0.015s	sys 0.023s
I20260709 13:10:24.552860 27292 raft_consensus.cc:2804] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.553303 27292 raft_consensus.cc:493] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.553402 27134 leader_election.cc:304] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: 
I20260709 13:10:24.553658 27292 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.552791 27288 tablet_bootstrap.cc:492] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c: Bootstrap starting.
I20260709 13:10:24.552281 27295 raft_consensus.cc:515] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.562661 27286 raft_consensus.cc:359] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.563680 27286 raft_consensus.cc:385] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.564062 27286 raft_consensus.cc:740] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.564158 27292 raft_consensus.cc:515] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.547320 27093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" is_pre_election: true
I20260709 13:10:24.551815 27094 raft_consensus.cc:2468] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 0.
I20260709 13:10:24.578883 27295 leader_election.cc:290] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 election: Requested vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.579538 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:24.580150 27169 raft_consensus.cc:3060] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.565747 27286 consensus_queue.cc:260] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.589408 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:24.597483 27321 raft_consensus.cc:493] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.597916 27242 raft_consensus.cc:3060] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.591862 27288 tablet_bootstrap.cc:654] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.598083 27321 raft_consensus.cc:515] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.600191 27316 raft_consensus.cc:697] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Becoming Leader. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:24.590441 27287 tablet_bootstrap.cc:492] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5: No bootstrap required, opened a new log
I20260709 13:10:24.603183 27322 raft_consensus.cc:493] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.604293 27322 raft_consensus.cc:515] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.602706 27287 ts_tablet_manager.cc:1403] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5: Time spent bootstrapping tablet: real 0.207s	user 0.018s	sys 0.017s
I20260709 13:10:24.611985 27169 raft_consensus.cc:2468] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 1.
I20260709 13:10:24.620574 27059 leader_election.cc:304] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:24.618875 27286 ts_tablet_manager.cc:1434] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9: Time spent starting tablet: real 0.118s	user 0.008s	sys 0.009s
I20260709 13:10:24.622206 27326 raft_consensus.cc:2804] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.623203 27326 raft_consensus.cc:697] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Becoming Leader. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:24.624115 27326 consensus_queue.cc:237] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.624199 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" is_pre_election: true
I20260709 13:10:24.623181 27286 tablet_bootstrap.cc:492] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9: Bootstrap starting.
I20260709 13:10:24.631318 27242 raft_consensus.cc:2468] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 1.
I20260709 13:10:24.629817 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5" is_pre_election: true
I20260709 13:10:24.632221 27321 leader_election.cc:290] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.634625 27168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3155a7e09dc74a369b6d5694c395bea7" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5" is_pre_election: true
I20260709 13:10:24.624668 27292 leader_election.cc:290] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.622641 27316 consensus_queue.cc:237] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.636752 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3155a7e09dc74a369b6d5694c395bea7" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
W20260709 13:10:24.695401 27134 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:35901, user_credentials={real_user=slave}} blocked reactor thread for 142046us
I20260709 13:10:24.637224 27241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
I20260709 13:10:24.717952 27327 raft_consensus.cc:2804] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.735780 27241 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 0.
I20260709 13:10:24.735570 27134 leader_election.cc:304] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: 
W20260709 13:10:24.709744 27208 leader_election.cc:343] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.735983 27327 raft_consensus.cc:493] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.703900 27286 tablet_bootstrap.cc:654] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.736835 27327 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.628983 27322 leader_election.cc:290] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.641824 27169 raft_consensus.cc:2393] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate af57968de4ac40e9ab7b6cf21ea18d7c in current term 1: Already voted for candidate 23b4242091514cf4887d6fdf87717de5 in this term.
I20260709 13:10:24.740319 27292 raft_consensus.cc:2804] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:10:24.740736 27208 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:44779, user_credentials={real_user=slave}} blocked reactor thread for 95144.1us
I20260709 13:10:24.740847 27292 raft_consensus.cc:493] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.741396 27292 raft_consensus.cc:3060] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.694895 27321 raft_consensus.cc:493] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.742810 27321 raft_consensus.cc:515] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.743768 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:24.744123 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5" is_pre_election: true
I20260709 13:10:24.744349 27094 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.698680 27287 raft_consensus.cc:359] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.745452 27287 raft_consensus.cc:385] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.745759 27287 raft_consensus.cc:740] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.746258 27286 tablet_bootstrap.cc:492] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9: No bootstrap required, opened a new log
I20260709 13:10:24.746600 27286 ts_tablet_manager.cc:1403] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9: Time spent bootstrapping tablet: real 0.124s	user 0.006s	sys 0.012s
I20260709 13:10:24.746477 27287 consensus_queue.cc:260] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.747587 27169 raft_consensus.cc:2468] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 0.
W20260709 13:10:24.641938 27059 leader_election.cc:343] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:24.752159 27059 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:33335, user_credentials={real_user=slave}} blocked reactor thread for 110258us
I20260709 13:10:24.755213 27321 leader_election.cc:290] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.751195 27286 raft_consensus.cc:359] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.755970 27286 raft_consensus.cc:385] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.756336 27286 raft_consensus.cc:740] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.758260 27288 tablet_bootstrap.cc:492] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c: No bootstrap required, opened a new log
I20260709 13:10:24.758730 27288 ts_tablet_manager.cc:1403] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent bootstrapping tablet: real 0.206s	user 0.010s	sys 0.016s
I20260709 13:10:24.759958 27094 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 1.
I20260709 13:10:24.760330 27292 raft_consensus.cc:515] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:24.761471 27168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c12f14d5944875ab26a08bc50e49b7" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5" is_pre_election: true
I20260709 13:10:24.761257 26991 catalog_manager.cc:5697] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193). New cstate: current_term: 1 leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.761746 27288 raft_consensus.cc:359] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:24.762162 27241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c12f14d5944875ab26a08bc50e49b7" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
I20260709 13:10:24.769479 27288 raft_consensus.cc:385] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.769896 27288 raft_consensus.cc:740] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.759382 27327 raft_consensus.cc:515] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.763446 27208 leader_election.cc:304] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: af57968de4ac40e9ab7b6cf21ea18d7c
I20260709 13:10:24.770819 27288 consensus_queue.cc:260] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:24.771371 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
W20260709 13:10:24.773047 27059 leader_election.cc:343] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.776057 27287 ts_tablet_manager.cc:1434] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5: Time spent starting tablet: real 0.163s	user 0.011s	sys 0.011s
I20260709 13:10:24.776610 27288 ts_tablet_manager.cc:1434] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.004s
I20260709 13:10:24.773626 27291 raft_consensus.cc:2804] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.776996 27287 tablet_bootstrap.cc:492] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5: Bootstrap starting.
I20260709 13:10:24.777110 27291 raft_consensus.cc:493] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.777529 27288 tablet_bootstrap.cc:492] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c: Bootstrap starting.
I20260709 13:10:24.777494 27291 raft_consensus.cc:3060] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.773379 27243 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.779230 27059 leader_election.cc:304] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9
W20260709 13:10:24.773315 27134 leader_election.cc:343] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.763783 27286 consensus_queue.cc:260] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.783232 27134 leader_election.cc:304] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:24.785131 27321 raft_consensus.cc:2749] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:24.785161 27332 raft_consensus.cc:2804] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.785563 27059 leader_election.cc:304] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:24.786273 27241 raft_consensus.cc:2468] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 0.
I20260709 13:10:24.786476 26991 catalog_manager.cc:5697] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193). New cstate: current_term: 1 leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.787793 27292 leader_election.cc:290] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 election: Requested vote from peers 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901), af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779)
I20260709 13:10:24.788899 27321 raft_consensus.cc:2764] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:10:24.789536 27241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:24.790177 27241 raft_consensus.cc:3060] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:24.794566 27059 leader_election.cc:343] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.795989 27321 raft_consensus.cc:493] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.797606 27059 leader_election.cc:304] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 23b4242091514cf4887d6fdf87717de5
I20260709 13:10:24.796914 27321 raft_consensus.cc:515] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.798892 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:24.800803 27094 raft_consensus.cc:3060] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.802018 27321 leader_election.cc:290] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.803929 27321 raft_consensus.cc:2804] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:24.804101 27168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5" is_pre_election: true
I20260709 13:10:24.804349 27321 raft_consensus.cc:493] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.804510 27287 tablet_bootstrap.cc:654] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.804842 27321 raft_consensus.cc:3060] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.811635 27094 raft_consensus.cc:2468] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 1.
I20260709 13:10:24.813166 27134 leader_election.cc:304] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:24.814358 27292 raft_consensus.cc:2804] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.814965 27292 raft_consensus.cc:697] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:24.815976 27241 raft_consensus.cc:2468] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 1.
I20260709 13:10:24.817094 27332 raft_consensus.cc:697] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:24.821099 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
I20260709 13:10:24.820092 27292 consensus_queue.cc:237] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
W20260709 13:10:24.822741 27059 leader_election.cc:343] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.821408 27286 ts_tablet_manager.cc:1434] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9: Time spent starting tablet: real 0.075s	user 0.015s	sys 0.000s
I20260709 13:10:24.821668 27243 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 1.
I20260709 13:10:24.821764 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
W20260709 13:10:24.824638 27059 leader_election.cc:343] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.826022 27094 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.826161 27059 leader_election.cc:304] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9
I20260709 13:10:24.828953 27316 raft_consensus.cc:2749] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:24.824628 27291 raft_consensus.cc:515] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.831110 27288 tablet_bootstrap.cc:654] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.831362 27327 leader_election.cc:290] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.835166 27286 tablet_bootstrap.cc:492] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9: Bootstrap starting.
I20260709 13:10:24.831935 27332 consensus_queue.cc:237] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.840050 27093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:24.837284 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:24.840443 27168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:24.841706 27168 raft_consensus.cc:3060] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.840520 27291 leader_election.cc:290] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 1 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:24.842665 27243 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:24.842255 27208 leader_election.cc:343] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:24.855453 27094 raft_consensus.cc:2468] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 1.
I20260709 13:10:24.852982 27287 tablet_bootstrap.cc:492] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5: No bootstrap required, opened a new log
I20260709 13:10:24.858647 27168 raft_consensus.cc:2468] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 1.
I20260709 13:10:24.859820 27134 leader_election.cc:304] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:24.860773 27292 raft_consensus.cc:2804] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.862018 27208 leader_election.cc:304] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: af57968de4ac40e9ab7b6cf21ea18d7c
I20260709 13:10:24.859692 27287 ts_tablet_manager.cc:1403] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5: Time spent bootstrapping tablet: real 0.083s	user 0.008s	sys 0.012s
I20260709 13:10:24.862202 27292 raft_consensus.cc:697] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:24.864238 27291 raft_consensus.cc:2804] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.863209 27321 raft_consensus.cc:515] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.868057 27288 tablet_bootstrap.cc:492] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c: No bootstrap required, opened a new log
I20260709 13:10:24.868386 27288 ts_tablet_manager.cc:1403] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent bootstrapping tablet: real 0.091s	user 0.015s	sys 0.008s
I20260709 13:10:24.867225 27291 raft_consensus.cc:697] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 LEADER]: Becoming Leader. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:24.868670 27292 consensus_queue.cc:237] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.870091 27243 raft_consensus.cc:2468] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 1.
I20260709 13:10:24.870151 27291 consensus_queue.cc:237] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.878319 27288 raft_consensus.cc:359] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.880010 27288 raft_consensus.cc:385] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.880502 27288 raft_consensus.cc:740] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.880637 27168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c12f14d5944875ab26a08bc50e49b7" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:24.881527 27288 consensus_queue.cc:260] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.881865 27321 leader_election.cc:290] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 election: Requested vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:24.879061 27286 tablet_bootstrap.cc:654] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.882138 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c12f14d5944875ab26a08bc50e49b7" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:24.883111 27243 raft_consensus.cc:3060] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.890290 27287 raft_consensus.cc:359] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.893117 27287 raft_consensus.cc:385] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.895946 27288 ts_tablet_manager.cc:1434] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 13:10:24.896531 27287 raft_consensus.cc:740] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.903847 27287 consensus_queue.cc:260] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.908159 27287 ts_tablet_manager.cc:1434] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.004s
I20260709 13:10:24.916388 27243 raft_consensus.cc:2468] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 1.
I20260709 13:10:24.917829 27059 leader_election.cc:304] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:24.918645 27321 raft_consensus.cc:2804] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:24.919237 27321 raft_consensus.cc:697] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Becoming Leader. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:24.920105 27321 consensus_queue.cc:237] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.921244 27287 tablet_bootstrap.cc:492] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5: Bootstrap starting.
I20260709 13:10:24.927592 27168 raft_consensus.cc:3060] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:24.932044 27287 tablet_bootstrap.cc:654] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.932677 27286 tablet_bootstrap.cc:492] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9: No bootstrap required, opened a new log
I20260709 13:10:24.935887 27286 ts_tablet_manager.cc:1403] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9: Time spent bootstrapping tablet: real 0.102s	user 0.016s	sys 0.007s
I20260709 13:10:24.938875 27286 raft_consensus.cc:359] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.939708 27286 raft_consensus.cc:385] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.940114 27286 raft_consensus.cc:740] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.940866 27286 consensus_queue.cc:260] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.943423 27286 ts_tablet_manager.cc:1434] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:24.944463 27286 tablet_bootstrap.cc:492] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9: Bootstrap starting.
W20260709 13:10:24.946557 27038 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:24.957085 27168 raft_consensus.cc:2468] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 1.
I20260709 13:10:24.957798 26988 catalog_manager.cc:5697] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195). New cstate: current_term: 1 leader_uuid: "78bbe9de354b47bb95b645fdc743bea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:24.951153 27286 tablet_bootstrap.cc:654] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.953303 27287 tablet_bootstrap.cc:492] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5: No bootstrap required, opened a new log
I20260709 13:10:24.961285 27287 ts_tablet_manager.cc:1403] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5: Time spent bootstrapping tablet: real 0.040s	user 0.006s	sys 0.006s
I20260709 13:10:24.963951 27287 raft_consensus.cc:359] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.964699 27287 raft_consensus.cc:385] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.964998 27287 raft_consensus.cc:740] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.965756 27287 consensus_queue.cc:260] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.967751 27287 ts_tablet_manager.cc:1434] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:10:24.968847 27287 tablet_bootstrap.cc:492] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5: Bootstrap starting.
I20260709 13:10:24.985970 26989 catalog_manager.cc:5697] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.987354 27287 tablet_bootstrap.cc:654] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:24.988004 26989 catalog_manager.cc:5697] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.992769 27286 tablet_bootstrap.cc:492] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9: No bootstrap required, opened a new log
I20260709 13:10:24.993218 27286 ts_tablet_manager.cc:1403] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9: Time spent bootstrapping tablet: real 0.049s	user 0.008s	sys 0.007s
I20260709 13:10:24.989889 26989 catalog_manager.cc:5697] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:24.995362 27286 raft_consensus.cc:359] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:24.996129 27286 raft_consensus.cc:385] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:24.996382 27286 raft_consensus.cc:740] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Initialized, Role: FOLLOWER
I20260709 13:10:24.997311 27286 consensus_queue.cc:260] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:25.002050 27287 tablet_bootstrap.cc:492] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5: No bootstrap required, opened a new log
I20260709 13:10:25.002624 27287 ts_tablet_manager.cc:1403] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.004s
I20260709 13:10:25.003329 27286 ts_tablet_manager.cc:1434] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:10:25.005263 27287 raft_consensus.cc:359] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:25.006055 27287 raft_consensus.cc:385] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:25.006421 27287 raft_consensus.cc:740] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Initialized, Role: FOLLOWER
I20260709 13:10:25.007164 27287 consensus_queue.cc:260] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:25.007331 26991 catalog_manager.cc:5697] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c reported cstate change: term changed from 0 to 1, leader changed from <none> to af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193). New cstate: current_term: 1 leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.009838 27287 ts_tablet_manager.cc:1434] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:25.028052 27321 consensus_queue.cc:1048] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.045660 27321 consensus_queue.cc:1048] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.057050 27292 raft_consensus.cc:493] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.057582 27292 raft_consensus.cc:515] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:25.059938 27292 leader_election.cc:290] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:25.061299 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3155a7e09dc74a369b6d5694c395bea7" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" is_pre_election: true
I20260709 13:10:25.062119 27094 raft_consensus.cc:2468] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 0.
I20260709 13:10:25.063333 27134 leader_election.cc:304] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:25.064229 27292 raft_consensus.cc:2804] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.064589 27292 raft_consensus.cc:493] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.064967 27292 raft_consensus.cc:3060] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.065279 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3155a7e09dc74a369b6d5694c395bea7" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
I20260709 13:10:25.066092 27243 raft_consensus.cc:2468] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 0.
I20260709 13:10:25.071766 27292 raft_consensus.cc:515] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:25.074100 27292 leader_election.cc:290] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:25.075084 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3155a7e09dc74a369b6d5694c395bea7" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:25.075296 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3155a7e09dc74a369b6d5694c395bea7" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:25.075627 27094 raft_consensus.cc:3060] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.075865 27243 raft_consensus.cc:3060] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.080236 27094 raft_consensus.cc:2468] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 1.
I20260709 13:10:25.081452 27134 leader_election.cc:304] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:25.082185 27292 raft_consensus.cc:2804] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:25.082680 27292 raft_consensus.cc:697] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:25.083652 27243 raft_consensus.cc:2468] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 1.
I20260709 13:10:25.083490 27292 consensus_queue.cc:237] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:25.091809 26991 catalog_manager.cc:5697] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.140281 27316 consensus_queue.cc:1048] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.154074 27322 consensus_queue.cc:1048] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:25.291008 27292 consensus_queue.cc:1048] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.302974 27327 consensus_queue.cc:1048] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:25.321645 27322 raft_consensus.cc:493] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.322146 27322 raft_consensus.cc:515] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:25.324427 27322 leader_election.cc:290] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:25.325202 27168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5" is_pre_election: true
I20260709 13:10:25.325582 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bbe9de354b47bb95b645fdc743bea9" is_pre_election: true
I20260709 13:10:25.326116 27168 raft_consensus.cc:2393] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate af57968de4ac40e9ab7b6cf21ea18d7c in current term 1: Already voted for candidate 78bbe9de354b47bb95b645fdc743bea9 in this term.
I20260709 13:10:25.327402 27059 leader_election.cc:304] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9
I20260709 13:10:25.328315 27322 raft_consensus.cc:3060] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.333496 27322 raft_consensus.cc:2749] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:25.343825 27322 consensus_queue.cc:1048] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.345409 27326 consensus_queue.cc:1048] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.413228 27327 consensus_queue.cc:1048] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.417552 27291 raft_consensus.cc:493] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.418156 27291 raft_consensus.cc:515] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:25.420295 27291 leader_election.cc:290] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:25.422168 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5" is_pre_election: true
I20260709 13:10:25.422824 27169 raft_consensus.cc:2468] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 0.
I20260709 13:10:25.424155 27208 leader_election.cc:304] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: 
I20260709 13:10:25.424540 27093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" is_pre_election: true
I20260709 13:10:25.425097 27291 raft_consensus.cc:2804] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:25.425282 27093 raft_consensus.cc:2468] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 0.
I20260709 13:10:25.425478 27291 raft_consensus.cc:493] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.425868 27291 raft_consensus.cc:3060] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.432420 27327 consensus_queue.cc:1048] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.433550 27291 raft_consensus.cc:515] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:25.435696 27291 leader_election.cc:290] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 1 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:25.436798 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:25.436908 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:25.437491 27169 raft_consensus.cc:3060] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.437570 27094 raft_consensus.cc:3060] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:25.444898 27169 raft_consensus.cc:2468] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 1.
I20260709 13:10:25.446221 27208 leader_election.cc:304] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: 
I20260709 13:10:25.451323 27291 raft_consensus.cc:2804] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:25.453356 27291 raft_consensus.cc:697] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 LEADER]: Becoming Leader. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:25.454309 27094 raft_consensus.cc:2468] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 1.
I20260709 13:10:25.455291 27291 consensus_queue.cc:237] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:25.455902 27327 consensus_queue.cc:1048] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.459359 27365 consensus_queue.cc:1048] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.466005 27334 consensus_queue.cc:1048] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.490427 27332 consensus_queue.cc:1048] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.517820 27363 consensus_queue.cc:1048] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.554737 27334 consensus_queue.cc:1048] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:25.568346 26991 catalog_manager.cc:5697] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195). New cstate: current_term: 1 leader_uuid: "78bbe9de354b47bb95b645fdc743bea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.881517 27291 consensus_queue.cc:1048] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:25.894337 27291 consensus_queue.cc:1048] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:25.980073 27169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
mode: GRACEFUL
new_leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:37948
I20260709 13:10:25.980801 27169 raft_consensus.cc:606] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Received request to transfer leadership to TS af57968de4ac40e9ab7b6cf21ea18d7c
I20260709 13:10:26.000154 27243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0"
dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
mode: GRACEFUL
new_leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:35866
I20260709 13:10:26.000905 27243 raft_consensus.cc:606] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 LEADER]: Received request to transfer leadership to TS af57968de4ac40e9ab7b6cf21ea18d7c
I20260709 13:10:26.006703 27169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
mode: GRACEFUL
new_leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:37948
I20260709 13:10:26.007814 27169 raft_consensus.cc:606] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Received request to transfer leadership to TS af57968de4ac40e9ab7b6cf21ea18d7c
I20260709 13:10:26.017969 27243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53"
dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
mode: GRACEFUL
new_leader_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:35866
I20260709 13:10:26.018693 27243 raft_consensus.cc:606] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 LEADER]: Received request to transfer leadership to TS 23b4242091514cf4887d6fdf87717de5
I20260709 13:10:26.038120 27094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
mode: GRACEFUL
new_leader_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:39140
I20260709 13:10:26.038718 27094 raft_consensus.cc:606] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Received request to transfer leadership to TS 23b4242091514cf4887d6fdf87717de5
I20260709 13:10:26.045315 27094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0c12f14d5944875ab26a08bc50e49b7"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
mode: GRACEFUL
new_leader_uuid: "78bbe9de354b47bb95b645fdc743bea9"
 from {username='slave'} at 127.0.0.1:39140
I20260709 13:10:26.045902 27094 raft_consensus.cc:606] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Received request to transfer leadership to TS 78bbe9de354b47bb95b645fdc743bea9
I20260709 13:10:26.053870 27363 raft_consensus.cc:993] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9: : Instructing follower af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:26.054406 27363 raft_consensus.cc:1081] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 LEADER]: Signalling peer af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:26.056313 27094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:39132
I20260709 13:10:26.057250 27094 raft_consensus.cc:493] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:26.057821 27094 raft_consensus.cc:3060] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.068223 27334 raft_consensus.cc:993] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5: : Instructing follower af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:26.068897 27332 raft_consensus.cc:1081] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Signalling peer af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:26.068888 27094 raft_consensus.cc:515] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.070739 27092 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:39124
I20260709 13:10:26.071337 27092 raft_consensus.cc:493] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:26.071332 27094 leader_election.cc:290] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 2 election: Requested vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:26.071931 27092 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.072180 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:26.072849 27169 raft_consensus.cc:3060] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.074136 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:26.074770 27243 raft_consensus.cc:3055] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:26.075222 27243 raft_consensus.cc:740] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:26.076395 27243 consensus_queue.cc:260] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.079890 27243 raft_consensus.cc:3060] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.085671 27169 raft_consensus.cc:2468] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 2.
I20260709 13:10:26.086390 27092 raft_consensus.cc:515] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.087136 27059 leader_election.cc:304] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:26.087628 27243 raft_consensus.cc:2468] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 2.
I20260709 13:10:26.088028 27321 raft_consensus.cc:2804] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:26.088526 27321 raft_consensus.cc:697] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Becoming Leader. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:26.089360 27321 consensus_queue.cc:237] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.089741 27092 leader_election.cc:290] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 2 election: Requested vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:26.090958 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:26.091574 27243 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.093178 27168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:26.093755 27168 raft_consensus.cc:3055] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:26.094059 27168 raft_consensus.cc:740] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:26.094812 27168 consensus_queue.cc:260] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.095955 27168 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.099030 27243 raft_consensus.cc:2468] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 2.
I20260709 13:10:26.100453 27059 leader_election.cc:304] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:26.101475 27326 raft_consensus.cc:2804] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:26.101923 27326 raft_consensus.cc:697] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Becoming Leader. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:26.102670 27326 consensus_queue.cc:237] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.103288 27168 raft_consensus.cc:2468] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 2.
I20260709 13:10:26.102849 26990 catalog_manager.cc:5697] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c reported cstate change: term changed from 1 to 2, leader changed from 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195) to af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193). New cstate: current_term: 2 leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.120949 26990 catalog_manager.cc:5697] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c reported cstate change: term changed from 1 to 2, leader changed from 23b4242091514cf4887d6fdf87717de5 (127.26.83.194) to af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193). New cstate: current_term: 2 leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.252628 27332 raft_consensus.cc:993] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5: : Instructing follower af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:26.253158 27334 raft_consensus.cc:1081] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Signalling peer af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:26.254724 27092 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:39124
I20260709 13:10:26.255448 27092 raft_consensus.cc:493] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:26.255774 27092 raft_consensus.cc:3060] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.260828 27092 raft_consensus.cc:515] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:26.262629 27092 leader_election.cc:290] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 2 election: Requested vote from peers 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901), 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:26.263672 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:26.263953 27168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:26.264210 27243 raft_consensus.cc:3060] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.264575 27168 raft_consensus.cc:3055] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:26.264902 27168 raft_consensus.cc:740] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:26.265578 27168 consensus_queue.cc:260] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:26.266480 27168 raft_consensus.cc:3060] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.269228 27243 raft_consensus.cc:2468] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 2.
I20260709 13:10:26.270627 27059 leader_election.cc:304] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:26.271723 27326 raft_consensus.cc:2804] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:26.272162 27168 raft_consensus.cc:2468] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 2.
I20260709 13:10:26.272311 27326 raft_consensus.cc:697] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Becoming Leader. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:26.273313 27326 consensus_queue.cc:237] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:26.280486 26990 catalog_manager.cc:5697] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c reported cstate change: term changed from 1 to 2, leader changed from 23b4242091514cf4887d6fdf87717de5 (127.26.83.194) to af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193). New cstate: current_term: 2 leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:26.429529 27321 raft_consensus.cc:993] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c: : Instructing follower 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:26.430032 27326 raft_consensus.cc:1081] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Signalling peer 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:26.431731 27168 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:37928
I20260709 13:10:26.432230 27168 raft_consensus.cc:493] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:26.432547 27168 raft_consensus.cc:3060] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.437458 27168 raft_consensus.cc:515] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.439167 27168 leader_election.cc:290] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 2 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:26.440258 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:26.440696 27092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:26.440938 27243 raft_consensus.cc:3060] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.441350 27092 raft_consensus.cc:3055] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:26.441682 27092 raft_consensus.cc:740] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:26.442414 27092 consensus_queue.cc:260] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.443313 27092 raft_consensus.cc:3060] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.445688 27243 raft_consensus.cc:2468] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 2.
I20260709 13:10:26.446743 27134 leader_election.cc:304] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: 
I20260709 13:10:26.447415 27334 raft_consensus.cc:2804] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:26.447804 27334 raft_consensus.cc:697] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 2 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:26.447916 27092 raft_consensus.cc:2468] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 2.
I20260709 13:10:26.448537 27334 consensus_queue.cc:237] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.455549 26990 catalog_manager.cc:5697] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 1 to 2, leader changed from af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193) to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.475174 27321 raft_consensus.cc:993] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c: : Instructing follower 78bbe9de354b47bb95b645fdc743bea9 to start an election
I20260709 13:10:26.475641 27326 raft_consensus.cc:1081] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Signalling peer 78bbe9de354b47bb95b645fdc743bea9 to start an election
I20260709 13:10:26.477325 27243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0c12f14d5944875ab26a08bc50e49b7"
dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
 from {username='slave'} at 127.0.0.1:35852
I20260709 13:10:26.477911 27243 raft_consensus.cc:493] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:26.478221 27243 raft_consensus.cc:3060] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.484336 27243 raft_consensus.cc:515] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.486622 27384 raft_consensus.cc:993] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9: : Instructing follower 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:26.487116 27384 raft_consensus.cc:1081] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 LEADER]: Signalling peer 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:26.487576 27092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c12f14d5944875ab26a08bc50e49b7" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:26.488332 27092 raft_consensus.cc:3055] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:26.488798 27092 raft_consensus.cc:740] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:26.488852 27168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c12f14d5944875ab26a08bc50e49b7" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:26.489658 27168 raft_consensus.cc:3060] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.489934 27092 consensus_queue.cc:260] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.491139 27169 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:37938
I20260709 13:10:26.491844 27169 raft_consensus.cc:493] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:26.492285 27169 raft_consensus.cc:3060] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.491276 27092 raft_consensus.cc:3060] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.495357 27167 raft_consensus.cc:1275] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.496708 27168 raft_consensus.cc:2468] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 2.
I20260709 13:10:26.497283 27326 consensus_queue.cc:1048] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:26.497968 27208 leader_election.cc:304] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: 
I20260709 13:10:26.498668 27384 raft_consensus.cc:2804] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:26.499166 27384 raft_consensus.cc:697] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Becoming Leader. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:26.499985 27384 consensus_queue.cc:237] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.501122 27169 raft_consensus.cc:515] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.502508 27092 raft_consensus.cc:2468] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 2.
I20260709 13:10:26.504236 27169 leader_election.cc:290] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 2 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:26.501896 27243 leader_election.cc:290] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 2 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:26.505581 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:26.512092 27242 raft_consensus.cc:3055] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:26.512704 27242 raft_consensus.cc:740] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:26.513876 27242 consensus_queue.cc:260] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.515278 27092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:26.515435 27242 raft_consensus.cc:3060] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.515938 27092 raft_consensus.cc:3060] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:26.525417 27092 raft_consensus.cc:2468] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 2.
I20260709 13:10:26.525445 27243 raft_consensus.cc:1275] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.527189 27134 leader_election.cc:304] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:26.528434 27334 raft_consensus.cc:2804] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:26.529122 27334 raft_consensus.cc:697] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 2 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:26.530191 27334 consensus_queue.cc:237] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:26.537647 27242 raft_consensus.cc:2468] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 2.
I20260709 13:10:26.538132 27321 consensus_queue.cc:1048] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:26.540985 26990 catalog_manager.cc:5697] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 1 to 2, leader changed from 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195) to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.559773 26991 catalog_manager.cc:5697] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 reported cstate change: term changed from 1 to 2, leader changed from af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193) to 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195). New cstate: current_term: 2 leader_uuid: "78bbe9de354b47bb95b645fdc743bea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:26.608006 27169 raft_consensus.cc:1275] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.609287 27326 consensus_queue.cc:1048] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:26.621505 27243 raft_consensus.cc:1275] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.623215 27389 consensus_queue.cc:1048] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:26.672700 27243 raft_consensus.cc:1275] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.673938 27389 consensus_queue.cc:1048] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:26.682940 27169 raft_consensus.cc:1275] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.684669 27321 consensus_queue.cc:1048] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:26.902763 27169 raft_consensus.cc:1275] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Refusing update from remote peer 78bbe9de354b47bb95b645fdc743bea9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.904208 27384 consensus_queue.cc:1048] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:26.913532 27092 raft_consensus.cc:1275] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Refusing update from remote peer 78bbe9de354b47bb95b645fdc743bea9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.915450 27291 consensus_queue.cc:1048] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:26.962852 27243 raft_consensus.cc:1275] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.964327 27334 consensus_queue.cc:1048] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:26.974092 27092 raft_consensus.cc:1275] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:26.975409 27332 consensus_queue.cc:1048] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:27.007249 27243 raft_consensus.cc:1275] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:27.008680 27332 consensus_queue.cc:1048] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:27.019058 27092 raft_consensus.cc:1275] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:27.020750 27334 consensus_queue.cc:1048] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:29.080848 26959 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:10:29.081709 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:29.103438 27092 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
mode: GRACEFUL
new_leader_uuid: "78bbe9de354b47bb95b645fdc743bea9"
 from {username='slave'} at 127.0.0.1:39148
I20260709 13:10:29.104007 27092 raft_consensus.cc:606] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Received request to transfer leadership to TS 78bbe9de354b47bb95b645fdc743bea9
I20260709 13:10:29.115537 27169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
mode: GRACEFUL
new_leader_uuid: "78bbe9de354b47bb95b645fdc743bea9"
 from {username='slave'} at 127.0.0.1:37958
I20260709 13:10:29.116106 27169 raft_consensus.cc:606] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Received request to transfer leadership to TS 78bbe9de354b47bb95b645fdc743bea9
I20260709 13:10:29.117405 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:29.117895 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:29.357924 27408 raft_consensus.cc:993] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5: : Instructing follower 78bbe9de354b47bb95b645fdc743bea9 to start an election
I20260709 13:10:29.358421 27401 raft_consensus.cc:1081] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Signalling peer 78bbe9de354b47bb95b645fdc743bea9 to start an election
I20260709 13:10:29.360338 27243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139"
dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
 from {username='slave'} at 127.0.0.1:35860
I20260709 13:10:29.361037 27243 raft_consensus.cc:493] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:29.361410 27243 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:29.365962 27243 raft_consensus.cc:515] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:29.367535 27243 leader_election.cc:290] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 2 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:29.368845 27092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:29.368976 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:29.369494 27092 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:29.369517 27169 raft_consensus.cc:3055] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:29.370018 27169 raft_consensus.cc:740] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:29.370774 27169 consensus_queue.cc:260] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:29.371840 27169 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:29.374922 27092 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 2.
I20260709 13:10:29.376247 27208 leader_election.cc:304] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:29.377264 27404 raft_consensus.cc:2804] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:29.377449 27169 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 2.
I20260709 13:10:29.378916 27404 raft_consensus.cc:697] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Becoming Leader. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:29.379851 27404 consensus_queue.cc:237] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:29.387478 26991 catalog_manager.cc:5697] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 reported cstate change: term changed from 1 to 2, leader changed from 23b4242091514cf4887d6fdf87717de5 (127.26.83.194) to 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195). New cstate: current_term: 2 leader_uuid: "78bbe9de354b47bb95b645fdc743bea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:29.521833 27416 raft_consensus.cc:993] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c: : Instructing follower 78bbe9de354b47bb95b645fdc743bea9 to start an election
I20260709 13:10:29.522365 27326 raft_consensus.cc:1081] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Signalling peer 78bbe9de354b47bb95b645fdc743bea9 to start an election
I20260709 13:10:29.524021 27243 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5"
dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
 from {username='slave'} at 127.0.0.1:35852
I20260709 13:10:29.524564 27243 raft_consensus.cc:493] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:29.524945 27243 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:29.529247 27243 raft_consensus.cc:515] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:29.530841 27243 leader_election.cc:290] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 2 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:29.531827 27092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:29.532261 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:29.532475 27092 raft_consensus.cc:3055] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:29.532905 27169 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:29.532902 27092 raft_consensus.cc:740] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:29.533814 27092 consensus_queue.cc:260] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:29.534821 27092 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:29.538386 27169 raft_consensus.cc:2468] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 2.
I20260709 13:10:29.539685 27208 leader_election.cc:304] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: 
I20260709 13:10:29.539947 27092 raft_consensus.cc:2468] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 2.
I20260709 13:10:29.540616 27404 raft_consensus.cc:2804] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:29.541158 27404 raft_consensus.cc:697] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Becoming Leader. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:29.542016 27404 consensus_queue.cc:237] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:29.550097 26991 catalog_manager.cc:5697] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 reported cstate change: term changed from 1 to 2, leader changed from af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193) to 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195). New cstate: current_term: 2 leader_uuid: "78bbe9de354b47bb95b645fdc743bea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:29.908182 27169 raft_consensus.cc:1275] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Refusing update from remote peer 78bbe9de354b47bb95b645fdc743bea9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:29.909603 27404 consensus_queue.cc:1048] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:29.914824 27092 raft_consensus.cc:1275] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Refusing update from remote peer 78bbe9de354b47bb95b645fdc743bea9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:29.916518 27409 consensus_queue.cc:1048] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:29.952499 27169 raft_consensus.cc:1275] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Refusing update from remote peer 78bbe9de354b47bb95b645fdc743bea9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:29.953917 27419 consensus_queue.cc:1048] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:29.963141 27092 raft_consensus.cc:1275] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Refusing update from remote peer 78bbe9de354b47bb95b645fdc743bea9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:29.965559 27404 consensus_queue.cc:1048] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:30.119060 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:30.129652 27092 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
mode: GRACEFUL
new_leader_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:39140
I20260709 13:10:30.130407 27092 raft_consensus.cc:606] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Received request to transfer leadership to TS 23b4242091514cf4887d6fdf87717de5
I20260709 13:10:30.135895 27243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5"
dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
mode: GRACEFUL
new_leader_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:35866
I20260709 13:10:30.136557 27243 raft_consensus.cc:606] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Received request to transfer leadership to TS 23b4242091514cf4887d6fdf87717de5
I20260709 13:10:30.142046 27094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
mode: GRACEFUL
new_leader_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:39140
I20260709 13:10:30.142589 27094 raft_consensus.cc:606] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Received request to transfer leadership to TS 23b4242091514cf4887d6fdf87717de5
I20260709 13:10:30.150136 27094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
mode: GRACEFUL
new_leader_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:39140
I20260709 13:10:30.150709 27094 raft_consensus.cc:606] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Received request to transfer leadership to TS 23b4242091514cf4887d6fdf87717de5
I20260709 13:10:30.155913 27243 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139"
dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
mode: GRACEFUL
new_leader_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:35866
I20260709 13:10:30.156486 27243 raft_consensus.cc:606] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Received request to transfer leadership to TS 23b4242091514cf4887d6fdf87717de5
I20260709 13:10:30.230379 27426 raft_consensus.cc:993] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c: : Instructing follower 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:30.230902 27326 raft_consensus.cc:1081] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Signalling peer 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:30.232712 27169 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:37928
I20260709 13:10:30.233275 27169 raft_consensus.cc:493] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:30.233589 27169 raft_consensus.cc:3060] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.238186 27169 raft_consensus.cc:515] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.239720 27169 leader_election.cc:290] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 3 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:30.240857 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:30.241158 27243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee0994c9a9ad4420bc35c6f7a9e489c0" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:30.241442 27094 raft_consensus.cc:3055] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:30.241679 27243 raft_consensus.cc:3060] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.241828 27094 raft_consensus.cc:740] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:30.242581 27094 consensus_queue.cc:260] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.243673 27094 raft_consensus.cc:3060] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.246763 27243 raft_consensus.cc:2468] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 3.
I20260709 13:10:30.247977 27134 leader_election.cc:304] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: 
I20260709 13:10:30.248421 27094 raft_consensus.cc:2468] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 3.
I20260709 13:10:30.248993 27401 raft_consensus.cc:2804] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:30.250002 27401 raft_consensus.cc:697] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:30.250830 27401 consensus_queue.cc:237] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.258512 26989 catalog_manager.cc:5697] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 2 to 3, leader changed from af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193) to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.377424 27409 raft_consensus.cc:993] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9: : Instructing follower 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:30.377961 27404 raft_consensus.cc:1081] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Signalling peer 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:30.379796 27169 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:37938
I20260709 13:10:30.380297 27169 raft_consensus.cc:493] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:30.380599 27169 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.384668 27169 raft_consensus.cc:515] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.386384 27169 leader_election.cc:290] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 3 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:30.387099 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:30.387326 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:30.387727 27094 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.387902 27242 raft_consensus.cc:3055] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:30.388270 27242 raft_consensus.cc:740] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:30.389087 27242 consensus_queue.cc:260] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.390194 27242 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.393265 27094 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 3.
I20260709 13:10:30.394495 27134 leader_election.cc:304] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:30.395423 27401 raft_consensus.cc:2804] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:30.395469 27242 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 3.
I20260709 13:10:30.395910 27401 raft_consensus.cc:697] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:30.396858 27401 consensus_queue.cc:237] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.403177 26989 catalog_manager.cc:5697] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 2 to 3, leader changed from 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195) to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.438472 27419 raft_consensus.cc:993] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9: : Instructing follower 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:30.438915 27404 raft_consensus.cc:1081] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Signalling peer 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:30.440817 27169 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:37938
I20260709 13:10:30.441499 27169 raft_consensus.cc:493] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:30.441807 27169 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.446023 27169 raft_consensus.cc:515] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.447572 27169 leader_election.cc:290] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 3 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:30.448817 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:30.449344 27094 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.449245 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:30.449955 27242 raft_consensus.cc:3055] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:30.450255 27242 raft_consensus.cc:740] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:30.450991 27242 consensus_queue.cc:260] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.452099 27242 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.454842 27094 raft_consensus.cc:2468] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 3.
I20260709 13:10:30.456104 27134 leader_election.cc:304] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:30.457224 27401 raft_consensus.cc:2804] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:30.457788 27401 raft_consensus.cc:697] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:30.458433 27242 raft_consensus.cc:2468] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 3.
I20260709 13:10:30.458674 27401 consensus_queue.cc:237] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.465517 26989 catalog_manager.cc:5697] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 2 to 3, leader changed from 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195) to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.513641 27426 raft_consensus.cc:993] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c: : Instructing follower 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:30.514252 27326 raft_consensus.cc:1081] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Signalling peer 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:30.515944 27169 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:37928
I20260709 13:10:30.516531 27169 raft_consensus.cc:493] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:30.516871 27169 raft_consensus.cc:3060] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.520792 27169 raft_consensus.cc:515] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:30.522284 27169 leader_election.cc:290] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 3 election: Requested vote from peers 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901), af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779)
I20260709 13:10:30.523392 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:30.523533 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a3e45712a842e0b1d0fc7568486753" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:30.524016 27242 raft_consensus.cc:3060] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.524075 27094 raft_consensus.cc:3055] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:30.524627 27094 raft_consensus.cc:740] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:30.525389 27094 consensus_queue.cc:260] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:30.526286 27094 raft_consensus.cc:3060] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.529608 27242 raft_consensus.cc:2468] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 3.
I20260709 13:10:30.530750 27134 leader_election.cc:304] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, 78bbe9de354b47bb95b645fdc743bea9; no voters: 
I20260709 13:10:30.531054 27094 raft_consensus.cc:2468] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 3.
I20260709 13:10:30.531688 27401 raft_consensus.cc:2804] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:30.532173 27401 raft_consensus.cc:697] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:30.532968 27401 consensus_queue.cc:237] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } }
I20260709 13:10:30.540097 26989 catalog_manager.cc:5697] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 2 to 3, leader changed from af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193) to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.650741 27426 raft_consensus.cc:993] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c: : Instructing follower 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:30.651261 27426 raft_consensus.cc:1081] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Signalling peer 23b4242091514cf4887d6fdf87717de5 to start an election
I20260709 13:10:30.653102 27169 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
 from {username='slave'} at 127.0.0.1:37928
I20260709 13:10:30.653559 27169 raft_consensus.cc:493] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:30.653829 27169 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.658442 27169 raft_consensus.cc:515] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.659994 27169 leader_election.cc:290] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 3 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:30.661062 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:30.661259 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e" candidate_uuid: "23b4242091514cf4887d6fdf87717de5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:30.661592 27094 raft_consensus.cc:3055] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:30.662014 27094 raft_consensus.cc:740] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:30.662086 27242 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.662930 27094 consensus_queue.cc:260] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.664345 27094 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.672544 27094 raft_consensus.cc:2468] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 3.
I20260709 13:10:30.672713 27242 raft_consensus.cc:2468] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b4242091514cf4887d6fdf87717de5 in term 3.
I20260709 13:10:30.673945 27134 leader_election.cc:304] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b4242091514cf4887d6fdf87717de5, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:30.674809 27401 raft_consensus.cc:2804] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:30.675395 27401 raft_consensus.cc:697] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Becoming Leader. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:30.676155 27401 consensus_queue.cc:237] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:30.682968 26989 catalog_manager.cc:5697] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 reported cstate change: term changed from 2 to 3, leader changed from af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193) to 23b4242091514cf4887d6fdf87717de5 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "23b4242091514cf4887d6fdf87717de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:30.819567 27242 raft_consensus.cc:1275] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:30.820876 27401 consensus_queue.cc:1048] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:30.829684 27094 raft_consensus.cc:1275] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:30.831363 27401 consensus_queue.cc:1048] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:30.834049 27242 raft_consensus.cc:1275] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:30.836457 27401 consensus_queue.cc:1048] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:30.853691 27094 raft_consensus.cc:1275] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:30.857161 27427 consensus_queue.cc:1048] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:30.883394 27242 raft_consensus.cc:1275] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:30.885006 27439 consensus_queue.cc:1048] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:30.893260 27094 raft_consensus.cc:1275] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:30.895165 27438 consensus_queue.cc:1048] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:31.130856 27094 raft_consensus.cc:1275] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:31.132501 27438 consensus_queue.cc:1048] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:31.139694 27242 raft_consensus.cc:1275] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:31.141482 27438 consensus_queue.cc:1048] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:31.173914 27094 raft_consensus.cc:1275] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:31.175437 27438 consensus_queue.cc:1048] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:31.185912 27242 raft_consensus.cc:1275] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Refusing update from remote peer 23b4242091514cf4887d6fdf87717de5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:31.187168 27438 consensus_queue.cc:1048] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:33.171003 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:33.179265 27169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
mode: GRACEFUL
new_leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:37958
I20260709 13:10:33.179998 27169 raft_consensus.cc:606] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Received request to transfer leadership to TS af57968de4ac40e9ab7b6cf21ea18d7c
I20260709 13:10:33.183058 27169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
mode: GRACEFUL
new_leader_uuid: "78bbe9de354b47bb95b645fdc743bea9"
 from {username='slave'} at 127.0.0.1:37958
I20260709 13:10:33.183627 27169 raft_consensus.cc:606] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 2 LEADER]: Received request to transfer leadership to TS 78bbe9de354b47bb95b645fdc743bea9
I20260709 13:10:33.185828 27449 raft_consensus.cc:993] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5: : Instructing follower af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:33.186002 27169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
mode: GRACEFUL
new_leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:37958
I20260709 13:10:33.186374 27438 raft_consensus.cc:1081] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Signalling peer af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:33.186522 27169 raft_consensus.cc:606] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Received request to transfer leadership to TS af57968de4ac40e9ab7b6cf21ea18d7c
I20260709 13:10:33.188220 27094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:39124
I20260709 13:10:33.189025 27094 raft_consensus.cc:493] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:33.189428 27094 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:33.189652 27169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
mode: GRACEFUL
new_leader_uuid: "78bbe9de354b47bb95b645fdc743bea9"
 from {username='slave'} at 127.0.0.1:37958
I20260709 13:10:33.190241 27169 raft_consensus.cc:606] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 2 LEADER]: Received request to transfer leadership to TS 78bbe9de354b47bb95b645fdc743bea9
I20260709 13:10:33.193248 27169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e"
dest_uuid: "23b4242091514cf4887d6fdf87717de5"
mode: GRACEFUL
new_leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:37958
I20260709 13:10:33.193755 27169 raft_consensus.cc:606] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Received request to transfer leadership to TS af57968de4ac40e9ab7b6cf21ea18d7c
I20260709 13:10:33.194875 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:33.195452 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:33.196208 27094 raft_consensus.cc:515] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.198261 27094 leader_election.cc:290] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 4 election: Requested vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:33.199314 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:33.199959 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b7ee44bd2e643e5a64fbb8627620fc5" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:33.200327 27169 raft_consensus.cc:3055] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:33.200572 27242 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:33.200676 27169 raft_consensus.cc:740] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:33.201647 27169 consensus_queue.cc:260] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.203042 27169 raft_consensus.cc:3060] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:33.209055 27242 raft_consensus.cc:2468] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 4.
I20260709 13:10:33.210660 27059 leader_election.cc:304] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:33.210685 27169 raft_consensus.cc:2468] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 4.
I20260709 13:10:33.212479 27450 raft_consensus.cc:2804] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:33.213449 27450 raft_consensus.cc:697] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 LEADER]: Becoming Leader. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:33.214486 27450 consensus_queue.cc:237] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.222366 26991 catalog_manager.cc:5697] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c reported cstate change: term changed from 3 to 4, leader changed from 23b4242091514cf4887d6fdf87717de5 (127.26.83.194) to af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193). New cstate: current_term: 4 leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:33.230400 27449 raft_consensus.cc:993] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5: : Instructing follower 78bbe9de354b47bb95b645fdc743bea9 to start an election
I20260709 13:10:33.231274 27438 raft_consensus.cc:1081] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 2 LEADER]: Signalling peer 78bbe9de354b47bb95b645fdc743bea9 to start an election
I20260709 13:10:33.233268 27242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9"
dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
 from {username='slave'} at 127.0.0.1:35860
I20260709 13:10:33.234020 27242 raft_consensus.cc:493] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:33.234458 27242 raft_consensus.cc:3060] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:33.240324 27242 raft_consensus.cc:515] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.242295 27242 leader_election.cc:290] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 3 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:33.243306 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:33.243594 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "889c1d5a24f7431f8b3008c5a61958e9" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:33.243886 27094 raft_consensus.cc:3060] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:33.244091 27169 raft_consensus.cc:3055] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:33.244388 27169 raft_consensus.cc:740] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:33.245108 27169 consensus_queue.cc:260] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.246183 27169 raft_consensus.cc:3060] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:33.250070 27094 raft_consensus.cc:2468] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 3.
I20260709 13:10:33.251474 27208 leader_election.cc:304] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:33.251965 27169 raft_consensus.cc:2468] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 3.
I20260709 13:10:33.252310 27404 raft_consensus.cc:2804] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:33.253538 27404 raft_consensus.cc:697] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 LEADER]: Becoming Leader. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:33.254289 27404 consensus_queue.cc:237] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.261083 26991 catalog_manager.cc:5697] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 reported cstate change: term changed from 2 to 3, leader changed from 23b4242091514cf4887d6fdf87717de5 (127.26.83.194) to 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195). New cstate: current_term: 3 leader_uuid: "78bbe9de354b47bb95b645fdc743bea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:33.428371 27449 raft_consensus.cc:993] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5: : Instructing follower af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:33.428865 27449 raft_consensus.cc:1081] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Signalling peer af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:33.430420 27094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:39124
I20260709 13:10:33.430990 27094 raft_consensus.cc:493] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:33.431454 27094 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:33.435997 27094 raft_consensus.cc:515] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.437665 27094 leader_election.cc:290] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 4 election: Requested vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:33.438638 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:33.439261 27169 raft_consensus.cc:3055] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:33.439056 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "835fc7cd2a2641ab9269c1c172a07139" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:33.439585 27169 raft_consensus.cc:740] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:33.439723 27242 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:33.440385 27169 consensus_queue.cc:260] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.441628 27169 raft_consensus.cc:3060] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:33.446252 27242 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 4.
I20260709 13:10:33.447576 27169 raft_consensus.cc:2468] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 4.
I20260709 13:10:33.447903 27059 leader_election.cc:304] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:33.448697 27450 raft_consensus.cc:2804] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:33.449188 27450 raft_consensus.cc:697] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 LEADER]: Becoming Leader. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:33.449949 27450 consensus_queue.cc:237] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.459160 26991 catalog_manager.cc:5697] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c reported cstate change: term changed from 3 to 4, leader changed from 23b4242091514cf4887d6fdf87717de5 (127.26.83.194) to af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193). New cstate: current_term: 4 leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:33.490239 27455 raft_consensus.cc:993] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5: : Instructing follower af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:33.490711 27455 raft_consensus.cc:1081] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Signalling peer af57968de4ac40e9ab7b6cf21ea18d7c to start an election
I20260709 13:10:33.492312 27094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e"
dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
 from {username='slave'} at 127.0.0.1:39124
I20260709 13:10:33.492846 27094 raft_consensus.cc:493] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:33.493342 27094 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:33.498672 27094 raft_consensus.cc:515] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.500396 27094 leader_election.cc:290] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 4 election: Requested vote from peers 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335), 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195:35901)
I20260709 13:10:33.501343 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:33.501453 27242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e1dadf7f86744d0b92a7ff1b03c896e" candidate_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
I20260709 13:10:33.501849 27169 raft_consensus.cc:3055] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:33.502005 27242 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:33.502143 27169 raft_consensus.cc:740] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:33.502949 27169 consensus_queue.cc:260] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.503919 27169 raft_consensus.cc:3060] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:33.506713 27242 raft_consensus.cc:2468] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 4.
I20260709 13:10:33.507791 27059 leader_election.cc:304] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:33.508256 27169 raft_consensus.cc:2468] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af57968de4ac40e9ab7b6cf21ea18d7c in term 4.
I20260709 13:10:33.508520 27450 raft_consensus.cc:2804] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:33.509088 27450 raft_consensus.cc:697] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 LEADER]: Becoming Leader. State: Replica: af57968de4ac40e9ab7b6cf21ea18d7c, State: Running, Role: LEADER
I20260709 13:10:33.510062 27450 consensus_queue.cc:237] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.517874 26991 catalog_manager.cc:5697] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c reported cstate change: term changed from 3 to 4, leader changed from 23b4242091514cf4887d6fdf87717de5 (127.26.83.194) to af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193). New cstate: current_term: 4 leader_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:33.638820 27449 raft_consensus.cc:993] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5: : Instructing follower 78bbe9de354b47bb95b645fdc743bea9 to start an election
I20260709 13:10:33.639397 27455 raft_consensus.cc:1081] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 2 LEADER]: Signalling peer 78bbe9de354b47bb95b645fdc743bea9 to start an election
I20260709 13:10:33.640903 27242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53"
dest_uuid: "78bbe9de354b47bb95b645fdc743bea9"
 from {username='slave'} at 127.0.0.1:35860
I20260709 13:10:33.641345 27242 raft_consensus.cc:493] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:33.641623 27242 raft_consensus.cc:3060] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:33.646010 27242 raft_consensus.cc:515] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.647778 27242 leader_election.cc:290] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 3 election: Requested vote from peers af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779), 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335)
I20260709 13:10:33.648670 27094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c"
I20260709 13:10:33.649163 27169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "625e7ef75e45444dbdfc17ba22057e53" candidate_uuid: "78bbe9de354b47bb95b645fdc743bea9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "23b4242091514cf4887d6fdf87717de5"
I20260709 13:10:33.649360 27094 raft_consensus.cc:3060] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:33.649714 27169 raft_consensus.cc:3055] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:33.649992 27169 raft_consensus.cc:740] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 23b4242091514cf4887d6fdf87717de5, State: Running, Role: LEADER
I20260709 13:10:33.650606 27169 consensus_queue.cc:260] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.651475 27169 raft_consensus.cc:3060] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:33.654309 27094 raft_consensus.cc:2468] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 3.
I20260709 13:10:33.655508 27208 leader_election.cc:304] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78bbe9de354b47bb95b645fdc743bea9, af57968de4ac40e9ab7b6cf21ea18d7c; no voters: 
I20260709 13:10:33.655704 27169 raft_consensus.cc:2468] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78bbe9de354b47bb95b645fdc743bea9 in term 3.
I20260709 13:10:33.656385 27404 raft_consensus.cc:2804] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:33.656956 27404 raft_consensus.cc:697] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 LEADER]: Becoming Leader. State: Replica: 78bbe9de354b47bb95b645fdc743bea9, State: Running, Role: LEADER
I20260709 13:10:33.657850 27404 consensus_queue.cc:237] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } }
I20260709 13:10:33.666476 26991 catalog_manager.cc:5697] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 reported cstate change: term changed from 2 to 3, leader changed from 23b4242091514cf4887d6fdf87717de5 (127.26.83.194) to 78bbe9de354b47bb95b645fdc743bea9 (127.26.83.195). New cstate: current_term: 3 leader_uuid: "78bbe9de354b47bb95b645fdc743bea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:33.750051 27242 raft_consensus.cc:1275] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:33.751611 27450 consensus_queue.cc:1048] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:33.763880 27169 raft_consensus.cc:1275] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:33.765534 27465 consensus_queue.cc:1048] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:33.808697 27094 raft_consensus.cc:1275] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Refusing update from remote peer 78bbe9de354b47bb95b645fdc743bea9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:33.810160 27404 consensus_queue.cc:1048] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:33.820312 27168 raft_consensus.cc:1275] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Refusing update from remote peer 78bbe9de354b47bb95b645fdc743bea9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:33.822304 27470 consensus_queue.cc:1048] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:33.873188 27168 raft_consensus.cc:1275] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:33.874655 27465 consensus_queue.cc:1048] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:33.884613 27242 raft_consensus.cc:1275] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:33.886322 27465 consensus_queue.cc:1048] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:33.915812 27168 raft_consensus.cc:1275] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:33.917245 27465 consensus_queue.cc:1048] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:33.926348 27242 raft_consensus.cc:1275] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Refusing update from remote peer af57968de4ac40e9ab7b6cf21ea18d7c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:33.928795 27451 consensus_queue.cc:1048] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bbe9de354b47bb95b645fdc743bea9" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:34.080904 27094 raft_consensus.cc:1275] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Refusing update from remote peer 78bbe9de354b47bb95b645fdc743bea9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:34.082119 27470 consensus_queue.cc:1048] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af57968de4ac40e9ab7b6cf21ea18d7c" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:34.091893 27168 raft_consensus.cc:1275] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Refusing update from remote peer 78bbe9de354b47bb95b645fdc743bea9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:34.093937 27470 consensus_queue.cc:1048] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b4242091514cf4887d6fdf87717de5" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 33335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:34.196564 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:34.202024 26959 tablet_server.cc:179] TabletServer@127.26.83.193:0 shutting down...
W20260709 13:10:34.213438 27134 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:10:34.220458 27134 consensus_peers.cc:597] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 -> Peer af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779): Couldn't send request to peer af57968de4ac40e9ab7b6cf21ea18d7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:34.242833 26959 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:34.243664 26959 tablet_replica.cc:333] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c: stopping tablet replica
I20260709 13:10:34.244369 26959 raft_consensus.cc:2243] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.244948 26959 raft_consensus.cc:2272] T ee0994c9a9ad4420bc35c6f7a9e489c0 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.247340 26959 tablet_replica.cc:333] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c: stopping tablet replica
I20260709 13:10:34.247856 26959 raft_consensus.cc:2243] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.248265 26959 raft_consensus.cc:2272] T 3155a7e09dc74a369b6d5694c395bea7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.249958 26959 tablet_replica.cc:333] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c: stopping tablet replica
I20260709 13:10:34.250445 26959 raft_consensus.cc:2243] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.250849 26959 raft_consensus.cc:2272] T 625e7ef75e45444dbdfc17ba22057e53 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.253012 26959 tablet_replica.cc:333] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c: stopping tablet replica
I20260709 13:10:34.253571 26959 raft_consensus.cc:2243] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.253971 26959 raft_consensus.cc:2272] T d0c12f14d5944875ab26a08bc50e49b7 P af57968de4ac40e9ab7b6cf21ea18d7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.255617 26959 tablet_replica.cc:333] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c: stopping tablet replica
I20260709 13:10:34.256122 26959 raft_consensus.cc:2243] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.256517 26959 raft_consensus.cc:2272] T 57a3e45712a842e0b1d0fc7568486753 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.258128 26959 tablet_replica.cc:333] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c: stopping tablet replica
I20260709 13:10:34.258618 26959 raft_consensus.cc:2243] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:34.259356 26959 raft_consensus.cc:2272] T 835fc7cd2a2641ab9269c1c172a07139 P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.261916 26959 tablet_replica.cc:333] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c: stopping tablet replica
I20260709 13:10:34.262405 26959 raft_consensus.cc:2243] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:34.263079 26959 raft_consensus.cc:2272] T 5b7ee44bd2e643e5a64fbb8627620fc5 P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.265053 26959 tablet_replica.cc:333] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c: stopping tablet replica
I20260709 13:10:34.265507 26959 raft_consensus.cc:2243] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:34.266212 26959 raft_consensus.cc:2272] T 8e1dadf7f86744d0b92a7ff1b03c896e P af57968de4ac40e9ab7b6cf21ea18d7c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.268220 26959 tablet_replica.cc:333] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c: stopping tablet replica
I20260709 13:10:34.268695 26959 raft_consensus.cc:2243] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.269306 26959 raft_consensus.cc:2272] T 889c1d5a24f7431f8b3008c5a61958e9 P af57968de4ac40e9ab7b6cf21ea18d7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.416529 26959 tablet_server.cc:196] TabletServer@127.26.83.193:0 shutdown complete.
W20260709 13:10:34.420904 27208 consensus_peers.cc:597] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 -> Peer af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779): Couldn't send request to peer af57968de4ac40e9ab7b6cf21ea18d7c. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:44779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:34.422688 27134 consensus_peers.cc:597] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 -> Peer af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779): Couldn't send request to peer af57968de4ac40e9ab7b6cf21ea18d7c. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:44779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:34.427688 27208 consensus_peers.cc:597] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 -> Peer af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779): Couldn't send request to peer af57968de4ac40e9ab7b6cf21ea18d7c. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:44779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:34.440418 26959 tablet_server.cc:179] TabletServer@127.26.83.194:0 shutting down...
I20260709 13:10:34.465798 26959 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:34.466511 26959 tablet_replica.cc:333] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5: stopping tablet replica
I20260709 13:10:34.467129 26959 raft_consensus.cc:2243] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:34.467924 26959 raft_consensus.cc:2272] T 3155a7e09dc74a369b6d5694c395bea7 P 23b4242091514cf4887d6fdf87717de5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.469910 26959 tablet_replica.cc:333] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5: stopping tablet replica
I20260709 13:10:34.470439 26959 raft_consensus.cc:2243] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.470829 26959 raft_consensus.cc:2272] T 625e7ef75e45444dbdfc17ba22057e53 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.473371 26959 tablet_replica.cc:333] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5: stopping tablet replica
I20260709 13:10:34.473894 26959 raft_consensus.cc:2243] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.474279 26959 raft_consensus.cc:2272] T d0c12f14d5944875ab26a08bc50e49b7 P 23b4242091514cf4887d6fdf87717de5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.476267 26959 tablet_replica.cc:333] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5: stopping tablet replica
I20260709 13:10:34.476841 26959 raft_consensus.cc:2243] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:34.477555 26959 raft_consensus.cc:2272] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.479713 26959 tablet_replica.cc:333] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5: stopping tablet replica
I20260709 13:10:34.480438 26959 raft_consensus.cc:2243] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.480968 26959 raft_consensus.cc:2272] T 8e1dadf7f86744d0b92a7ff1b03c896e P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.483558 26959 tablet_replica.cc:333] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5: stopping tablet replica
I20260709 13:10:34.484000 26959 raft_consensus.cc:2243] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.484341 26959 raft_consensus.cc:2272] T 889c1d5a24f7431f8b3008c5a61958e9 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.486238 26959 tablet_replica.cc:333] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5: stopping tablet replica
I20260709 13:10:34.486670 26959 raft_consensus.cc:2243] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.487015 26959 raft_consensus.cc:2272] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.489022 26959 tablet_replica.cc:333] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5: stopping tablet replica
I20260709 13:10:34.489421 26959 raft_consensus.cc:2243] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.489758 26959 raft_consensus.cc:2272] T 835fc7cd2a2641ab9269c1c172a07139 P 23b4242091514cf4887d6fdf87717de5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.491729 26959 tablet_replica.cc:333] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5: stopping tablet replica
I20260709 13:10:34.492112 26959 raft_consensus.cc:2243] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:34.492844 26959 raft_consensus.cc:2272] T 57a3e45712a842e0b1d0fc7568486753 P 23b4242091514cf4887d6fdf87717de5 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:34.498034 27208 consensus_peers.cc:597] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 -> Peer 23b4242091514cf4887d6fdf87717de5 (127.26.83.194:33335): Couldn't send request to peer 23b4242091514cf4887d6fdf87717de5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:34.522780 26959 tablet_server.cc:196] TabletServer@127.26.83.194:0 shutdown complete.
I20260709 13:10:34.541164 26959 tablet_server.cc:179] TabletServer@127.26.83.195:0 shutting down...
W20260709 13:10:34.550295 27208 consensus_peers.cc:597] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 -> Peer af57968de4ac40e9ab7b6cf21ea18d7c (127.26.83.193:44779): Couldn't send request to peer af57968de4ac40e9ab7b6cf21ea18d7c. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:44779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:34.575110 26959 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:34.576186 26959 tablet_replica.cc:333] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9: stopping tablet replica
I20260709 13:10:34.577071 26959 raft_consensus.cc:2243] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:34.577874 26959 raft_consensus.cc:2272] T 625e7ef75e45444dbdfc17ba22057e53 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.580236 26959 tablet_replica.cc:333] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9: stopping tablet replica
I20260709 13:10:34.580814 26959 raft_consensus.cc:2243] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.581200 26959 raft_consensus.cc:2272] T 3155a7e09dc74a369b6d5694c395bea7 P 78bbe9de354b47bb95b645fdc743bea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.582919 26959 tablet_replica.cc:333] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9: stopping tablet replica
I20260709 13:10:34.583420 26959 raft_consensus.cc:2243] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:34.584159 26959 raft_consensus.cc:2272] T d0c12f14d5944875ab26a08bc50e49b7 P 78bbe9de354b47bb95b645fdc743bea9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.585873 26959 tablet_replica.cc:333] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9: stopping tablet replica
I20260709 13:10:34.586354 26959 raft_consensus.cc:2243] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.586756 26959 raft_consensus.cc:2272] T ee0994c9a9ad4420bc35c6f7a9e489c0 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.588366 26959 tablet_replica.cc:333] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9: stopping tablet replica
I20260709 13:10:34.588905 26959 raft_consensus.cc:2243] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.589287 26959 raft_consensus.cc:2272] T 57a3e45712a842e0b1d0fc7568486753 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.591262 26959 tablet_replica.cc:333] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9: stopping tablet replica
I20260709 13:10:34.591712 26959 raft_consensus.cc:2243] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.592099 26959 raft_consensus.cc:2272] T 835fc7cd2a2641ab9269c1c172a07139 P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.594321 26959 tablet_replica.cc:333] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9: stopping tablet replica
I20260709 13:10:34.594914 26959 raft_consensus.cc:2243] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.595372 26959 raft_consensus.cc:2272] T 5b7ee44bd2e643e5a64fbb8627620fc5 P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.597630 26959 tablet_replica.cc:333] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9: stopping tablet replica
I20260709 13:10:34.598153 26959 raft_consensus.cc:2243] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:34.598877 26959 raft_consensus.cc:2272] T 889c1d5a24f7431f8b3008c5a61958e9 P 78bbe9de354b47bb95b645fdc743bea9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.601019 26959 tablet_replica.cc:333] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9: stopping tablet replica
I20260709 13:10:34.601537 26959 raft_consensus.cc:2243] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:34.602001 26959 raft_consensus.cc:2272] T 8e1dadf7f86744d0b92a7ff1b03c896e P 78bbe9de354b47bb95b645fdc743bea9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.630642 26959 tablet_server.cc:196] TabletServer@127.26.83.195:0 shutdown complete.
I20260709 13:10:34.647042 26959 master.cc:562] Master@127.26.83.254:42853 shutting down...
I20260709 13:10:34.724850 27046 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 13:10:34.785040 26959 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:34.785766 26959 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:34.786161 26959 tablet_replica.cc:333] T 00000000000000000000000000000000 P c8a1d887873e49259848993e8211b778: stopping tablet replica
I20260709 13:10:34.806196 26959 master.cc:584] Master@127.26.83.254:42853 shutdown complete.
23b4242091514cf4887d6fdf87717de5  4
78bbe9de354b47bb95b645fdc743bea9  1
af57968de4ac40e9ab7b6cf21ea18d7c  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (13592 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:10:34.847644 26959 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.83.254:34083
I20260709 13:10:34.848886 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:34.855170 27482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:34.855170 27483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:34.856102 27485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:34.856885 26959 server_base.cc:1061] running on GCE node
I20260709 13:10:34.857824 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:34.858028 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:34.858251 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602634858229 us; error 0 us; skew 500 ppm
I20260709 13:10:34.858826 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:34.862145 26959 webserver.cc:533] Webserver started at http://127.26.83.254:45221/ using document root <none> and password file <none>
I20260709 13:10:34.862711 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:34.862885 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:34.863224 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:34.864387 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/master-0-root/instance:
uuid: "ed97f5f52dd34409b3fde4a5e6b3b5eb"
format_stamp: "Formatted at 2026-07-09 13:10:34 on dist-test-slave-329l"
I20260709 13:10:34.868956 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:10:34.872368 27490 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:34.873355 26959 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:34.873610 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/master-0-root
uuid: "ed97f5f52dd34409b3fde4a5e6b3b5eb"
format_stamp: "Formatted at 2026-07-09 13:10:34 on dist-test-slave-329l"
I20260709 13:10:34.873864 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:34.887522 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:34.888846 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:34.931133 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.254:34083
I20260709 13:10:34.931348 27541 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.254:34083 every 8 connection(s)
I20260709 13:10:34.935390 27542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:34.945909 27542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb: Bootstrap starting.
I20260709 13:10:34.951067 27542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:34.955679 27542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb: No bootstrap required, opened a new log
I20260709 13:10:34.957970 27542 raft_consensus.cc:359] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed97f5f52dd34409b3fde4a5e6b3b5eb" member_type: VOTER }
I20260709 13:10:34.958384 27542 raft_consensus.cc:385] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:34.958580 27542 raft_consensus.cc:740] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed97f5f52dd34409b3fde4a5e6b3b5eb, State: Initialized, Role: FOLLOWER
I20260709 13:10:34.959156 27542 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed97f5f52dd34409b3fde4a5e6b3b5eb" member_type: VOTER }
I20260709 13:10:34.959632 27542 raft_consensus.cc:399] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:34.959857 27542 raft_consensus.cc:493] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:34.960106 27542 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:34.964560 27542 raft_consensus.cc:515] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed97f5f52dd34409b3fde4a5e6b3b5eb" member_type: VOTER }
I20260709 13:10:34.965157 27542 leader_election.cc:304] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed97f5f52dd34409b3fde4a5e6b3b5eb; no voters: 
I20260709 13:10:34.966398 27542 leader_election.cc:290] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:34.966738 27545 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:34.968199 27545 raft_consensus.cc:697] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 1 LEADER]: Becoming Leader. State: Replica: ed97f5f52dd34409b3fde4a5e6b3b5eb, State: Running, Role: LEADER
I20260709 13:10:34.969022 27545 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed97f5f52dd34409b3fde4a5e6b3b5eb" member_type: VOTER }
I20260709 13:10:34.970053 27542 sys_catalog.cc:565] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:34.974067 27546 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed97f5f52dd34409b3fde4a5e6b3b5eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed97f5f52dd34409b3fde4a5e6b3b5eb" member_type: VOTER } }
I20260709 13:10:34.974339 27547 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed97f5f52dd34409b3fde4a5e6b3b5eb. Latest consensus state: current_term: 1 leader_uuid: "ed97f5f52dd34409b3fde4a5e6b3b5eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed97f5f52dd34409b3fde4a5e6b3b5eb" member_type: VOTER } }
I20260709 13:10:34.975024 27546 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:34.975445 27547 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:34.978991 27551 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:34.988520 27551 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:34.991125 26959 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:34.999152 27551 catalog_manager.cc:1383] Generated new cluster ID: a448b0bc75cb4324970e79ebe6bc84b5
I20260709 13:10:34.999596 27551 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:35.011323 27551 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:35.012594 27551 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:35.028014 27551 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb: Generated new TSK 0
I20260709 13:10:35.028789 27551 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:35.058292 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:35.064419 27563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:35.067005 27564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:35.067797 27566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:35.068838 26959 server_base.cc:1061] running on GCE node
I20260709 13:10:35.069708 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:35.069916 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:35.070093 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602635070074 us; error 0 us; skew 500 ppm
I20260709 13:10:35.070628 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:35.073329 26959 webserver.cc:533] Webserver started at http://127.26.83.193:45187/ using document root <none> and password file <none>
I20260709 13:10:35.073825 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:35.074008 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:35.074260 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:35.075443 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-0-root/instance:
uuid: "b489ccf32e7542de8d18ba7e1730caf7"
format_stamp: "Formatted at 2026-07-09 13:10:35 on dist-test-slave-329l"
I20260709 13:10:35.080396 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:10:35.083930 27571 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:35.084873 26959 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:35.085188 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-0-root
uuid: "b489ccf32e7542de8d18ba7e1730caf7"
format_stamp: "Formatted at 2026-07-09 13:10:35 on dist-test-slave-329l"
I20260709 13:10:35.085489 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:35.112085 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:35.113415 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:35.114972 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:35.117357 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:35.117534 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:35.117848 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:35.118005 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:35.163491 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.193:36085
I20260709 13:10:35.163604 27633 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.193:36085 every 8 connection(s)
I20260709 13:10:35.168779 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:35.177572 27638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:35.178717 27639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:35.183247 27641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:35.187274 26959 server_base.cc:1061] running on GCE node
I20260709 13:10:35.188270 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:35.188508 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:35.188688 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602635188672 us; error 0 us; skew 500 ppm
I20260709 13:10:35.189424 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:35.192826 26959 webserver.cc:533] Webserver started at http://127.26.83.194:45933/ using document root <none> and password file <none>
I20260709 13:10:35.193537 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:35.193799 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:35.194152 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:35.194201 27634 heartbeater.cc:344] Connected to a master server at 127.26.83.254:34083
I20260709 13:10:35.194650 27634 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:35.195367 27634 heartbeater.cc:507] Master 127.26.83.254:34083 requested a full tablet report, sending...
I20260709 13:10:35.195561 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-1-root/instance:
uuid: "81de5334c2aa49909f114e12900b357c"
format_stamp: "Formatted at 2026-07-09 13:10:35 on dist-test-slave-329l"
I20260709 13:10:35.197898 27507 ts_manager.cc:194] Registered new tserver with Master: b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085)
I20260709 13:10:35.200076 27507 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32934
I20260709 13:10:35.201402 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:10:35.205489 27646 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:35.206682 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:10:35.207083 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-1-root
uuid: "81de5334c2aa49909f114e12900b357c"
format_stamp: "Formatted at 2026-07-09 13:10:35 on dist-test-slave-329l"
I20260709 13:10:35.207472 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:35.222075 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:35.223742 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:35.225234 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:35.227535 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:35.227730 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:35.228039 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:35.228197 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:35.271561 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.194:46029
I20260709 13:10:35.271688 27708 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.194:46029 every 8 connection(s)
I20260709 13:10:35.277495 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:35.284332 27712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:35.285385 27713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:35.290133 27715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:35.291010 26959 server_base.cc:1061] running on GCE node
I20260709 13:10:35.291307 27709 heartbeater.cc:344] Connected to a master server at 127.26.83.254:34083
I20260709 13:10:35.291785 27709 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:35.291992 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:35.292250 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:35.292423 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602635292389 us; error 0 us; skew 500 ppm
I20260709 13:10:35.292667 27709 heartbeater.cc:507] Master 127.26.83.254:34083 requested a full tablet report, sending...
I20260709 13:10:35.293108 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:35.295123 27507 ts_manager.cc:194] Registered new tserver with Master: 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:35.296072 26959 webserver.cc:533] Webserver started at http://127.26.83.195:38161/ using document root <none> and password file <none>
I20260709 13:10:35.296705 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:35.296813 27507 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32946
I20260709 13:10:35.297178 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:35.297499 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:35.298707 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-2-root/instance:
uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
format_stamp: "Formatted at 2026-07-09 13:10:35 on dist-test-slave-329l"
I20260709 13:10:35.303426 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:10:35.307163 27720 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:35.308120 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 13:10:35.308542 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-2-root
uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
format_stamp: "Formatted at 2026-07-09 13:10:35 on dist-test-slave-329l"
I20260709 13:10:35.308943 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602621086139-26959-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:35.333994 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:35.336170 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:35.339155 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:35.343472 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:35.343899 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:10:35.344331 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:35.344666 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:35.398983 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.195:40721
I20260709 13:10:35.399082 27782 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.195:40721 every 8 connection(s)
I20260709 13:10:35.418143 27783 heartbeater.cc:344] Connected to a master server at 127.26.83.254:34083
I20260709 13:10:35.418555 27783 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:35.419459 27783 heartbeater.cc:507] Master 127.26.83.254:34083 requested a full tablet report, sending...
I20260709 13:10:35.421703 27507 ts_manager.cc:194] Registered new tserver with Master: 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:35.421814 26959 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019564301s
I20260709 13:10:35.423470 27507 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32958
I20260709 13:10:35.451556 27507 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32962:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:35.456126 27507 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:35.544602 27748 tablet_service.cc:1511] Processing CreateTablet for tablet 39b5a991814a494c8e3ac259a6d66915 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:35.546337 27748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b5a991814a494c8e3ac259a6d66915. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.547132 27745 tablet_service.cc:1511] Processing CreateTablet for tablet 814ca7b7dd7849658d9093302fc97388 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:35.548856 27747 tablet_service.cc:1511] Processing CreateTablet for tablet d0d3499349e2425798b193e641999c0e (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:35.551349 27746 tablet_service.cc:1511] Processing CreateTablet for tablet a4163203b22b452985b9774ad4d7a2c9 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:35.553001 27745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 814ca7b7dd7849658d9093302fc97388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.555976 27744 tablet_service.cc:1511] Processing CreateTablet for tablet c49c4ac4d6fa4ce18035f3cafb56340a (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:35.557101 27746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4163203b22b452985b9774ad4d7a2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.558138 27744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49c4ac4d6fa4ce18035f3cafb56340a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.561002 27747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d3499349e2425798b193e641999c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.571167 27674 tablet_service.cc:1511] Processing CreateTablet for tablet 39b5a991814a494c8e3ac259a6d66915 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:35.573381 27674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b5a991814a494c8e3ac259a6d66915. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.575809 27743 tablet_service.cc:1511] Processing CreateTablet for tablet 404737675c1a4dacb0283ccccea7a99d (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:35.577344 27743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404737675c1a4dacb0283ccccea7a99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.587325 27673 tablet_service.cc:1511] Processing CreateTablet for tablet d0d3499349e2425798b193e641999c0e (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:35.589551 27673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d3499349e2425798b193e641999c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.599263 27672 tablet_service.cc:1511] Processing CreateTablet for tablet a4163203b22b452985b9774ad4d7a2c9 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:35.601610 27672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4163203b22b452985b9774ad4d7a2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.610671 27798 tablet_bootstrap.cc:492] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac: Bootstrap starting.
I20260709 13:10:35.614053 27671 tablet_service.cc:1511] Processing CreateTablet for tablet 814ca7b7dd7849658d9093302fc97388 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:35.615614 27671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 814ca7b7dd7849658d9093302fc97388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.621388 27748 tablet_service.cc:1511] Processing CreateTablet for tablet 91da424edc5d4e5fb4ed0a6c8549339b (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:35.623122 27748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91da424edc5d4e5fb4ed0a6c8549339b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.633968 27670 tablet_service.cc:1511] Processing CreateTablet for tablet c49c4ac4d6fa4ce18035f3cafb56340a (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:35.636270 27670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49c4ac4d6fa4ce18035f3cafb56340a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.641070 27669 tablet_service.cc:1511] Processing CreateTablet for tablet 404737675c1a4dacb0283ccccea7a99d (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:35.643293 27669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404737675c1a4dacb0283ccccea7a99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.651612 27798 tablet_bootstrap.cc:654] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.651300 27668 tablet_service.cc:1511] Processing CreateTablet for tablet 3904a7744db64435a28bb973da2089f5 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:35.653566 27668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3904a7744db64435a28bb973da2089f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.653908 27743 tablet_service.cc:1511] Processing CreateTablet for tablet 681a55081a294370909a44c09d683d34 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:35.655287 27743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681a55081a294370909a44c09d683d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.664527 27747 tablet_service.cc:1511] Processing CreateTablet for tablet 15983c9724104e7c86e9ea2e6ca0e7ff (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:35.666409 27747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15983c9724104e7c86e9ea2e6ca0e7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.686744 27667 tablet_service.cc:1511] Processing CreateTablet for tablet 91da424edc5d4e5fb4ed0a6c8549339b (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:35.688285 27667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91da424edc5d4e5fb4ed0a6c8549339b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.693980 27666 tablet_service.cc:1511] Processing CreateTablet for tablet 681a55081a294370909a44c09d683d34 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:35.601380 27744 tablet_service.cc:1511] Processing CreateTablet for tablet 3904a7744db64435a28bb973da2089f5 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:35.696766 27665 tablet_service.cc:1511] Processing CreateTablet for tablet 15983c9724104e7c86e9ea2e6ca0e7ff (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:35.698357 27666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681a55081a294370909a44c09d683d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.709470 27744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3904a7744db64435a28bb973da2089f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.711897 27665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15983c9724104e7c86e9ea2e6ca0e7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.728552 27798 tablet_bootstrap.cc:492] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac: No bootstrap required, opened a new log
I20260709 13:10:35.729133 27798 ts_tablet_manager.cc:1403] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent bootstrapping tablet: real 0.119s	user 0.011s	sys 0.022s
I20260709 13:10:35.734746 27799 tablet_bootstrap.cc:492] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c: Bootstrap starting.
I20260709 13:10:35.734869 27798 raft_consensus.cc:359] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:35.735704 27798 raft_consensus.cc:385] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.736029 27798 raft_consensus.cc:740] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.741032 27798 consensus_queue.cc:260] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:35.748055 27799 tablet_bootstrap.cc:654] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.756237 27798 ts_tablet_manager.cc:1434] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.004s
I20260709 13:10:35.757061 27783 heartbeater.cc:499] Master 127.26.83.254:34083 was elected leader, sending a full tablet report...
I20260709 13:10:35.761577 27798 tablet_bootstrap.cc:492] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac: Bootstrap starting.
I20260709 13:10:35.768342 27798 tablet_bootstrap.cc:654] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.769739 27799 tablet_bootstrap.cc:492] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c: No bootstrap required, opened a new log
I20260709 13:10:35.770224 27799 ts_tablet_manager.cc:1403] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.012s
I20260709 13:10:35.774016 27599 tablet_service.cc:1511] Processing CreateTablet for tablet 39b5a991814a494c8e3ac259a6d66915 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:10:35.775409 27599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b5a991814a494c8e3ac259a6d66915. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.776330 27596 tablet_service.cc:1511] Processing CreateTablet for tablet 814ca7b7dd7849658d9093302fc97388 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:10:35.777788 27596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 814ca7b7dd7849658d9093302fc97388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.782073 27597 tablet_service.cc:1511] Processing CreateTablet for tablet a4163203b22b452985b9774ad4d7a2c9 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:10:35.783469 27597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4163203b22b452985b9774ad4d7a2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.785879 27598 tablet_service.cc:1511] Processing CreateTablet for tablet d0d3499349e2425798b193e641999c0e (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:10:35.787242 27598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0d3499349e2425798b193e641999c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.791213 27595 tablet_service.cc:1511] Processing CreateTablet for tablet c49c4ac4d6fa4ce18035f3cafb56340a (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:10:35.792654 27595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c49c4ac4d6fa4ce18035f3cafb56340a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.794072 27804 tablet_bootstrap.cc:492] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7: Bootstrap starting.
I20260709 13:10:35.794030 27594 tablet_service.cc:1511] Processing CreateTablet for tablet 404737675c1a4dacb0283ccccea7a99d (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:10:35.795255 27594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 404737675c1a4dacb0283ccccea7a99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.797735 27593 tablet_service.cc:1511] Processing CreateTablet for tablet 3904a7744db64435a28bb973da2089f5 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:10:35.798938 27593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3904a7744db64435a28bb973da2089f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.800423 27804 tablet_bootstrap.cc:654] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.802563 27591 tablet_service.cc:1511] Processing CreateTablet for tablet 15983c9724104e7c86e9ea2e6ca0e7ff (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:10:35.804010 27591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15983c9724104e7c86e9ea2e6ca0e7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.800498 27592 tablet_service.cc:1511] Processing CreateTablet for tablet 681a55081a294370909a44c09d683d34 (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:10:35.792613 27798 tablet_bootstrap.cc:492] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac: No bootstrap required, opened a new log
I20260709 13:10:35.807236 27798 ts_tablet_manager.cc:1403] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent bootstrapping tablet: real 0.046s	user 0.018s	sys 0.011s
I20260709 13:10:35.807938 27599 tablet_service.cc:1511] Processing CreateTablet for tablet 91da424edc5d4e5fb4ed0a6c8549339b (DEFAULT_TABLE table=test-workload [id=f2ffb67fcbaa4864b6ed65faccaadfd4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:10:35.809343 27592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 681a55081a294370909a44c09d683d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.811292 27599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91da424edc5d4e5fb4ed0a6c8549339b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:35.830864 27798 raft_consensus.cc:359] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.831491 27798 raft_consensus.cc:385] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.831776 27798 raft_consensus.cc:740] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.832670 27798 consensus_queue.cc:260] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.834311 27804 tablet_bootstrap.cc:492] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7: No bootstrap required, opened a new log
I20260709 13:10:35.848505 27804 ts_tablet_manager.cc:1403] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.013s
I20260709 13:10:35.851742 27804 raft_consensus.cc:359] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:35.834074 27799 raft_consensus.cc:359] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.841955 27798 ts_tablet_manager.cc:1434] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:10:35.852883 27799 raft_consensus.cc:385] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.853946 27799 raft_consensus.cc:740] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.852494 27804 raft_consensus.cc:385] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.854713 27798 tablet_bootstrap.cc:492] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac: Bootstrap starting.
I20260709 13:10:35.854714 27799 consensus_queue.cc:260] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.857388 27804 raft_consensus.cc:740] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.858331 27804 consensus_queue.cc:260] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:35.858809 27709 heartbeater.cc:499] Master 127.26.83.254:34083 was elected leader, sending a full tablet report...
I20260709 13:10:35.861145 27799 ts_tablet_manager.cc:1434] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c: Time spent starting tablet: real 0.091s	user 0.008s	sys 0.002s
I20260709 13:10:35.862169 27799 tablet_bootstrap.cc:492] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c: Bootstrap starting.
I20260709 13:10:35.862164 27798 tablet_bootstrap.cc:654] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.869299 27799 tablet_bootstrap.cc:654] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.878650 27634 heartbeater.cc:499] Master 127.26.83.254:34083 was elected leader, sending a full tablet report...
I20260709 13:10:35.879101 27804 ts_tablet_manager.cc:1434] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7: Time spent starting tablet: real 0.030s	user 0.013s	sys 0.004s
I20260709 13:10:35.880134 27804 tablet_bootstrap.cc:492] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7: Bootstrap starting.
I20260709 13:10:35.882758 27798 tablet_bootstrap.cc:492] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac: No bootstrap required, opened a new log
I20260709 13:10:35.883416 27798 ts_tablet_manager.cc:1403] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent bootstrapping tablet: real 0.029s	user 0.022s	sys 0.002s
I20260709 13:10:35.886541 27798 raft_consensus.cc:359] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.887130 27804 tablet_bootstrap.cc:654] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.888212 27798 raft_consensus.cc:385] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.887850 27799 tablet_bootstrap.cc:492] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c: No bootstrap required, opened a new log
I20260709 13:10:35.888626 27798 raft_consensus.cc:740] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.889091 27799 ts_tablet_manager.cc:1403] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.006s
I20260709 13:10:35.889816 27798 consensus_queue.cc:260] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.892246 27798 ts_tablet_manager.cc:1434] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:35.892158 27799 raft_consensus.cc:359] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.893042 27799 raft_consensus.cc:385] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.893378 27799 raft_consensus.cc:740] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.893579 27798 tablet_bootstrap.cc:492] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac: Bootstrap starting.
I20260709 13:10:35.894136 27799 consensus_queue.cc:260] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.896646 27799 ts_tablet_manager.cc:1434] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:10:35.897650 27799 tablet_bootstrap.cc:492] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c: Bootstrap starting.
I20260709 13:10:35.903183 27798 tablet_bootstrap.cc:654] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.904196 27799 tablet_bootstrap.cc:654] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.940474 27804 tablet_bootstrap.cc:492] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7: No bootstrap required, opened a new log
I20260709 13:10:35.941049 27804 ts_tablet_manager.cc:1403] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.019s
I20260709 13:10:35.942474 27798 tablet_bootstrap.cc:492] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac: No bootstrap required, opened a new log
I20260709 13:10:35.943009 27798 ts_tablet_manager.cc:1403] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent bootstrapping tablet: real 0.050s	user 0.023s	sys 0.024s
I20260709 13:10:35.943102 27799 tablet_bootstrap.cc:492] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c: No bootstrap required, opened a new log
I20260709 13:10:35.943615 27799 ts_tablet_manager.cc:1403] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c: Time spent bootstrapping tablet: real 0.046s	user 0.033s	sys 0.008s
I20260709 13:10:35.944280 27807 raft_consensus.cc:493] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.944590 27804 raft_consensus.cc:359] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.944780 27807 raft_consensus.cc:515] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:35.945356 27804 raft_consensus.cc:385] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.945998 27804 raft_consensus.cc:740] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.946175 27798 raft_consensus.cc:359] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.946584 27799 raft_consensus.cc:359] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.946949 27798 raft_consensus.cc:385] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.947325 27799 raft_consensus.cc:385] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:35.946828 27804 consensus_queue.cc:260] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.947666 27798 raft_consensus.cc:740] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.947736 27799 raft_consensus.cc:740] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Initialized, Role: FOLLOWER
I20260709 13:10:35.948767 27798 consensus_queue.cc:260] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.948884 27799 consensus_queue.cc:260] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:35.951191 27798 ts_tablet_manager.cc:1434] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:10:35.951372 27799 ts_tablet_manager.cc:1434] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:35.952204 27798 tablet_bootstrap.cc:492] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac: Bootstrap starting.
I20260709 13:10:35.953612 27801 raft_consensus.cc:493] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.954069 27801 raft_consensus.cc:515] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:35.955143 27799 tablet_bootstrap.cc:492] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c: Bootstrap starting.
I20260709 13:10:35.958405 27807 leader_election.cc:290] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:35.959113 27798 tablet_bootstrap.cc:654] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.962303 27799 tablet_bootstrap.cc:654] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:35.971139 27801 leader_election.cc:290] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
W20260709 13:10:36.015763 27555 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:36.016031 27799 tablet_bootstrap.cc:492] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c: No bootstrap required, opened a new log
I20260709 13:10:36.016516 27799 ts_tablet_manager.cc:1403] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.008s
I20260709 13:10:36.019157 27798 tablet_bootstrap.cc:492] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac: No bootstrap required, opened a new log
I20260709 13:10:36.019655 27798 ts_tablet_manager.cc:1403] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent bootstrapping tablet: real 0.068s	user 0.018s	sys 0.025s
I20260709 13:10:36.019716 27799 raft_consensus.cc:359] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.023890 27798 raft_consensus.cc:359] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.020514 27799 raft_consensus.cc:385] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.024950 27798 raft_consensus.cc:385] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.026932 27798 raft_consensus.cc:740] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.026264 27799 raft_consensus.cc:740] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.028311 27798 consensus_queue.cc:260] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.029613 27799 consensus_queue.cc:260] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.030624 27798 ts_tablet_manager.cc:1434] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:10:36.031359 27798 tablet_bootstrap.cc:492] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac: Bootstrap starting.
I20260709 13:10:36.031198 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b5a991814a494c8e3ac259a6d66915" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:36.031869 27799 ts_tablet_manager.cc:1434] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.002s
I20260709 13:10:36.031834 27758 raft_consensus.cc:2468] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 0.
I20260709 13:10:36.032902 27799 tablet_bootstrap.cc:492] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c: Bootstrap starting.
I20260709 13:10:36.033234 27573 leader_election.cc:304] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:36.034142 27813 raft_consensus.cc:2804] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:36.035022 27813 raft_consensus.cc:493] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.035372 27813 raft_consensus.cc:3060] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.036375 27804 ts_tablet_manager.cc:1434] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7: Time spent starting tablet: real 0.095s	user 0.007s	sys 0.010s
I20260709 13:10:36.038635 27804 tablet_bootstrap.cc:492] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7: Bootstrap starting.
I20260709 13:10:36.042467 27813 raft_consensus.cc:515] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:36.045646 27813 leader_election.cc:290] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 election: Requested vote from peers 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:36.048696 27801 raft_consensus.cc:493] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.049307 27801 raft_consensus.cc:515] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.051257 27798 tablet_bootstrap.cc:654] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.051323 27801 leader_election.cc:290] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:36.053632 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b5a991814a494c8e3ac259a6d66915" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.054021 27801 raft_consensus.cc:493] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.054409 27804 tablet_bootstrap.cc:654] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.054497 27801 raft_consensus.cc:515] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.055109 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49c4ac4d6fa4ce18035f3cafb56340a" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
W20260709 13:10:36.056603 27722 leader_election.cc:343] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.057044 27801 leader_election.cc:290] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:36.056951 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b5a991814a494c8e3ac259a6d66915" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:36.057708 27758 raft_consensus.cc:3060] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.058465 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.059159 27683 raft_consensus.cc:2468] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 0.
I20260709 13:10:36.059784 27799 tablet_bootstrap.cc:654] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:36.057715 27722 leader_election.cc:343] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.060878 27722 leader_election.cc:304] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: 
I20260709 13:10:36.061719 27801 raft_consensus.cc:2804] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:36.062079 27801 raft_consensus.cc:493] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.062413 27801 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.066030 27758 raft_consensus.cc:2468] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 1.
I20260709 13:10:36.069906 27609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b5a991814a494c8e3ac259a6d66915" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.073266 27609 raft_consensus.cc:2393] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bc2e45f2cf84eeda313dee6e571a1ac in current term 1: Already voted for candidate b489ccf32e7542de8d18ba7e1730caf7 in this term.
I20260709 13:10:36.073526 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b5a991814a494c8e3ac259a6d66915" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.071349 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49c4ac4d6fa4ce18035f3cafb56340a" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.070185 27801 raft_consensus.cc:515] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.071506 27798 tablet_bootstrap.cc:492] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac: No bootstrap required, opened a new log
I20260709 13:10:36.072054 27607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.074739 27721 leader_election.cc:304] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac; no voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:36.074975 27798 ts_tablet_manager.cc:1403] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.000s
I20260709 13:10:36.073601 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b5a991814a494c8e3ac259a6d66915" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c"
W20260709 13:10:36.076400 27573 leader_election.cc:343] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:36.077386 27573 leader_election.cc:343] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.071502 27804 tablet_bootstrap.cc:492] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7: No bootstrap required, opened a new log
I20260709 13:10:36.078151 27804 ts_tablet_manager.cc:1403] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.000s
I20260709 13:10:36.078504 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:36.079098 27684 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.078775 27798 raft_consensus.cc:359] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.079833 27798 raft_consensus.cc:385] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.080263 27798 raft_consensus.cc:740] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.081063 27804 raft_consensus.cc:359] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.081779 27804 raft_consensus.cc:385] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.082055 27804 raft_consensus.cc:740] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.082520 27799 tablet_bootstrap.cc:492] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c: No bootstrap required, opened a new log
I20260709 13:10:36.083210 27799 ts_tablet_manager.cc:1403] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.008s
I20260709 13:10:36.082886 27804 consensus_queue.cc:260] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.085305 27804 ts_tablet_manager.cc:1434] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:36.087669 27684 raft_consensus.cc:2468] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 1.
I20260709 13:10:36.088487 27798 consensus_queue.cc:260] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.089284 27722 leader_election.cc:304] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: 
I20260709 13:10:36.087215 27799 raft_consensus.cc:359] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.091261 27799 raft_consensus.cc:385] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.091724 27799 raft_consensus.cc:740] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.094148 27825 raft_consensus.cc:2749] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:36.095350 27801 leader_election.cc:290] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:36.096408 27801 raft_consensus.cc:2804] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:36.097036 27798 ts_tablet_manager.cc:1434] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.001s
I20260709 13:10:36.097599 27801 raft_consensus.cc:697] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Becoming Leader. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:36.097936 27799 consensus_queue.cc:260] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.098935 27801 consensus_queue.cc:237] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.099438 27804 tablet_bootstrap.cc:492] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7: Bootstrap starting.
I20260709 13:10:36.104252 27798 tablet_bootstrap.cc:492] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac: Bootstrap starting.
I20260709 13:10:36.109869 27573 leader_election.cc:304] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 81de5334c2aa49909f114e12900b357c
I20260709 13:10:36.111092 27813 raft_consensus.cc:2804] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:36.111616 27813 raft_consensus.cc:697] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:36.112462 27813 consensus_queue.cc:237] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:36.115922 27804 tablet_bootstrap.cc:654] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.118414 27799 ts_tablet_manager.cc:1434] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.003s
I20260709 13:10:36.119167 27607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
W20260709 13:10:36.121254 27721 leader_election.cc:343] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.121759 27721 leader_election.cc:304] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac; no voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:36.122876 27798 tablet_bootstrap.cc:654] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.124017 27801 raft_consensus.cc:2749] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:36.126082 27804 tablet_bootstrap.cc:492] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7: No bootstrap required, opened a new log
W20260709 13:10:36.125545 27721 leader_election.cc:343] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.126708 27804 ts_tablet_manager.cc:1403] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.004s
W20260709 13:10:36.127365 27721 leader_election.cc:343] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.127640 27506 catalog_manager.cc:5697] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195). New cstate: current_term: 1 leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:36.129923 27799 tablet_bootstrap.cc:492] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c: Bootstrap starting.
I20260709 13:10:36.129951 27804 raft_consensus.cc:359] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.131021 27804 raft_consensus.cc:385] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.131385 27804 raft_consensus.cc:740] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.132201 27804 consensus_queue.cc:260] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.134764 27804 ts_tablet_manager.cc:1434] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:36.135818 27804 tablet_bootstrap.cc:492] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7: Bootstrap starting.
I20260709 13:10:36.136787 27799 tablet_bootstrap.cc:654] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.139876 27504 catalog_manager.cc:5697] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 1 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.142307 27804 tablet_bootstrap.cc:654] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.154896 27804 tablet_bootstrap.cc:492] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7: No bootstrap required, opened a new log
I20260709 13:10:36.155429 27804 ts_tablet_manager.cc:1403] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:10:36.158785 27804 raft_consensus.cc:359] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.159582 27804 raft_consensus.cc:385] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.159885 27804 raft_consensus.cc:740] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.160876 27804 consensus_queue.cc:260] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.163455 27804 ts_tablet_manager.cc:1434] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:10:36.164446 27804 tablet_bootstrap.cc:492] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7: Bootstrap starting.
I20260709 13:10:36.173519 27804 tablet_bootstrap.cc:654] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.181977 27798 tablet_bootstrap.cc:492] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac: No bootstrap required, opened a new log
I20260709 13:10:36.182595 27798 ts_tablet_manager.cc:1403] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent bootstrapping tablet: real 0.079s	user 0.027s	sys 0.028s
I20260709 13:10:36.183766 27799 tablet_bootstrap.cc:492] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c: No bootstrap required, opened a new log
I20260709 13:10:36.184289 27799 ts_tablet_manager.cc:1403] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.032s
I20260709 13:10:36.187168 27799 raft_consensus.cc:359] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:36.187918 27799 raft_consensus.cc:385] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.188192 27799 raft_consensus.cc:740] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.187079 27798 raft_consensus.cc:359] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.188931 27799 consensus_queue.cc:260] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:36.189538 27798 raft_consensus.cc:385] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.192358 27799 ts_tablet_manager.cc:1434] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:36.194705 27799 tablet_bootstrap.cc:492] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c: Bootstrap starting.
I20260709 13:10:36.193682 27798 raft_consensus.cc:740] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.200173 27806 raft_consensus.cc:493] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.200675 27806 raft_consensus.cc:515] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.203657 27806 leader_election.cc:290] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.205693 27798 consensus_queue.cc:260] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.213255 27798 ts_tablet_manager.cc:1434] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:10:36.215770 27799 tablet_bootstrap.cc:654] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.216149 27804 tablet_bootstrap.cc:492] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7: No bootstrap required, opened a new log
I20260709 13:10:36.216634 27804 ts_tablet_manager.cc:1403] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7: Time spent bootstrapping tablet: real 0.052s	user 0.022s	sys 0.019s
I20260709 13:10:36.218756 27798 tablet_bootstrap.cc:492] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac: Bootstrap starting.
I20260709 13:10:36.223045 27804 raft_consensus.cc:359] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.223866 27804 raft_consensus.cc:385] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.224195 27804 raft_consensus.cc:740] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.224404 27806 raft_consensus.cc:493] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.225031 27804 consensus_queue.cc:260] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.227540 27804 ts_tablet_manager.cc:1434] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:10:36.224979 27806 raft_consensus.cc:515] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.228572 27804 tablet_bootstrap.cc:492] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7: Bootstrap starting.
I20260709 13:10:36.230199 27798 tablet_bootstrap.cc:654] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.233701 27806 leader_election.cc:290] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.243445 27799 tablet_bootstrap.cc:492] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c: No bootstrap required, opened a new log
I20260709 13:10:36.250805 27804 tablet_bootstrap.cc:654] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.244336 27799 ts_tablet_manager.cc:1403] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.006s
I20260709 13:10:36.256011 27757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4163203b22b452985b9774ad4d7a2c9" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:36.256439 27798 tablet_bootstrap.cc:492] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac: No bootstrap required, opened a new log
I20260709 13:10:36.257207 27798 ts_tablet_manager.cc:1403] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.004s
W20260709 13:10:36.258138 27648 leader_election.cc:343] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.259706 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:36.261372 27798 raft_consensus.cc:359] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.262406 27798 raft_consensus.cc:385] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.262838 27798 raft_consensus.cc:740] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.261798 27758 raft_consensus.cc:2468] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 0.
I20260709 13:10:36.264091 27798 consensus_queue.cc:260] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.266008 27648 leader_election.cc:304] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: 
I20260709 13:10:36.266680 27798 ts_tablet_manager.cc:1434] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:10:36.266903 27806 raft_consensus.cc:2804] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:36.267221 27806 raft_consensus.cc:493] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.267560 27806 raft_consensus.cc:3060] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.267694 27607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.268607 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4163203b22b452985b9774ad4d7a2c9" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.275130 27607 raft_consensus.cc:2468] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 0.
I20260709 13:10:36.277797 27813 raft_consensus.cc:493] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.277619 27806 raft_consensus.cc:515] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.278074 27608 raft_consensus.cc:2468] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 0.
I20260709 13:10:36.278299 27813 raft_consensus.cc:515] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.279156 27799 raft_consensus.cc:359] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.279709 27647 leader_election.cc:304] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 1bc2e45f2cf84eeda313dee6e571a1ac
I20260709 13:10:36.280104 27799 raft_consensus.cc:385] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.280582 27799 raft_consensus.cc:740] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.281416 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3904a7744db64435a28bb973da2089f5" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.282562 27684 raft_consensus.cc:2468] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 0.
I20260709 13:10:36.283381 27804 tablet_bootstrap.cc:492] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7: No bootstrap required, opened a new log
I20260709 13:10:36.277369 27798 tablet_bootstrap.cc:492] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac: Bootstrap starting.
I20260709 13:10:36.281417 27813 leader_election.cc:290] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.283043 27799 consensus_queue.cc:260] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.283890 27573 leader_election.cc:304] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:36.284915 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3904a7744db64435a28bb973da2089f5" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:36.283890 27804 ts_tablet_manager.cc:1403] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.007s
I20260709 13:10:36.286090 27757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
W20260709 13:10:36.286377 27573 leader_election.cc:343] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.286747 27757 raft_consensus.cc:3060] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.288251 27804 raft_consensus.cc:359] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.289026 27804 raft_consensus.cc:385] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.289331 27804 raft_consensus.cc:740] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.290051 27804 consensus_queue.cc:260] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.290835 27798 tablet_bootstrap.cc:654] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.285249 27813 raft_consensus.cc:2804] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:36.291615 27813 raft_consensus.cc:493] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.291996 27813 raft_consensus.cc:3060] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.292006 27807 raft_consensus.cc:493] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.293361 27807 raft_consensus.cc:515] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.294873 27806 leader_election.cc:290] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.295668 27806 raft_consensus.cc:2804] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:36.296271 27806 raft_consensus.cc:493] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.296689 27806 raft_consensus.cc:3060] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.298717 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.299499 27684 raft_consensus.cc:2468] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 0.
I20260709 13:10:36.302150 27573 leader_election.cc:304] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:36.303318 27804 ts_tablet_manager.cc:1434] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 13:10:36.295604 27807 leader_election.cc:290] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.304379 27807 raft_consensus.cc:2804] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:36.305035 27807 raft_consensus.cc:493] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.305629 27807 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.311300 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:36.312153 27758 raft_consensus.cc:2468] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 0.
I20260709 13:10:36.312943 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:36.313623 27608 raft_consensus.cc:3060] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.314210 27804 tablet_bootstrap.cc:492] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7: Bootstrap starting.
I20260709 13:10:36.317886 27757 raft_consensus.cc:2468] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 1.
I20260709 13:10:36.322599 27806 raft_consensus.cc:515] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.325992 27799 ts_tablet_manager.cc:1434] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c: Time spent starting tablet: real 0.073s	user 0.005s	sys 0.002s
I20260709 13:10:36.333514 27799 tablet_bootstrap.cc:492] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c: Bootstrap starting.
I20260709 13:10:36.334610 27648 leader_election.cc:304] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: 
I20260709 13:10:36.327263 27806 leader_election.cc:290] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.328536 27607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4163203b22b452985b9774ad4d7a2c9" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:36.335791 27607 raft_consensus.cc:3060] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.338999 27804 tablet_bootstrap.cc:654] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.335862 27806 raft_consensus.cc:2804] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:36.339998 27806 raft_consensus.cc:697] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Becoming Leader. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:36.328657 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4163203b22b452985b9774ad4d7a2c9" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
W20260709 13:10:36.341606 27648 leader_election.cc:343] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.331880 27807 raft_consensus.cc:515] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.341153 27806 consensus_queue.cc:237] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.343708 27807 leader_election.cc:290] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 election: Requested vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.329473 27608 raft_consensus.cc:2468] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 1.
I20260709 13:10:36.332434 27813 raft_consensus.cc:515] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.344570 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:36.345366 27684 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.346879 27813 leader_election.cc:290] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 election: Requested vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.345355 27607 raft_consensus.cc:2468] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 1.
I20260709 13:10:36.350590 27647 leader_election.cc:304] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 1bc2e45f2cf84eeda313dee6e571a1ac
I20260709 13:10:36.351488 27806 raft_consensus.cc:2804] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:36.352324 27806 raft_consensus.cc:697] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Becoming Leader. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:36.355295 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:36.355660 27806 consensus_queue.cc:237] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.356271 27758 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.357002 27799 tablet_bootstrap.cc:654] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.361999 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3904a7744db64435a28bb973da2089f5" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:36.362824 27683 raft_consensus.cc:3060] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.364502 27684 raft_consensus.cc:2468] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 1.
I20260709 13:10:36.365701 27804 tablet_bootstrap.cc:492] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7: No bootstrap required, opened a new log
I20260709 13:10:36.366237 27804 ts_tablet_manager.cc:1403] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.005s
I20260709 13:10:36.367573 27798 tablet_bootstrap.cc:492] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac: No bootstrap required, opened a new log
I20260709 13:10:36.368016 27798 ts_tablet_manager.cc:1403] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent bootstrapping tablet: real 0.091s	user 0.031s	sys 0.026s
I20260709 13:10:36.368903 27804 raft_consensus.cc:359] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.369580 27804 raft_consensus.cc:385] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.369843 27804 raft_consensus.cc:740] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.370568 27798 raft_consensus.cc:359] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.370483 27804 consensus_queue.cc:260] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.371189 27798 raft_consensus.cc:385] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.371470 27798 raft_consensus.cc:740] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.372100 27798 consensus_queue.cc:260] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.372802 27804 ts_tablet_manager.cc:1434] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:10:36.373982 27804 tablet_bootstrap.cc:492] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7: Bootstrap starting.
I20260709 13:10:36.375422 27757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3904a7744db64435a28bb973da2089f5" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:36.376981 27806 raft_consensus.cc:493] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.377594 27806 raft_consensus.cc:515] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.380323 27798 ts_tablet_manager.cc:1434] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:10:36.381255 27804 tablet_bootstrap.cc:654] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.382378 27806 leader_election.cc:290] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
W20260709 13:10:36.387321 27573 leader_election.cc:343] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.387902 27804 tablet_bootstrap.cc:492] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7: No bootstrap required, opened a new log
I20260709 13:10:36.388266 27573 leader_election.cc:304] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:36.389241 27813 raft_consensus.cc:2804] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:36.389755 27813 raft_consensus.cc:697] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:36.390245 27798 tablet_bootstrap.cc:492] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac: Bootstrap starting.
I20260709 13:10:36.390264 27607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15983c9724104e7c86e9ea2e6ca0e7ff" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.390533 27813 consensus_queue.cc:237] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
W20260709 13:10:36.391901 27647 leader_election.cc:343] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.388446 27804 ts_tablet_manager.cc:1403] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:10:36.390120 27757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15983c9724104e7c86e9ea2e6ca0e7ff" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:36.394891 27757 raft_consensus.cc:2468] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 0.
I20260709 13:10:36.396509 27648 leader_election.cc:304] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:36.397532 27798 tablet_bootstrap.cc:654] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.397275 27804 raft_consensus.cc:359] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.397826 27806 raft_consensus.cc:2804] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:36.398257 27804 raft_consensus.cc:385] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.398394 27806 raft_consensus.cc:493] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.398702 27804 raft_consensus.cc:740] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.398922 27806 raft_consensus.cc:3060] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.399689 27804 consensus_queue.cc:260] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.401274 27801 raft_consensus.cc:493] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.402105 27801 raft_consensus.cc:515] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.403399 27758 raft_consensus.cc:2468] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 1.
I20260709 13:10:36.405339 27825 raft_consensus.cc:493] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.405903 27607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.406581 27607 raft_consensus.cc:2468] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 0.
I20260709 13:10:36.406955 27683 raft_consensus.cc:2468] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 1.
I20260709 13:10:36.407902 27721 leader_election.cc:304] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:36.408921 27824 raft_consensus.cc:2804] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:36.409405 27804 ts_tablet_manager.cc:1434] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:10:36.409324 27824 raft_consensus.cc:493] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.409785 27824 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.410398 27804 tablet_bootstrap.cc:492] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7: Bootstrap starting.
I20260709 13:10:36.406059 27825 raft_consensus.cc:515] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.414112 27573 leader_election.cc:304] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 1bc2e45f2cf84eeda313dee6e571a1ac
I20260709 13:10:36.415820 27813 raft_consensus.cc:2804] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:36.415251 27505 catalog_manager.cc:5697] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c reported cstate change: term changed from 0 to 1, leader changed from <none> to 81de5334c2aa49909f114e12900b357c (127.26.83.194). New cstate: current_term: 1 leader_uuid: "81de5334c2aa49909f114e12900b357c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.416173 27607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681a55081a294370909a44c09d683d34" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.416592 27813 raft_consensus.cc:697] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:36.416851 27607 raft_consensus.cc:2468] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 0.
I20260709 13:10:36.417196 27505 catalog_manager.cc:5697] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c reported cstate change: term changed from 0 to 1, leader changed from <none> to 81de5334c2aa49909f114e12900b357c (127.26.83.194). New cstate: current_term: 1 leader_uuid: "81de5334c2aa49909f114e12900b357c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.418365 27721 leader_election.cc:304] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:36.417870 27813 consensus_queue.cc:237] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.419158 27826 raft_consensus.cc:2804] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:36.414492 27825 leader_election.cc:290] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:36.419560 27826 raft_consensus.cc:493] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.420109 27826 raft_consensus.cc:3060] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.421185 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681a55081a294370909a44c09d683d34" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.421522 27807 raft_consensus.cc:493] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.422065 27807 raft_consensus.cc:515] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
W20260709 13:10:36.422591 27722 leader_election.cc:343] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.425846 27804 tablet_bootstrap.cc:654] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.426800 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:36.430539 27824 raft_consensus.cc:515] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.431927 27801 leader_election.cc:290] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:36.424571 27807 leader_election.cc:290] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.434216 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.435911 27607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:36.436779 27607 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.440308 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.441322 27683 raft_consensus.cc:2393] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b489ccf32e7542de8d18ba7e1730caf7 in current term 1: Already voted for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in this term.
W20260709 13:10:36.441483 27722 leader_election.cc:343] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.441411 27806 raft_consensus.cc:515] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.443017 27573 leader_election.cc:304] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b489ccf32e7542de8d18ba7e1730caf7; no voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c
I20260709 13:10:36.443959 27807 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.443913 27826 raft_consensus.cc:515] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.447811 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c"
W20260709 13:10:36.449855 27722 leader_election.cc:343] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.450276 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15983c9724104e7c86e9ea2e6ca0e7ff" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:36.452652 27824 leader_election.cc:290] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
W20260709 13:10:36.452529 27647 leader_election.cc:343] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.454790 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681a55081a294370909a44c09d683d34" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:36.455466 27608 raft_consensus.cc:3060] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.455626 27806 leader_election.cc:290] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.456001 27799 tablet_bootstrap.cc:492] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c: No bootstrap required, opened a new log
I20260709 13:10:36.452533 27807 raft_consensus.cc:2749] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:36.456921 27799 ts_tablet_manager.cc:1403] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c: Time spent bootstrapping tablet: real 0.124s	user 0.020s	sys 0.019s
I20260709 13:10:36.456928 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15983c9724104e7c86e9ea2e6ca0e7ff" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:36.457687 27758 raft_consensus.cc:3060] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.460232 27799 raft_consensus.cc:359] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.461040 27799 raft_consensus.cc:385] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.461373 27799 raft_consensus.cc:740] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.462406 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681a55081a294370909a44c09d683d34" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:36.462263 27799 consensus_queue.cc:260] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
W20260709 13:10:36.463861 27722 leader_election.cc:343] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:36.464540 27799 ts_tablet_manager.cc:1434] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:36.465791 27608 raft_consensus.cc:2468] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 1.
I20260709 13:10:36.456923 27826 leader_election.cc:290] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:36.466444 27506 catalog_manager.cc:5697] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 1 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.468086 27799 tablet_bootstrap.cc:492] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c: Bootstrap starting.
I20260709 13:10:36.469437 27758 raft_consensus.cc:2468] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 1.
I20260709 13:10:36.468827 27506 catalog_manager.cc:5697] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 1 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.470980 27648 leader_election.cc:304] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:36.471563 27721 leader_election.cc:304] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 81de5334c2aa49909f114e12900b357c
I20260709 13:10:36.472241 27806 raft_consensus.cc:2804] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:36.472859 27806 raft_consensus.cc:697] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Becoming Leader. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:36.473769 27806 consensus_queue.cc:237] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.474774 27826 raft_consensus.cc:2804] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:36.475068 27799 tablet_bootstrap.cc:654] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.475257 27826 raft_consensus.cc:697] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Becoming Leader. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:36.476190 27826 consensus_queue.cc:237] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.452538 27607 raft_consensus.cc:2468] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 1.
I20260709 13:10:36.478837 27721 leader_election.cc:304] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 81de5334c2aa49909f114e12900b357c
I20260709 13:10:36.479979 27824 raft_consensus.cc:2804] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:36.480432 27824 raft_consensus.cc:697] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Becoming Leader. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:36.481268 27824 consensus_queue.cc:237] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.462591 27798 tablet_bootstrap.cc:492] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac: No bootstrap required, opened a new log
I20260709 13:10:36.483374 27798 ts_tablet_manager.cc:1403] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent bootstrapping tablet: real 0.093s	user 0.017s	sys 0.016s
I20260709 13:10:36.487241 27804 tablet_bootstrap.cc:492] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7: No bootstrap required, opened a new log
I20260709 13:10:36.487078 27798 raft_consensus.cc:359] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.487726 27804 ts_tablet_manager.cc:1403] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7: Time spent bootstrapping tablet: real 0.078s	user 0.008s	sys 0.008s
I20260709 13:10:36.487766 27798 raft_consensus.cc:385] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.488161 27798 raft_consensus.cc:740] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.488830 27798 consensus_queue.cc:260] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.490708 27804 raft_consensus.cc:359] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.491204 27798 ts_tablet_manager.cc:1434] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:36.491448 27804 raft_consensus.cc:385] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.491777 27804 raft_consensus.cc:740] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.492980 27506 catalog_manager.cc:5697] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c reported cstate change: term changed from 0 to 1, leader changed from <none> to 81de5334c2aa49909f114e12900b357c (127.26.83.194). New cstate: current_term: 1 leader_uuid: "81de5334c2aa49909f114e12900b357c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.492422 27804 consensus_queue.cc:260] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.498610 27505 catalog_manager.cc:5697] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195). New cstate: current_term: 1 leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:36.500419 27804 ts_tablet_manager.cc:1434] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:10:36.502486 27799 tablet_bootstrap.cc:492] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c: No bootstrap required, opened a new log
I20260709 13:10:36.502995 27799 ts_tablet_manager.cc:1403] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.003s
I20260709 13:10:36.505637 27799 raft_consensus.cc:359] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.506163 27799 raft_consensus.cc:385] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.506412 27799 raft_consensus.cc:740] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.507046 27799 consensus_queue.cc:260] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.509343 27799 ts_tablet_manager.cc:1434] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:10:36.510257 27799 tablet_bootstrap.cc:492] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c: Bootstrap starting.
I20260709 13:10:36.516069 27799 tablet_bootstrap.cc:654] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:36.517892 27506 catalog_manager.cc:5697] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195). New cstate: current_term: 1 leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:36.521363 27799 tablet_bootstrap.cc:492] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c: No bootstrap required, opened a new log
I20260709 13:10:36.521770 27799 ts_tablet_manager.cc:1403] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c: Time spent bootstrapping tablet: real 0.012s	user 0.002s	sys 0.007s
I20260709 13:10:36.523710 27799 raft_consensus.cc:359] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.524235 27799 raft_consensus.cc:385] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:36.524492 27799 raft_consensus.cc:740] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Initialized, Role: FOLLOWER
I20260709 13:10:36.525103 27799 consensus_queue.cc:260] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.526892 27799 ts_tablet_manager.cc:1434] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:10:36.553707 27807 raft_consensus.cc:493] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.554251 27807 raft_consensus.cc:515] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.556087 27807 leader_election.cc:290] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.557202 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49c4ac4d6fa4ce18035f3cafb56340a" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.557395 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49c4ac4d6fa4ce18035f3cafb56340a" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:36.557900 27683 raft_consensus.cc:2468] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 0.
I20260709 13:10:36.557901 27758 raft_consensus.cc:2468] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 0.
I20260709 13:10:36.559217 27573 leader_election.cc:304] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:36.559922 27807 raft_consensus.cc:2804] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:36.560307 27807 raft_consensus.cc:493] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.560601 27807 raft_consensus.cc:3060] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.565376 27807 raft_consensus.cc:515] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.567397 27807 leader_election.cc:290] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 election: Requested vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.568193 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49c4ac4d6fa4ce18035f3cafb56340a" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:36.568434 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c49c4ac4d6fa4ce18035f3cafb56340a" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:36.568763 27683 raft_consensus.cc:3060] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.569099 27758 raft_consensus.cc:3060] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.574510 27758 raft_consensus.cc:2468] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 1.
I20260709 13:10:36.574625 27683 raft_consensus.cc:2468] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 1.
I20260709 13:10:36.575659 27573 leader_election.cc:304] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:36.576475 27807 raft_consensus.cc:2804] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:36.576963 27807 raft_consensus.cc:697] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:36.577761 27807 consensus_queue.cc:237] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.585248 27507 catalog_manager.cc:5697] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 1 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.586844 27683 raft_consensus.cc:3060] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.594841 27807 consensus_queue.cc:1048] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.609603 27845 consensus_queue.cc:1048] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:36.612694 27824 consensus_queue.cc:1048] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.613664 27826 raft_consensus.cc:493] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.614142 27826 raft_consensus.cc:515] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.616379 27826 leader_election.cc:290] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:36.617864 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4163203b22b452985b9774ad4d7a2c9" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.618418 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4163203b22b452985b9774ad4d7a2c9" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.620044 27824 raft_consensus.cc:493] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.622509 27608 raft_consensus.cc:2393] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bc2e45f2cf84eeda313dee6e571a1ac in current term 1: Already voted for candidate 81de5334c2aa49909f114e12900b357c in this term.
I20260709 13:10:36.622275 27824 raft_consensus.cc:515] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.624336 27721 leader_election.cc:304] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac; no voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:36.624903 27824 leader_election.cc:290] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:36.625396 27824 raft_consensus.cc:3060] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.626632 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3904a7744db64435a28bb973da2089f5" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.628930 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3904a7744db64435a28bb973da2089f5" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.629981 27684 raft_consensus.cc:2393] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1bc2e45f2cf84eeda313dee6e571a1ac in current term 1: Already voted for candidate b489ccf32e7542de8d18ba7e1730caf7 in this term.
I20260709 13:10:36.631516 27824 raft_consensus.cc:2749] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:36.631417 27722 leader_election.cc:304] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac; no voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:36.632292 27826 raft_consensus.cc:3060] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.641798 27826 raft_consensus.cc:2749] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:36.643626 27826 consensus_queue.cc:1048] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.679109 27806 raft_consensus.cc:493] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.679723 27806 raft_consensus.cc:515] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.681679 27806 leader_election.cc:290] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.682874 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681a55081a294370909a44c09d683d34" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:36.683318 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681a55081a294370909a44c09d683d34" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:36.683768 27608 raft_consensus.cc:2393] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 81de5334c2aa49909f114e12900b357c in current term 1: Already voted for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in this term.
I20260709 13:10:36.685106 27647 leader_election.cc:304] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81de5334c2aa49909f114e12900b357c; no voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:36.686008 27806 raft_consensus.cc:3060] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.690477 27758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0d3499349e2425798b193e641999c0e"
dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
mode: GRACEFUL
new_leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:53900
I20260709 13:10:36.691130 27758 raft_consensus.cc:606] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Received request to transfer leadership to TS b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:36.692174 27806 raft_consensus.cc:2749] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:36.705298 27683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4163203b22b452985b9774ad4d7a2c9"
dest_uuid: "81de5334c2aa49909f114e12900b357c"
mode: GRACEFUL
new_leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:43978
I20260709 13:10:36.705912 27683 raft_consensus.cc:606] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Received request to transfer leadership to TS b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:36.707213 27845 raft_consensus.cc:493] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.707811 27845 raft_consensus.cc:515] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.709832 27845 leader_election.cc:290] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.710896 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15983c9724104e7c86e9ea2e6ca0e7ff" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:36.711021 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15983c9724104e7c86e9ea2e6ca0e7ff" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81de5334c2aa49909f114e12900b357c" is_pre_election: true
I20260709 13:10:36.711717 27758 raft_consensus.cc:2393] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b489ccf32e7542de8d18ba7e1730caf7 in current term 1: Already voted for candidate 81de5334c2aa49909f114e12900b357c in this term.
I20260709 13:10:36.712659 27573 leader_election.cc:304] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b489ccf32e7542de8d18ba7e1730caf7; no voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c
I20260709 13:10:36.713375 27845 raft_consensus.cc:3060] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:36.718117 27758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d"
dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
mode: GRACEFUL
new_leader_uuid: "81de5334c2aa49909f114e12900b357c"
 from {username='slave'} at 127.0.0.1:53900
I20260709 13:10:36.718715 27758 raft_consensus.cc:606] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Received request to transfer leadership to TS 81de5334c2aa49909f114e12900b357c
I20260709 13:10:36.718958 27845 raft_consensus.cc:2749] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:36.732030 27608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3904a7744db64435a28bb973da2089f5"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
mode: GRACEFUL
new_leader_uuid: "81de5334c2aa49909f114e12900b357c"
 from {username='slave'} at 127.0.0.1:40200
I20260709 13:10:36.732683 27608 raft_consensus.cc:606] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Received request to transfer leadership to TS 81de5334c2aa49909f114e12900b357c
I20260709 13:10:36.740810 27758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "681a55081a294370909a44c09d683d34"
dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
mode: GRACEFUL
new_leader_uuid: "81de5334c2aa49909f114e12900b357c"
 from {username='slave'} at 127.0.0.1:53900
I20260709 13:10:36.741288 27758 raft_consensus.cc:606] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Received request to transfer leadership to TS 81de5334c2aa49909f114e12900b357c
I20260709 13:10:36.746491 27683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15983c9724104e7c86e9ea2e6ca0e7ff"
dest_uuid: "81de5334c2aa49909f114e12900b357c"
mode: GRACEFUL
new_leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
 from {username='slave'} at 127.0.0.1:43978
I20260709 13:10:36.746712 27806 consensus_queue.cc:1048] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:36.747241 27683 raft_consensus.cc:606] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Received request to transfer leadership to TS 1bc2e45f2cf84eeda313dee6e571a1ac
I20260709 13:10:36.760775 27806 consensus_queue.cc:1048] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.769433 27843 raft_consensus.cc:993] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c: : Instructing follower b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:36.769870 27806 raft_consensus.cc:1081] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Signalling peer b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:36.771497 27608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4163203b22b452985b9774ad4d7a2c9"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:40192
I20260709 13:10:36.772073 27608 raft_consensus.cc:493] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:36.772451 27608 raft_consensus.cc:3060] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.777374 27608 raft_consensus.cc:515] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.779335 27608 leader_election.cc:290] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 2 election: Requested vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:36.780125 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4163203b22b452985b9774ad4d7a2c9" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:36.780537 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4163203b22b452985b9774ad4d7a2c9" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:36.780718 27683 raft_consensus.cc:3055] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:36.781114 27758 raft_consensus.cc:3060] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.781080 27683 raft_consensus.cc:740] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:36.782120 27683 consensus_queue.cc:260] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.783166 27683 raft_consensus.cc:3060] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.787781 27758 raft_consensus.cc:2468] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 2.
I20260709 13:10:36.789083 27573 leader_election.cc:304] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:36.789693 27683 raft_consensus.cc:2468] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 2.
I20260709 13:10:36.789893 27845 raft_consensus.cc:2804] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:36.790450 27845 raft_consensus.cc:697] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:36.791172 27845 consensus_queue.cc:237] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.798146 27506 catalog_manager.cc:5697] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 1 to 2, leader changed from 81de5334c2aa49909f114e12900b357c (127.26.83.194) to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 2 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:36.844607 27806 consensus_queue.cc:1048] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.852427 27843 consensus_queue.cc:1048] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.900992 27871 consensus_queue.cc:1048] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.911428 27806 consensus_queue.cc:1048] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.920173 27843 raft_consensus.cc:993] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c: : Instructing follower 1bc2e45f2cf84eeda313dee6e571a1ac to start an election
I20260709 13:10:36.920627 27871 raft_consensus.cc:1081] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Signalling peer 1bc2e45f2cf84eeda313dee6e571a1ac to start an election
I20260709 13:10:36.921350 27845 consensus_queue.cc:1048] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.922397 27758 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15983c9724104e7c86e9ea2e6ca0e7ff"
dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
 from {username='slave'} at 127.0.0.1:53886
I20260709 13:10:36.923081 27758 raft_consensus.cc:493] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:36.923493 27758 raft_consensus.cc:3060] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.928210 27813 consensus_queue.cc:1048] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.929705 27807 consensus_queue.cc:1048] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.933516 27758 raft_consensus.cc:515] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.936385 27758 leader_election.cc:290] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 2 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:36.937186 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15983c9724104e7c86e9ea2e6ca0e7ff" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:36.937856 27608 raft_consensus.cc:3060] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.941968 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15983c9724104e7c86e9ea2e6ca0e7ff" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:36.942646 27684 raft_consensus.cc:3055] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:36.942975 27684 raft_consensus.cc:740] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:36.943892 27684 consensus_queue.cc:260] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.945261 27684 raft_consensus.cc:3060] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.950549 27608 raft_consensus.cc:2468] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 2.
I20260709 13:10:36.956353 27684 raft_consensus.cc:2468] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 2.
I20260709 13:10:36.957357 27721 leader_election.cc:304] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:36.958235 27826 raft_consensus.cc:2804] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:36.958421 27845 consensus_queue.cc:1048] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:36.958750 27826 raft_consensus.cc:697] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 LEADER]: Becoming Leader. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:36.959618 27826 consensus_queue.cc:237] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.970351 27845 raft_consensus.cc:993] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7: : Instructing follower 81de5334c2aa49909f114e12900b357c to start an election
I20260709 13:10:36.970845 27813 raft_consensus.cc:1081] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Signalling peer 81de5334c2aa49909f114e12900b357c to start an election
I20260709 13:10:36.973121 27683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3904a7744db64435a28bb973da2089f5"
dest_uuid: "81de5334c2aa49909f114e12900b357c"
 from {username='slave'} at 127.0.0.1:43972
I20260709 13:10:36.973707 27683 raft_consensus.cc:493] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:36.974234 27683 raft_consensus.cc:3060] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.973744 27506 catalog_manager.cc:5697] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac reported cstate change: term changed from 1 to 2, leader changed from 81de5334c2aa49909f114e12900b357c (127.26.83.194) to 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195). New cstate: current_term: 2 leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:36.988682 27683 raft_consensus.cc:515] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.990684 27871 raft_consensus.cc:493] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.991204 27871 raft_consensus.cc:515] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.991428 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3904a7744db64435a28bb973da2089f5" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:36.992664 27608 raft_consensus.cc:3055] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:36.993036 27608 raft_consensus.cc:740] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:36.993611 27757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3904a7744db64435a28bb973da2089f5" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:36.994264 27757 raft_consensus.cc:3060] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:36.993872 27608 consensus_queue.cc:260] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:36.995004 27608 raft_consensus.cc:3060] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.000043 27871 leader_election.cc:290] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:37.000699 27607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7" is_pre_election: true
I20260709 13:10:37.001525 27607 raft_consensus.cc:2393] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 81de5334c2aa49909f114e12900b357c in current term 1: Already voted for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in this term.
I20260709 13:10:37.001961 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" is_pre_election: true
I20260709 13:10:37.002218 27608 raft_consensus.cc:2468] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 2.
I20260709 13:10:37.002755 27683 leader_election.cc:290] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 2 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:37.003877 27647 leader_election.cc:304] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:37.003872 27757 raft_consensus.cc:2468] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 2.
I20260709 13:10:37.004801 27871 raft_consensus.cc:2804] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:37.005249 27648 leader_election.cc:304] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81de5334c2aa49909f114e12900b357c; no voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:37.005347 27871 raft_consensus.cc:697] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Becoming Leader. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:37.006119 27843 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:37.006650 27871 consensus_queue.cc:237] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:37.013664 27843 raft_consensus.cc:2749] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:37.015208 27825 consensus_queue.cc:1048] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:37.015115 27507 catalog_manager.cc:5697] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c reported cstate change: term changed from 1 to 2, leader changed from b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193) to 81de5334c2aa49909f114e12900b357c (127.26.83.194). New cstate: current_term: 2 leader_uuid: "81de5334c2aa49909f114e12900b357c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:37.017443 27825 consensus_queue.cc:1048] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:37.023759 27826 raft_consensus.cc:993] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac: : Instructing follower b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:37.024385 27824 raft_consensus.cc:1081] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Signalling peer b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:37.027257 27607 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0d3499349e2425798b193e641999c0e"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:40182
I20260709 13:10:37.033794 27607 raft_consensus.cc:493] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:37.034273 27607 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.054136 27607 raft_consensus.cc:515] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:37.055222 27824 consensus_queue.cc:1048] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:37.062224 27607 leader_election.cc:290] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 2 election: Requested vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:37.063831 27757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:37.064594 27757 raft_consensus.cc:3055] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:37.065026 27757 raft_consensus.cc:740] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:37.065584 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:37.066215 27684 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.065907 27757 consensus_queue.cc:260] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:37.067130 27757 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.072399 27825 raft_consensus.cc:993] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac: : Instructing follower 81de5334c2aa49909f114e12900b357c to start an election
I20260709 13:10:37.072932 27825 raft_consensus.cc:1081] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Signalling peer 81de5334c2aa49909f114e12900b357c to start an election
I20260709 13:10:37.073467 27684 raft_consensus.cc:2468] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 2.
I20260709 13:10:37.074719 27573 leader_election.cc:304] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:37.075533 27845 raft_consensus.cc:2804] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:37.076021 27845 raft_consensus.cc:697] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:37.076913 27683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "681a55081a294370909a44c09d683d34"
dest_uuid: "81de5334c2aa49909f114e12900b357c"
 from {username='slave'} at 127.0.0.1:43964
I20260709 13:10:37.076861 27845 consensus_queue.cc:237] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:37.077591 27683 raft_consensus.cc:493] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:37.078047 27683 raft_consensus.cc:3060] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.083735 27824 consensus_queue.cc:1048] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:37.085502 27683 raft_consensus.cc:515] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:37.086221 27757 raft_consensus.cc:2468] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 2.
I20260709 13:10:37.087319 27505 catalog_manager.cc:5697] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 1 to 2, leader changed from 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195) to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 2 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:37.089362 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681a55081a294370909a44c09d683d34" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:37.091041 27757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "681a55081a294370909a44c09d683d34" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:37.091931 27757 raft_consensus.cc:3055] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:37.092490 27683 leader_election.cc:290] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 2 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:37.092989 27757 raft_consensus.cc:740] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:37.094796 27757 consensus_queue.cc:260] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:37.096306 27757 raft_consensus.cc:3060] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.103564 27757 raft_consensus.cc:2468] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 2.
I20260709 13:10:37.105418 27648 leader_election.cc:304] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:37.106302 27843 raft_consensus.cc:2804] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:37.106840 27843 raft_consensus.cc:697] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Becoming Leader. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:37.107534 27825 raft_consensus.cc:993] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac: : Instructing follower 81de5334c2aa49909f114e12900b357c to start an election
I20260709 13:10:37.107981 27824 raft_consensus.cc:1081] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Signalling peer 81de5334c2aa49909f114e12900b357c to start an election
I20260709 13:10:37.107733 27843 consensus_queue.cc:237] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:37.109879 27683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d"
dest_uuid: "81de5334c2aa49909f114e12900b357c"
 from {username='slave'} at 127.0.0.1:43964
I20260709 13:10:37.110612 27683 raft_consensus.cc:493] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:37.110991 27683 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.117904 27683 raft_consensus.cc:515] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:37.117718 27505 catalog_manager.cc:5697] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c reported cstate change: term changed from 1 to 2, leader changed from 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195) to 81de5334c2aa49909f114e12900b357c (127.26.83.194). New cstate: current_term: 2 leader_uuid: "81de5334c2aa49909f114e12900b357c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:37.120210 27683 leader_election.cc:290] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 2 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:37.121022 27607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:37.121678 27607 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.124756 27757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:37.125412 27757 raft_consensus.cc:3055] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:37.125916 27757 raft_consensus.cc:740] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:37.126847 27757 consensus_queue.cc:260] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:37.128331 27757 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.129731 27607 raft_consensus.cc:2468] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 2.
I20260709 13:10:37.131084 27647 leader_election.cc:304] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:37.131930 27843 raft_consensus.cc:2804] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:37.132367 27843 raft_consensus.cc:697] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Becoming Leader. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:37.133143 27843 consensus_queue.cc:237] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:37.135141 27757 raft_consensus.cc:2468] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 2.
I20260709 13:10:37.139367 27505 catalog_manager.cc:5697] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c reported cstate change: term changed from 1 to 2, leader changed from 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195) to 81de5334c2aa49909f114e12900b357c (127.26.83.194). New cstate: current_term: 2 leader_uuid: "81de5334c2aa49909f114e12900b357c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:37.172777 27845 consensus_queue.cc:1048] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:37.184291 27813 consensus_queue.cc:1048] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:37.225715 27683 raft_consensus.cc:1275] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.227228 27813 consensus_queue.cc:1048] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.236030 27757 raft_consensus.cc:1275] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.237814 27881 consensus_queue.cc:1048] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.354152 27683 raft_consensus.cc:1275] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Refusing update from remote peer 1bc2e45f2cf84eeda313dee6e571a1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.355628 27824 consensus_queue.cc:1048] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.363915 27607 raft_consensus.cc:1275] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Refusing update from remote peer 1bc2e45f2cf84eeda313dee6e571a1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.366068 27825 consensus_queue.cc:1048] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.396001 27607 raft_consensus.cc:1275] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Refusing update from remote peer 81de5334c2aa49909f114e12900b357c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.397545 27843 consensus_queue.cc:1048] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.405781 27757 raft_consensus.cc:1275] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Refusing update from remote peer 81de5334c2aa49909f114e12900b357c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.407573 27806 consensus_queue.cc:1048] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.493443 27757 raft_consensus.cc:1275] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Refusing update from remote peer 81de5334c2aa49909f114e12900b357c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.494957 27806 consensus_queue.cc:1048] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.504457 27607 raft_consensus.cc:3060] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:37.512312 27607 raft_consensus.cc:1275] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Refusing update from remote peer 81de5334c2aa49909f114e12900b357c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.513583 27871 consensus_queue.cc:1048] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.547343 27683 raft_consensus.cc:1275] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.548625 27881 consensus_queue.cc:1048] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.556573 27607 raft_consensus.cc:1275] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Refusing update from remote peer 81de5334c2aa49909f114e12900b357c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.557914 27757 raft_consensus.cc:1275] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.558069 27871 consensus_queue.cc:1048] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.559648 27881 consensus_queue.cc:1048] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:37.567214 27758 raft_consensus.cc:1275] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Refusing update from remote peer 81de5334c2aa49909f114e12900b357c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:37.568851 27889 consensus_queue.cc:1048] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:39.785615 26959 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:10:39.786623 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:39.803642 27608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
mode: GRACEFUL
new_leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
 from {username='slave'} at 127.0.0.1:40208
I20260709 13:10:39.804272 27608 raft_consensus.cc:606] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Received request to transfer leadership to TS 1bc2e45f2cf84eeda313dee6e571a1ac
I20260709 13:10:39.807331 27608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0d3499349e2425798b193e641999c0e"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
mode: GRACEFUL
new_leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
 from {username='slave'} at 127.0.0.1:40208
I20260709 13:10:39.807852 27608 raft_consensus.cc:606] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Received request to transfer leadership to TS 1bc2e45f2cf84eeda313dee6e571a1ac
I20260709 13:10:39.808947 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:39.809412 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:39.987275 27845 raft_consensus.cc:993] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7: : Instructing follower 1bc2e45f2cf84eeda313dee6e571a1ac to start an election
I20260709 13:10:39.987851 27845 raft_consensus.cc:1081] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Signalling peer 1bc2e45f2cf84eeda313dee6e571a1ac to start an election
I20260709 13:10:39.989543 27757 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388"
dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
 from {username='slave'} at 127.0.0.1:53882
I20260709 13:10:39.990036 27757 raft_consensus.cc:493] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:39.990490 27757 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:39.995430 27757 raft_consensus.cc:515] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:39.997514 27757 leader_election.cc:290] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 2 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:39.998401 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:39.998543 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:39.999189 27683 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:39.999176 27608 raft_consensus.cc:3055] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:39.999691 27608 raft_consensus.cc:740] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:40.000392 27608 consensus_queue.cc:260] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:40.001477 27608 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:40.004014 27683 raft_consensus.cc:2468] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 2.
I20260709 13:10:40.005245 27722 leader_election.cc:304] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: 
I20260709 13:10:40.005976 27892 raft_consensus.cc:2804] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:40.007114 27892 raft_consensus.cc:697] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 LEADER]: Becoming Leader. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:40.007153 27608 raft_consensus.cc:2468] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 2.
I20260709 13:10:40.008270 27892 consensus_queue.cc:237] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:40.017299 27504 catalog_manager.cc:5697] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac reported cstate change: term changed from 1 to 2, leader changed from b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193) to 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195). New cstate: current_term: 2 leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:40.239683 27894 raft_consensus.cc:993] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7: : Instructing follower 1bc2e45f2cf84eeda313dee6e571a1ac to start an election
I20260709 13:10:40.240998 27845 raft_consensus.cc:1081] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Signalling peer 1bc2e45f2cf84eeda313dee6e571a1ac to start an election
I20260709 13:10:40.242941 27758 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0d3499349e2425798b193e641999c0e"
dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
 from {username='slave'} at 127.0.0.1:53882
I20260709 13:10:40.243566 27758 raft_consensus.cc:493] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.243875 27758 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:40.248383 27758 raft_consensus.cc:515] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:40.250239 27758 leader_election.cc:290] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 3 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:40.251288 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:40.251490 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:40.251830 27608 raft_consensus.cc:3055] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:40.251991 27683 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:40.252167 27608 raft_consensus.cc:740] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:40.252955 27608 consensus_queue.cc:260] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:40.253860 27608 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:40.256323 27683 raft_consensus.cc:2468] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 3.
I20260709 13:10:40.257404 27722 leader_election.cc:304] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: 
I20260709 13:10:40.258101 27892 raft_consensus.cc:2804] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:40.258512 27892 raft_consensus.cc:697] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 LEADER]: Becoming Leader. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:40.258579 27608 raft_consensus.cc:2468] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 3.
I20260709 13:10:40.259299 27892 consensus_queue.cc:237] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:40.266435 27504 catalog_manager.cc:5697] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac reported cstate change: term changed from 2 to 3, leader changed from b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193) to 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195). New cstate: current_term: 3 leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:40.573539 27683 raft_consensus.cc:1275] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Refusing update from remote peer 1bc2e45f2cf84eeda313dee6e571a1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.574893 27892 consensus_queue.cc:1048] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.588855 27608 raft_consensus.cc:1275] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Refusing update from remote peer 1bc2e45f2cf84eeda313dee6e571a1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:40.590351 27892 consensus_queue.cc:1048] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:40.653837 27683 raft_consensus.cc:1275] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Refusing update from remote peer 1bc2e45f2cf84eeda313dee6e571a1ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:40.655215 27892 consensus_queue.cc:1048] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:40.668578 27607 raft_consensus.cc:1275] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Refusing update from remote peer 1bc2e45f2cf84eeda313dee6e571a1ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:40.670300 27892 consensus_queue.cc:1048] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:40.810411 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:40.824270 27758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0d3499349e2425798b193e641999c0e"
dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
mode: GRACEFUL
new_leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:53900
I20260709 13:10:40.824873 27758 raft_consensus.cc:606] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 LEADER]: Received request to transfer leadership to TS b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:40.833204 27758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388"
dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
mode: GRACEFUL
new_leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:53900
I20260709 13:10:40.833757 27758 raft_consensus.cc:606] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 LEADER]: Received request to transfer leadership to TS b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:40.842247 27683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d"
dest_uuid: "81de5334c2aa49909f114e12900b357c"
mode: GRACEFUL
new_leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:43978
I20260709 13:10:40.842818 27683 raft_consensus.cc:606] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Received request to transfer leadership to TS b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:40.848404 27683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3904a7744db64435a28bb973da2089f5"
dest_uuid: "81de5334c2aa49909f114e12900b357c"
mode: GRACEFUL
new_leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:43978
I20260709 13:10:40.848946 27683 raft_consensus.cc:606] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Received request to transfer leadership to TS b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:40.854055 27683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b"
dest_uuid: "81de5334c2aa49909f114e12900b357c"
mode: GRACEFUL
new_leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:43978
I20260709 13:10:40.854848 27683 raft_consensus.cc:606] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Received request to transfer leadership to TS b489ccf32e7542de8d18ba7e1730caf7
I20260709 13:10:40.884851 27903 raft_consensus.cc:993] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c: : Instructing follower b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:40.885443 27911 raft_consensus.cc:1081] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Signalling peer b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:40.887044 27608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3904a7744db64435a28bb973da2089f5"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:40192
I20260709 13:10:40.887714 27608 raft_consensus.cc:493] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.887992 27608 raft_consensus.cc:3060] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:40.892588 27608 raft_consensus.cc:515] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:40.894138 27608 leader_election.cc:290] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 3 election: Requested vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:40.895301 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3904a7744db64435a28bb973da2089f5" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:40.895551 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3904a7744db64435a28bb973da2089f5" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:40.895855 27683 raft_consensus.cc:3055] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:40.896075 27758 raft_consensus.cc:3060] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:40.896232 27683 raft_consensus.cc:740] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:40.897087 27683 consensus_queue.cc:260] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:40.897993 27683 raft_consensus.cc:3060] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:40.901264 27758 raft_consensus.cc:2468] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 3.
I20260709 13:10:40.902518 27573 leader_election.cc:304] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:40.903334 27845 raft_consensus.cc:2804] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:40.903339 27683 raft_consensus.cc:2468] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 3.
I20260709 13:10:40.905020 27845 raft_consensus.cc:697] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:40.905893 27845 consensus_queue.cc:237] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:40.913378 27504 catalog_manager.cc:5697] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 2 to 3, leader changed from 81de5334c2aa49909f114e12900b357c (127.26.83.194) to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 3 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:40.993638 27897 raft_consensus.cc:993] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac: : Instructing follower b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:40.994177 27897 raft_consensus.cc:1081] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 LEADER]: Signalling peer b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:40.995819 27608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:40182
I20260709 13:10:40.996276 27608 raft_consensus.cc:493] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:40.996609 27608 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:41.000813 27608 raft_consensus.cc:515] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.002345 27608 leader_election.cc:290] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 3 election: Requested vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:41.003338 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:41.003824 27683 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:41.003607 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:41.004418 27758 raft_consensus.cc:3055] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:41.004711 27758 raft_consensus.cc:740] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:41.005503 27758 consensus_queue.cc:260] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.006922 27758 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:41.009203 27683 raft_consensus.cc:2468] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 3.
I20260709 13:10:41.010375 27573 leader_election.cc:304] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:41.011204 27845 raft_consensus.cc:2804] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:41.011698 27845 raft_consensus.cc:697] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:41.012521 27845 consensus_queue.cc:237] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.013059 27758 raft_consensus.cc:2468] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 3.
I20260709 13:10:41.019584 27504 catalog_manager.cc:5697] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 2 to 3, leader changed from 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195) to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 3 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:41.074492 27913 raft_consensus.cc:993] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac: : Instructing follower b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:41.075052 27897 raft_consensus.cc:1081] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 LEADER]: Signalling peer b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:41.076908 27608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0d3499349e2425798b193e641999c0e"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:40182
I20260709 13:10:41.077536 27608 raft_consensus.cc:493] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:41.077915 27608 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:41.082286 27608 raft_consensus.cc:515] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.083671 27608 leader_election.cc:290] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 4 election: Requested vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:41.084653 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:41.085207 27683 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:41.085320 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0d3499349e2425798b193e641999c0e" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:41.085873 27758 raft_consensus.cc:3055] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:41.086174 27758 raft_consensus.cc:740] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:41.086978 27758 consensus_queue.cc:260] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.088148 27758 raft_consensus.cc:3060] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:41.091624 27683 raft_consensus.cc:2468] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 4.
I20260709 13:10:41.094177 27573 leader_election.cc:304] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81de5334c2aa49909f114e12900b357c, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:41.095000 27758 raft_consensus.cc:2468] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 4.
I20260709 13:10:41.095140 27845 raft_consensus.cc:2804] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:41.096043 27845 raft_consensus.cc:697] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 4 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:41.097048 27845 consensus_queue.cc:237] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.103792 27504 catalog_manager.cc:5697] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 3 to 4, leader changed from 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195) to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 4 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:41.197832 27903 raft_consensus.cc:993] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c: : Instructing follower b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:41.198323 27911 raft_consensus.cc:1081] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Signalling peer b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:41.200392 27608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:40192
I20260709 13:10:41.200979 27608 raft_consensus.cc:493] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:41.201347 27608 raft_consensus.cc:3060] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:41.206173 27608 raft_consensus.cc:515] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.207644 27608 leader_election.cc:290] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 2 election: Requested vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:41.208572 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:41.209327 27683 raft_consensus.cc:3055] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:41.209224 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:41.209748 27683 raft_consensus.cc:740] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:41.210000 27758 raft_consensus.cc:3060] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:41.211261 27683 consensus_queue.cc:260] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.212558 27683 raft_consensus.cc:3060] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:41.217185 27758 raft_consensus.cc:2468] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 2.
I20260709 13:10:41.218411 27573 leader_election.cc:304] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:41.219368 27845 raft_consensus.cc:2804] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:41.219435 27683 raft_consensus.cc:2468] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 2.
I20260709 13:10:41.219832 27845 raft_consensus.cc:697] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:41.220706 27845 consensus_queue.cc:237] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.227162 27504 catalog_manager.cc:5697] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 1 to 2, leader changed from 81de5334c2aa49909f114e12900b357c (127.26.83.194) to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 2 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:41.289804 27903 raft_consensus.cc:993] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c: : Instructing follower b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:41.290241 27911 raft_consensus.cc:1081] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Signalling peer b489ccf32e7542de8d18ba7e1730caf7 to start an election
I20260709 13:10:41.291982 27608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
 from {username='slave'} at 127.0.0.1:40192
I20260709 13:10:41.292503 27608 raft_consensus.cc:493] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:41.292943 27608 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:41.297818 27608 raft_consensus.cc:515] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.299656 27608 leader_election.cc:290] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 3 election: Requested vote from peers 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:41.300832 27683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:41.300920 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "b489ccf32e7542de8d18ba7e1730caf7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:41.301437 27683 raft_consensus.cc:3055] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:41.301610 27758 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:41.301759 27683 raft_consensus.cc:740] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:41.302536 27683 consensus_queue.cc:260] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.303447 27683 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:41.306315 27758 raft_consensus.cc:2468] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 3.
I20260709 13:10:41.307435 27573 leader_election.cc:304] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, b489ccf32e7542de8d18ba7e1730caf7; no voters: 
I20260709 13:10:41.308328 27683 raft_consensus.cc:2468] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b489ccf32e7542de8d18ba7e1730caf7 in term 3.
I20260709 13:10:41.308378 27845 raft_consensus.cc:2804] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:41.308966 27845 raft_consensus.cc:697] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Becoming Leader. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:41.309828 27845 consensus_queue.cc:237] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:41.316634 27504 catalog_manager.cc:5697] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 reported cstate change: term changed from 2 to 3, leader changed from 81de5334c2aa49909f114e12900b357c (127.26.83.194) to b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193). New cstate: current_term: 3 leader_uuid: "b489ccf32e7542de8d18ba7e1730caf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:41.365083 27684 raft_consensus.cc:1275] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:41.367460 27915 consensus_queue.cc:1048] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:41.384259 27758 raft_consensus.cc:1275] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:41.386391 27845 consensus_queue.cc:1048] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:41.437134 27684 raft_consensus.cc:1275] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:41.438802 27845 consensus_queue.cc:1048] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:41.448774 27758 raft_consensus.cc:1275] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:41.450764 27928 consensus_queue.cc:1048] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:41.578722 27684 raft_consensus.cc:1275] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:41.580083 27928 consensus_queue.cc:1048] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:41.589659 27758 raft_consensus.cc:1275] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:41.592248 27915 consensus_queue.cc:1048] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:41.695917 27684 raft_consensus.cc:1275] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:41.697329 27915 consensus_queue.cc:1048] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:41.708860 27757 raft_consensus.cc:1275] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:41.710297 27928 consensus_queue.cc:1048] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:41.768164 27684 raft_consensus.cc:1275] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:41.770068 27845 consensus_queue.cc:1048] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:41.770841 27757 raft_consensus.cc:1275] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Refusing update from remote peer b489ccf32e7542de8d18ba7e1730caf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:41.772248 27845 consensus_queue.cc:1048] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:43.863397 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:43.871217 27608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39b5a991814a494c8e3ac259a6d66915"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
mode: GRACEFUL
new_leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
 from {username='slave'} at 127.0.0.1:40208
I20260709 13:10:43.871927 27608 raft_consensus.cc:606] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Received request to transfer leadership to TS 1bc2e45f2cf84eeda313dee6e571a1ac
I20260709 13:10:43.875072 27608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
mode: GRACEFUL
new_leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
 from {username='slave'} at 127.0.0.1:40208
I20260709 13:10:43.875818 27608 raft_consensus.cc:606] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Received request to transfer leadership to TS 1bc2e45f2cf84eeda313dee6e571a1ac
I20260709 13:10:43.878867 27608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
mode: GRACEFUL
new_leader_uuid: "81de5334c2aa49909f114e12900b357c"
 from {username='slave'} at 127.0.0.1:40208
I20260709 13:10:43.879603 27608 raft_consensus.cc:606] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Received request to transfer leadership to TS 81de5334c2aa49909f114e12900b357c
I20260709 13:10:43.882231 27608 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b"
dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
mode: GRACEFUL
new_leader_uuid: "81de5334c2aa49909f114e12900b357c"
 from {username='slave'} at 127.0.0.1:40208
I20260709 13:10:43.882753 27608 raft_consensus.cc:606] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Received request to transfer leadership to TS 81de5334c2aa49909f114e12900b357c
I20260709 13:10:43.884037 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:10:43.884653 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:43.973273 27915 raft_consensus.cc:993] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7: : Instructing follower 81de5334c2aa49909f114e12900b357c to start an election
I20260709 13:10:43.973932 27928 raft_consensus.cc:1081] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Signalling peer 81de5334c2aa49909f114e12900b357c to start an election
I20260709 13:10:43.976003 27684 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b"
dest_uuid: "81de5334c2aa49909f114e12900b357c"
 from {username='slave'} at 127.0.0.1:43972
I20260709 13:10:43.976523 27684 raft_consensus.cc:493] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:43.976933 27684 raft_consensus.cc:3060] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:43.983328 27684 raft_consensus.cc:515] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:43.985323 27684 leader_election.cc:290] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 3 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:43.986315 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:43.986617 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91da424edc5d4e5fb4ed0a6c8549339b" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:43.987049 27608 raft_consensus.cc:3055] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:43.987386 27758 raft_consensus.cc:3060] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:43.987398 27608 raft_consensus.cc:740] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:43.988387 27608 consensus_queue.cc:260] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:43.989352 27608 raft_consensus.cc:3060] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:43.992580 27758 raft_consensus.cc:2468] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 3.
I20260709 13:10:43.994158 27608 raft_consensus.cc:2468] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 3.
I20260709 13:10:43.994030 27648 leader_election.cc:304] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: 
I20260709 13:10:43.995373 27911 raft_consensus.cc:2804] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:43.996373 27911 raft_consensus.cc:697] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 3 LEADER]: Becoming Leader. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:43.997319 27911 consensus_queue.cc:237] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:44.004961 27505 catalog_manager.cc:5697] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c reported cstate change: term changed from 2 to 3, leader changed from b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193) to 81de5334c2aa49909f114e12900b357c (127.26.83.194). New cstate: current_term: 3 leader_uuid: "81de5334c2aa49909f114e12900b357c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:44.123980 27915 raft_consensus.cc:993] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7: : Instructing follower 1bc2e45f2cf84eeda313dee6e571a1ac to start an election
I20260709 13:10:44.124446 27928 raft_consensus.cc:1081] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Signalling peer 1bc2e45f2cf84eeda313dee6e571a1ac to start an election
I20260709 13:10:44.126024 27758 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39b5a991814a494c8e3ac259a6d66915"
dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
 from {username='slave'} at 127.0.0.1:53882
I20260709 13:10:44.126447 27758 raft_consensus.cc:493] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:44.126925 27758 raft_consensus.cc:3060] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.131202 27758 raft_consensus.cc:515] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:44.132856 27758 leader_election.cc:290] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 2 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:44.133723 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b5a991814a494c8e3ac259a6d66915" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:44.134075 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b5a991814a494c8e3ac259a6d66915" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:44.134346 27608 raft_consensus.cc:3055] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:44.134590 27684 raft_consensus.cc:3060] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.134804 27608 raft_consensus.cc:740] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:44.135644 27608 consensus_queue.cc:260] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:44.136965 27608 raft_consensus.cc:3060] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:44.141933 27684 raft_consensus.cc:2468] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 2.
I20260709 13:10:44.142246 27608 raft_consensus.cc:2468] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 2.
I20260709 13:10:44.143281 27722 leader_election.cc:304] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: 
I20260709 13:10:44.144259 27897 raft_consensus.cc:2804] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:44.145805 27897 raft_consensus.cc:697] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 LEADER]: Becoming Leader. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:44.146708 27897 consensus_queue.cc:237] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } }
I20260709 13:10:44.154462 27505 catalog_manager.cc:5697] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac reported cstate change: term changed from 1 to 2, leader changed from b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193) to 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195). New cstate: current_term: 2 leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:44.188154 27915 raft_consensus.cc:993] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7: : Instructing follower 81de5334c2aa49909f114e12900b357c to start an election
I20260709 13:10:44.188656 27928 raft_consensus.cc:1081] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Signalling peer 81de5334c2aa49909f114e12900b357c to start an election
I20260709 13:10:44.190227 27684 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388"
dest_uuid: "81de5334c2aa49909f114e12900b357c"
 from {username='slave'} at 127.0.0.1:43972
I20260709 13:10:44.190712 27684 raft_consensus.cc:493] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:44.190991 27684 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:44.195945 27684 raft_consensus.cc:515] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:44.198501 27684 leader_election.cc:290] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 4 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195:40721)
I20260709 13:10:44.199012 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:44.199568 27608 raft_consensus.cc:3055] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:44.199594 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "814ca7b7dd7849658d9093302fc97388" candidate_uuid: "81de5334c2aa49909f114e12900b357c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
I20260709 13:10:44.199949 27608 raft_consensus.cc:740] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:44.200124 27758 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:44.200698 27608 consensus_queue.cc:260] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:44.201668 27608 raft_consensus.cc:3060] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:44.205955 27758 raft_consensus.cc:2468] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 4.
I20260709 13:10:44.207021 27648 leader_election.cc:304] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: 
I20260709 13:10:44.207774 27608 raft_consensus.cc:2468] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81de5334c2aa49909f114e12900b357c in term 4.
I20260709 13:10:44.207857 27911 raft_consensus.cc:2804] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:44.208276 27911 raft_consensus.cc:697] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 4 LEADER]: Becoming Leader. State: Replica: 81de5334c2aa49909f114e12900b357c, State: Running, Role: LEADER
I20260709 13:10:44.209235 27911 consensus_queue.cc:237] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:44.216454 27505 catalog_manager.cc:5697] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c reported cstate change: term changed from 3 to 4, leader changed from b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193) to 81de5334c2aa49909f114e12900b357c (127.26.83.194). New cstate: current_term: 4 leader_uuid: "81de5334c2aa49909f114e12900b357c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:44.385437 27915 raft_consensus.cc:993] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7: : Instructing follower 1bc2e45f2cf84eeda313dee6e571a1ac to start an election
I20260709 13:10:44.385951 27928 raft_consensus.cc:1081] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Signalling peer 1bc2e45f2cf84eeda313dee6e571a1ac to start an election
I20260709 13:10:44.387473 27758 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d"
dest_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac"
 from {username='slave'} at 127.0.0.1:53882
I20260709 13:10:44.387907 27758 raft_consensus.cc:493] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:44.388170 27758 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:44.392657 27758 raft_consensus.cc:515] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:44.394308 27758 leader_election.cc:290] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 4 election: Requested vote from peers b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085), 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029)
I20260709 13:10:44.395385 27608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b489ccf32e7542de8d18ba7e1730caf7"
I20260709 13:10:44.395385 27684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "404737675c1a4dacb0283ccccea7a99d" candidate_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "81de5334c2aa49909f114e12900b357c"
I20260709 13:10:44.396209 27608 raft_consensus.cc:3055] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:44.396322 27684 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:44.396554 27608 raft_consensus.cc:740] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b489ccf32e7542de8d18ba7e1730caf7, State: Running, Role: LEADER
I20260709 13:10:44.397420 27608 consensus_queue.cc:260] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:44.398638 27608 raft_consensus.cc:3060] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:44.403034 27758 raft_consensus.cc:1275] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Refusing update from remote peer 81de5334c2aa49909f114e12900b357c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:44.403081 27684 raft_consensus.cc:2468] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 4.
I20260709 13:10:44.404618 27722 leader_election.cc:304] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bc2e45f2cf84eeda313dee6e571a1ac, 81de5334c2aa49909f114e12900b357c; no voters: 
I20260709 13:10:44.404717 27911 consensus_queue.cc:1048] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:44.405571 27897 raft_consensus.cc:2804] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:44.404685 27608 raft_consensus.cc:2468] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bc2e45f2cf84eeda313dee6e571a1ac in term 4.
I20260709 13:10:44.406113 27897 raft_consensus.cc:697] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 LEADER]: Becoming Leader. State: Replica: 1bc2e45f2cf84eeda313dee6e571a1ac, State: Running, Role: LEADER
I20260709 13:10:44.407189 27897 consensus_queue.cc:237] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } }
I20260709 13:10:44.418208 27505 catalog_manager.cc:5697] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac reported cstate change: term changed from 3 to 4, leader changed from b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193) to 1bc2e45f2cf84eeda313dee6e571a1ac (127.26.83.195). New cstate: current_term: 4 leader_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:44.421828 27608 raft_consensus.cc:1275] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Refusing update from remote peer 81de5334c2aa49909f114e12900b357c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:44.424327 27946 consensus_queue.cc:1048] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:44.555392 27608 raft_consensus.cc:1275] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Refusing update from remote peer 1bc2e45f2cf84eeda313dee6e571a1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:44.556567 27897 consensus_queue.cc:1048] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:44.566105 27684 raft_consensus.cc:1275] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Refusing update from remote peer 1bc2e45f2cf84eeda313dee6e571a1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:44.567909 27950 consensus_queue.cc:1048] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:44.717088 27758 raft_consensus.cc:1275] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Refusing update from remote peer 81de5334c2aa49909f114e12900b357c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:44.718405 27933 consensus_queue.cc:1048] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bc2e45f2cf84eeda313dee6e571a1ac" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:44.728057 27608 raft_consensus.cc:1275] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 4 FOLLOWER]: Refusing update from remote peer 81de5334c2aa49909f114e12900b357c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:44.730392 27946 consensus_queue.cc:1048] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b489ccf32e7542de8d18ba7e1730caf7" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:44.885663 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:44.891822 26959 tablet_server.cc:179] TabletServer@127.26.83.193:0 shutting down...
I20260709 13:10:44.921737 26959 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:44.923631 26959 tablet_replica.cc:333] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7: stopping tablet replica
W20260709 13:10:44.924083 27647 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
I20260709 13:10:44.924794 26959 raft_consensus.cc:2243] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.925454 26959 raft_consensus.cc:2272] T 404737675c1a4dacb0283ccccea7a99d P b489ccf32e7542de8d18ba7e1730caf7 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:44.928280 27647 consensus_peers.cc:597] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c -> Peer b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085): Couldn't send request to peer b489ccf32e7542de8d18ba7e1730caf7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:44.929172 26959 tablet_replica.cc:333] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7: stopping tablet replica
I20260709 13:10:44.930045 26959 raft_consensus.cc:2243] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.930990 26959 raft_consensus.cc:2272] T d0d3499349e2425798b193e641999c0e P b489ccf32e7542de8d18ba7e1730caf7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.933353 26959 tablet_replica.cc:333] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7: stopping tablet replica
I20260709 13:10:44.933959 26959 raft_consensus.cc:2243] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.934449 26959 raft_consensus.cc:2272] T 681a55081a294370909a44c09d683d34 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.936302 26959 tablet_replica.cc:333] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7: stopping tablet replica
I20260709 13:10:44.937062 26959 raft_consensus.cc:2243] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.938169 26959 raft_consensus.cc:2272] T a4163203b22b452985b9774ad4d7a2c9 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.940331 26959 tablet_replica.cc:333] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7: stopping tablet replica
I20260709 13:10:44.941089 26959 raft_consensus.cc:2243] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.941543 26959 raft_consensus.cc:2272] T 91da424edc5d4e5fb4ed0a6c8549339b P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:10:44.943209 27721 consensus_peers.cc:597] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac -> Peer b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085): Couldn't send request to peer b489ccf32e7542de8d18ba7e1730caf7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:44.944008 26959 tablet_replica.cc:333] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7: stopping tablet replica
I20260709 13:10:44.944617 26959 raft_consensus.cc:2243] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.945175 26959 raft_consensus.cc:2272] T 15983c9724104e7c86e9ea2e6ca0e7ff P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.946987 26959 tablet_replica.cc:333] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7: stopping tablet replica
I20260709 13:10:44.947546 26959 raft_consensus.cc:2243] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.948350 26959 raft_consensus.cc:2272] T c49c4ac4d6fa4ce18035f3cafb56340a P b489ccf32e7542de8d18ba7e1730caf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.950269 26959 tablet_replica.cc:333] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7: stopping tablet replica
I20260709 13:10:44.950821 26959 raft_consensus.cc:2243] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.951263 26959 raft_consensus.cc:2272] T 814ca7b7dd7849658d9093302fc97388 P b489ccf32e7542de8d18ba7e1730caf7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.953467 26959 tablet_replica.cc:333] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7: stopping tablet replica
I20260709 13:10:44.954005 26959 raft_consensus.cc:2243] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:44.954763 26959 raft_consensus.cc:2272] T 3904a7744db64435a28bb973da2089f5 P b489ccf32e7542de8d18ba7e1730caf7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.956552 26959 tablet_replica.cc:333] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7: stopping tablet replica
I20260709 13:10:44.957063 26959 raft_consensus.cc:2243] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:44.957497 26959 raft_consensus.cc:2272] T 39b5a991814a494c8e3ac259a6d66915 P b489ccf32e7542de8d18ba7e1730caf7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:44.979727 27684 raft_consensus.cc:1275] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Refusing update from remote peer 1bc2e45f2cf84eeda313dee6e571a1ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:44.981093 27950 consensus_queue.cc:1048] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "81de5334c2aa49909f114e12900b357c" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:44.995397 26959 tablet_server.cc:196] TabletServer@127.26.83.193:0 shutdown complete.
I20260709 13:10:45.018356 26959 tablet_server.cc:179] TabletServer@127.26.83.194:0 shutting down...
W20260709 13:10:45.025954 27722 consensus_peers.cc:597] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac -> Peer 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029): Couldn't send request to peer 81de5334c2aa49909f114e12900b357c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:45.043265 27722 consensus_peers.cc:597] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac -> Peer 81de5334c2aa49909f114e12900b357c (127.26.83.194:46029): Couldn't send request to peer 81de5334c2aa49909f114e12900b357c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:45.050310 26959 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:45.051175 26959 tablet_replica.cc:333] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c: stopping tablet replica
I20260709 13:10:45.051846 26959 raft_consensus.cc:2243] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:45.052863 26959 raft_consensus.cc:2272] T 681a55081a294370909a44c09d683d34 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.054925 26959 tablet_replica.cc:333] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c: stopping tablet replica
I20260709 13:10:45.055532 26959 raft_consensus.cc:2243] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.056039 26959 raft_consensus.cc:2272] T d0d3499349e2425798b193e641999c0e P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.057853 26959 tablet_replica.cc:333] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c: stopping tablet replica
I20260709 13:10:45.058414 26959 raft_consensus.cc:2243] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.058887 26959 raft_consensus.cc:2272] T c49c4ac4d6fa4ce18035f3cafb56340a P 81de5334c2aa49909f114e12900b357c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.060779 26959 tablet_replica.cc:333] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c: stopping tablet replica
I20260709 13:10:45.061329 26959 raft_consensus.cc:2243] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.061800 26959 raft_consensus.cc:2272] T 15983c9724104e7c86e9ea2e6ca0e7ff P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.063546 26959 tablet_replica.cc:333] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c: stopping tablet replica
I20260709 13:10:45.064081 26959 raft_consensus.cc:2243] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.064522 26959 raft_consensus.cc:2272] T 39b5a991814a494c8e3ac259a6d66915 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.066776 26959 tablet_replica.cc:333] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c: stopping tablet replica
I20260709 13:10:45.067443 26959 raft_consensus.cc:2243] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.067888 26959 raft_consensus.cc:2272] T a4163203b22b452985b9774ad4d7a2c9 P 81de5334c2aa49909f114e12900b357c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.069743 26959 tablet_replica.cc:333] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c: stopping tablet replica
I20260709 13:10:45.070226 26959 raft_consensus.cc:2243] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:10:45.070951 26959 raft_consensus.cc:2272] T 91da424edc5d4e5fb4ed0a6c8549339b P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.073127 26959 tablet_replica.cc:333] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c: stopping tablet replica
I20260709 13:10:45.073689 26959 raft_consensus.cc:2243] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:45.074580 26959 raft_consensus.cc:2272] T 814ca7b7dd7849658d9093302fc97388 P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.076990 26959 tablet_replica.cc:333] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c: stopping tablet replica
I20260709 13:10:45.077518 26959 raft_consensus.cc:2243] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.077958 26959 raft_consensus.cc:2272] T 3904a7744db64435a28bb973da2089f5 P 81de5334c2aa49909f114e12900b357c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.079614 26959 tablet_replica.cc:333] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c: stopping tablet replica
I20260709 13:10:45.080060 26959 raft_consensus.cc:2243] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.080485 26959 raft_consensus.cc:2272] T 404737675c1a4dacb0283ccccea7a99d P 81de5334c2aa49909f114e12900b357c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.110453 26959 tablet_server.cc:196] TabletServer@127.26.83.194:0 shutdown complete.
I20260709 13:10:45.129792 26959 tablet_server.cc:179] TabletServer@127.26.83.195:0 shutting down...
W20260709 13:10:45.160869 27721 consensus_peers.cc:597] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac -> Peer b489ccf32e7542de8d18ba7e1730caf7 (127.26.83.193:36085): Couldn't send request to peer b489ccf32e7542de8d18ba7e1730caf7. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:36085: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:45.161360 26959 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:10:45.162171 26959 tablet_replica.cc:333] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac: stopping tablet replica
I20260709 13:10:45.162819 26959 raft_consensus.cc:2243] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.163312 26959 raft_consensus.cc:2272] T 3904a7744db64435a28bb973da2089f5 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.165136 26959 tablet_replica.cc:333] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac: stopping tablet replica
I20260709 13:10:45.165673 26959 raft_consensus.cc:2243] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.166155 26959 raft_consensus.cc:2272] T 681a55081a294370909a44c09d683d34 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.167903 26959 tablet_replica.cc:333] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac: stopping tablet replica
I20260709 13:10:45.168419 26959 raft_consensus.cc:2243] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.168903 26959 raft_consensus.cc:2272] T a4163203b22b452985b9774ad4d7a2c9 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.170478 26959 tablet_replica.cc:333] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac: stopping tablet replica
I20260709 13:10:45.170964 26959 raft_consensus.cc:2243] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.171425 26959 raft_consensus.cc:2272] T 91da424edc5d4e5fb4ed0a6c8549339b P 1bc2e45f2cf84eeda313dee6e571a1ac [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.173619 26959 tablet_replica.cc:333] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac: stopping tablet replica
I20260709 13:10:45.174149 26959 raft_consensus.cc:2243] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:10:45.174965 26959 raft_consensus.cc:2272] T 404737675c1a4dacb0283ccccea7a99d P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.177192 26959 tablet_replica.cc:333] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac: stopping tablet replica
I20260709 13:10:45.177748 26959 raft_consensus.cc:2243] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.178202 26959 raft_consensus.cc:2272] T d0d3499349e2425798b193e641999c0e P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.179793 26959 tablet_replica.cc:333] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac: stopping tablet replica
I20260709 13:10:45.180281 26959 raft_consensus.cc:2243] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.180707 26959 raft_consensus.cc:2272] T 814ca7b7dd7849658d9093302fc97388 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.182950 26959 tablet_replica.cc:333] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac: stopping tablet replica
I20260709 13:10:45.183470 26959 raft_consensus.cc:2243] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:45.184204 26959 raft_consensus.cc:2272] T 15983c9724104e7c86e9ea2e6ca0e7ff P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.186035 26959 tablet_replica.cc:333] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac: stopping tablet replica
I20260709 13:10:45.186527 26959 raft_consensus.cc:2243] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:45.186911 26959 raft_consensus.cc:2272] T c49c4ac4d6fa4ce18035f3cafb56340a P 1bc2e45f2cf84eeda313dee6e571a1ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.188700 26959 tablet_replica.cc:333] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac: stopping tablet replica
I20260709 13:10:45.189248 26959 raft_consensus.cc:2243] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:10:45.189988 26959 raft_consensus.cc:2272] T 39b5a991814a494c8e3ac259a6d66915 P 1bc2e45f2cf84eeda313dee6e571a1ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.219146 26959 tablet_server.cc:196] TabletServer@127.26.83.195:0 shutdown complete.
I20260709 13:10:45.239889 26959 master.cc:562] Master@127.26.83.254:34083 shutting down...
I20260709 13:10:45.262837 26959 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:10:45.263454 26959 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:45.263808 26959 tablet_replica.cc:333] T 00000000000000000000000000000000 P ed97f5f52dd34409b3fde4a5e6b3b5eb: stopping tablet replica
I20260709 13:10:45.285370 26959 master.cc:584] Master@127.26.83.254:34083 shutdown complete.
1bc2e45f2cf84eeda313dee6e571a1ac  1
81de5334c2aa49909f114e12900b357c  4
b489ccf32e7542de8d18ba7e1730caf7  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10478 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:10:45.326347 26959 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.83.254:42913
I20260709 13:10:45.327636 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:45.334177 27959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:45.334205 27960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:45.334488 27962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:45.335418 26959 server_base.cc:1061] running on GCE node
I20260709 13:10:45.336320 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:45.336510 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:45.336714 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602645336678 us; error 0 us; skew 500 ppm
I20260709 13:10:45.337265 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:45.339794 26959 webserver.cc:533] Webserver started at http://127.26.83.254:35233/ using document root <none> and password file <none>
I20260709 13:10:45.340281 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:45.340471 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:45.340884 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:45.342061 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/master-0-root/instance:
uuid: "fea5cdb8fcad442a901284de409f4a11"
format_stamp: "Formatted at 2026-07-09 13:10:45 on dist-test-slave-329l"
I20260709 13:10:45.346767 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:10:45.350567 27967 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.351600 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:10:45.351967 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/master-0-root
uuid: "fea5cdb8fcad442a901284de409f4a11"
format_stamp: "Formatted at 2026-07-09 13:10:45 on dist-test-slave-329l"
I20260709 13:10:45.352257 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:45.370151 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:45.371430 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:45.414623 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.254:42913
I20260709 13:10:45.414759 28018 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.254:42913 every 8 connection(s)
I20260709 13:10:45.419186 28019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:45.430241 28019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Bootstrap starting.
I20260709 13:10:45.435322 28019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:45.440294 28019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: No bootstrap required, opened a new log
I20260709 13:10:45.442521 28019 raft_consensus.cc:359] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea5cdb8fcad442a901284de409f4a11" member_type: VOTER }
I20260709 13:10:45.442919 28019 raft_consensus.cc:385] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:45.443094 28019 raft_consensus.cc:740] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea5cdb8fcad442a901284de409f4a11, State: Initialized, Role: FOLLOWER
I20260709 13:10:45.443699 28019 consensus_queue.cc:260] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea5cdb8fcad442a901284de409f4a11" member_type: VOTER }
I20260709 13:10:45.444176 28019 raft_consensus.cc:399] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:10:45.444372 28019 raft_consensus.cc:493] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:10:45.444585 28019 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:45.448848 28019 raft_consensus.cc:515] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea5cdb8fcad442a901284de409f4a11" member_type: VOTER }
I20260709 13:10:45.449358 28019 leader_election.cc:304] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fea5cdb8fcad442a901284de409f4a11; no voters: 
I20260709 13:10:45.450495 28019 leader_election.cc:290] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:10:45.450861 28022 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:45.452310 28022 raft_consensus.cc:697] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 1 LEADER]: Becoming Leader. State: Replica: fea5cdb8fcad442a901284de409f4a11, State: Running, Role: LEADER
I20260709 13:10:45.453043 28022 consensus_queue.cc:237] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea5cdb8fcad442a901284de409f4a11" member_type: VOTER }
I20260709 13:10:45.453622 28019 sys_catalog.cc:565] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:10:45.458034 28023 sys_catalog.cc:455] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fea5cdb8fcad442a901284de409f4a11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea5cdb8fcad442a901284de409f4a11" member_type: VOTER } }
I20260709 13:10:45.458606 28024 sys_catalog.cc:455] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fea5cdb8fcad442a901284de409f4a11. Latest consensus state: current_term: 1 leader_uuid: "fea5cdb8fcad442a901284de409f4a11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea5cdb8fcad442a901284de409f4a11" member_type: VOTER } }
I20260709 13:10:45.458912 28023 sys_catalog.cc:458] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:45.459407 28024 sys_catalog.cc:458] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [sys.catalog]: This master's current role is: LEADER
I20260709 13:10:45.464478 28029 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:10:45.469760 28029 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:10:45.474923 26959 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:10:45.479478 28029 catalog_manager.cc:1383] Generated new cluster ID: 16fcbebb21ce4813a3c84aaf36212498
I20260709 13:10:45.479794 28029 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:10:45.495447 28029 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:10:45.496711 28029 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:10:45.511425 28029 catalog_manager.cc:6070] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Generated new TSK 0
I20260709 13:10:45.512048 28029 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:10:45.542973 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:45.549424 28040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:45.550256 28041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:45.552166 28043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:45.553128 26959 server_base.cc:1061] running on GCE node
I20260709 13:10:45.554225 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:45.554417 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:45.554641 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602645554606 us; error 0 us; skew 500 ppm
I20260709 13:10:45.555299 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:45.558148 26959 webserver.cc:533] Webserver started at http://127.26.83.193:46237/ using document root <none> and password file <none>
I20260709 13:10:45.558655 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:45.558854 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:45.559170 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:45.560333 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/instance:
uuid: "0faaace2b36c4bcebf2fa6835235e92b"
format_stamp: "Formatted at 2026-07-09 13:10:45 on dist-test-slave-329l"
I20260709 13:10:45.565912 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:10:45.569646 28048 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.570552 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 13:10:45.570883 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root
uuid: "0faaace2b36c4bcebf2fa6835235e92b"
format_stamp: "Formatted at 2026-07-09 13:10:45 on dist-test-slave-329l"
I20260709 13:10:45.571153 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:45.587599 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:45.588991 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:45.590586 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:45.593209 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:45.593413 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.593729 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:45.593896 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.641232 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.193:45587
I20260709 13:10:45.641350 28110 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.193:45587 every 8 connection(s)
I20260709 13:10:45.646245 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:45.655094 28115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:45.656961 28116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:45.660491 26959 server_base.cc:1061] running on GCE node
W20260709 13:10:45.660612 28118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:45.662390 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:45.662637 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:45.662830 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602645662815 us; error 0 us; skew 500 ppm
I20260709 13:10:45.663507 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:45.665468 28111 heartbeater.cc:344] Connected to a master server at 127.26.83.254:42913
I20260709 13:10:45.665881 28111 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:45.666764 26959 webserver.cc:533] Webserver started at http://127.26.83.194:40193/ using document root <none> and password file <none>
I20260709 13:10:45.666828 28111 heartbeater.cc:507] Master 127.26.83.254:42913 requested a full tablet report, sending...
I20260709 13:10:45.667337 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:45.667563 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:45.667832 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:45.669092 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/instance:
uuid: "611ec64ec58244fe82058d6130e4e6f3"
format_stamp: "Formatted at 2026-07-09 13:10:45 on dist-test-slave-329l"
I20260709 13:10:45.669490 27984 ts_manager.cc:194] Registered new tserver with Master: 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:10:45.671444 27984 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58546
I20260709 13:10:45.674012 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:10:45.677551 28123 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.678535 26959 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:10:45.678804 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root
uuid: "611ec64ec58244fe82058d6130e4e6f3"
format_stamp: "Formatted at 2026-07-09 13:10:45 on dist-test-slave-329l"
I20260709 13:10:45.679100 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:45.691776 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:45.693351 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:45.694854 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:45.697190 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:45.697369 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.697674 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:45.697829 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.743618 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.194:38757
I20260709 13:10:45.743742 28185 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.194:38757 every 8 connection(s)
I20260709 13:10:45.748224 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:45.755352 28189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:45.756299 28190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:45.759739 28192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:45.760469 26959 server_base.cc:1061] running on GCE node
I20260709 13:10:45.761581 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:10:45.761662 28186 heartbeater.cc:344] Connected to a master server at 127.26.83.254:42913
W20260709 13:10:45.761859 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:45.762133 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602645762115 us; error 0 us; skew 500 ppm
I20260709 13:10:45.762189 28186 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:45.762733 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:45.763034 28186 heartbeater.cc:507] Master 127.26.83.254:42913 requested a full tablet report, sending...
I20260709 13:10:45.765170 27984 ts_manager.cc:194] Registered new tserver with Master: 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:45.765699 26959 webserver.cc:533] Webserver started at http://127.26.83.195:45715/ using document root <none> and password file <none>
I20260709 13:10:45.766197 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:45.766397 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:45.766680 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:45.766647 27984 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58554
I20260709 13:10:45.767999 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/instance:
uuid: "11f10c12c03e47dcac88b050d4c1431b"
format_stamp: "Formatted at 2026-07-09 13:10:45 on dist-test-slave-329l"
I20260709 13:10:45.773298 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:10:45.777601 28197 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.778524 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:10:45.778851 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root
uuid: "11f10c12c03e47dcac88b050d4c1431b"
format_stamp: "Formatted at 2026-07-09 13:10:45 on dist-test-slave-329l"
I20260709 13:10:45.779184 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:45.806499 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:45.807694 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:45.809346 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:45.811743 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:45.812022 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.812306 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:45.812490 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:45.855839 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.195:46875
I20260709 13:10:45.855948 28259 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.195:46875 every 8 connection(s)
I20260709 13:10:45.871411 28260 heartbeater.cc:344] Connected to a master server at 127.26.83.254:42913
I20260709 13:10:45.871846 28260 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:45.872830 28260 heartbeater.cc:507] Master 127.26.83.254:42913 requested a full tablet report, sending...
I20260709 13:10:45.874969 27984 ts_manager.cc:194] Registered new tserver with Master: 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:45.875344 26959 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016279785s
I20260709 13:10:45.876822 27984 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58556
I20260709 13:10:45.905404 27984 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58564:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:10:45.911831 27984 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:10:46.148165 28150 tablet_service.cc:1511] Processing CreateTablet for tablet dd7077e3c7d94184913764cd90b92757 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:46.149791 28150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd7077e3c7d94184913764cd90b92757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.149878 28151 tablet_service.cc:1511] Processing CreateTablet for tablet 2facf6a0a91849a5a177c2481e42aa47 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:46.150339 28147 tablet_service.cc:1511] Processing CreateTablet for tablet 4b129d69618f48999f257e73b5b8b1d8 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:46.151330 28151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2facf6a0a91849a5a177c2481e42aa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.152829 28076 tablet_service.cc:1511] Processing CreateTablet for tablet 2facf6a0a91849a5a177c2481e42aa47 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:46.154285 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2facf6a0a91849a5a177c2481e42aa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.154336 28072 tablet_service.cc:1511] Processing CreateTablet for tablet 4b129d69618f48999f257e73b5b8b1d8 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:46.155956 28072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b129d69618f48999f257e73b5b8b1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.155609 28073 tablet_service.cc:1511] Processing CreateTablet for tablet d8d07cbc383b4089b326bc02e8b33622 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:46.155771 28224 tablet_service.cc:1511] Processing CreateTablet for tablet dd7077e3c7d94184913764cd90b92757 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:46.157184 28224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd7077e3c7d94184913764cd90b92757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.157855 28074 tablet_service.cc:1511] Processing CreateTablet for tablet 923fa739e01e4f05a05099628c53a6c3 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:46.158473 28071 tablet_service.cc:1511] Processing CreateTablet for tablet 9145e2ecf82c4a03b0034029f10bab74 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:46.158946 28148 tablet_service.cc:1511] Processing CreateTablet for tablet d8d07cbc383b4089b326bc02e8b33622 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:46.160349 28148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8d07cbc383b4089b326bc02e8b33622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.160290 28075 tablet_service.cc:1511] Processing CreateTablet for tablet dd7077e3c7d94184913764cd90b92757 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:10:46.152856 28146 tablet_service.cc:1511] Processing CreateTablet for tablet 9145e2ecf82c4a03b0034029f10bab74 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:46.154035 28225 tablet_service.cc:1511] Processing CreateTablet for tablet 2facf6a0a91849a5a177c2481e42aa47 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:10:46.161613 28070 tablet_service.cc:1511] Processing CreateTablet for tablet 5c654c99561449cb823642a1b85879ac (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:46.161873 28223 tablet_service.cc:1511] Processing CreateTablet for tablet 923fa739e01e4f05a05099628c53a6c3 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:46.162848 28225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2facf6a0a91849a5a177c2481e42aa47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.169634 28073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8d07cbc383b4089b326bc02e8b33622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.170310 28220 tablet_service.cc:1511] Processing CreateTablet for tablet 9145e2ecf82c4a03b0034029f10bab74 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:10:46.170679 28144 tablet_service.cc:1511] Processing CreateTablet for tablet d56e9de7a9c54453bc5fc3aac0fa589c (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:46.171959 28070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c654c99561449cb823642a1b85879ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.175515 28075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd7077e3c7d94184913764cd90b92757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.176065 28218 tablet_service.cc:1511] Processing CreateTablet for tablet d56e9de7a9c54453bc5fc3aac0fa589c (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:46.173005 28219 tablet_service.cc:1511] Processing CreateTablet for tablet 5c654c99561449cb823642a1b85879ac (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:46.165241 28069 tablet_service.cc:1511] Processing CreateTablet for tablet d56e9de7a9c54453bc5fc3aac0fa589c (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:10:46.157868 28149 tablet_service.cc:1511] Processing CreateTablet for tablet 923fa739e01e4f05a05099628c53a6c3 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:10:46.173728 28143 tablet_service.cc:1511] Processing CreateTablet for tablet 4c9b061a7ca84a61ac13d29946a28169 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:46.164378 28222 tablet_service.cc:1511] Processing CreateTablet for tablet d8d07cbc383b4089b326bc02e8b33622 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:10:46.154140 28145 tablet_service.cc:1511] Processing CreateTablet for tablet 5c654c99561449cb823642a1b85879ac (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:10:46.174451 28223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 923fa739e01e4f05a05099628c53a6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.173724 28221 tablet_service.cc:1511] Processing CreateTablet for tablet 4b129d69618f48999f257e73b5b8b1d8 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:10:46.177656 28142 tablet_service.cc:1511] Processing CreateTablet for tablet 53c6cdf8c6704ee59d0a736273582a48 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:46.180464 28147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b129d69618f48999f257e73b5b8b1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.181200 28071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9145e2ecf82c4a03b0034029f10bab74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.185752 28142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c6cdf8c6704ee59d0a736273582a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.185703 28067 tablet_service.cc:1511] Processing CreateTablet for tablet 53c6cdf8c6704ee59d0a736273582a48 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:46.181530 28068 tablet_service.cc:1511] Processing CreateTablet for tablet 4c9b061a7ca84a61ac13d29946a28169 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:46.188400 28219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c654c99561449cb823642a1b85879ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.189152 28222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8d07cbc383b4089b326bc02e8b33622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.191591 28218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56e9de7a9c54453bc5fc3aac0fa589c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.193142 28221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b129d69618f48999f257e73b5b8b1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.195657 28220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9145e2ecf82c4a03b0034029f10bab74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.196996 28149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 923fa739e01e4f05a05099628c53a6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.197973 28144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56e9de7a9c54453bc5fc3aac0fa589c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.198912 28143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c9b061a7ca84a61ac13d29946a28169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.199656 28217 tablet_service.cc:1511] Processing CreateTablet for tablet 4c9b061a7ca84a61ac13d29946a28169 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:10:46.201712 28069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56e9de7a9c54453bc5fc3aac0fa589c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.201960 28217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c9b061a7ca84a61ac13d29946a28169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.202008 28216 tablet_service.cc:1511] Processing CreateTablet for tablet 53c6cdf8c6704ee59d0a736273582a48 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:10:46.203035 28067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c6cdf8c6704ee59d0a736273582a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.203662 28216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c6cdf8c6704ee59d0a736273582a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.205230 28068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c9b061a7ca84a61ac13d29946a28169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.213856 28074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 923fa739e01e4f05a05099628c53a6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.240309 28150 tablet_service.cc:1511] Processing CreateTablet for tablet 1115d5b02e774b14935e3f48488f82ea (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:46.242591 28150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1115d5b02e774b14935e3f48488f82ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.250958 28146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9145e2ecf82c4a03b0034029f10bab74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.255007 28145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c654c99561449cb823642a1b85879ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.256009 28148 tablet_service.cc:1511] Processing CreateTablet for tablet a8381961442846ab97965dc11b100d75 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:46.259146 28148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8381961442846ab97965dc11b100d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.257711 28219 tablet_service.cc:1511] Processing CreateTablet for tablet 1115d5b02e774b14935e3f48488f82ea (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:46.263075 28219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1115d5b02e774b14935e3f48488f82ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.287295 28150 tablet_service.cc:1511] Processing CreateTablet for tablet d65f83feb5bb4b468e71fc5b9b708de4 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:46.310019 28150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d65f83feb5bb4b468e71fc5b9b708de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.326583 28221 tablet_service.cc:1511] Processing CreateTablet for tablet a8381961442846ab97965dc11b100d75 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:46.327807 28073 tablet_service.cc:1511] Processing CreateTablet for tablet 1115d5b02e774b14935e3f48488f82ea (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:10:46.329254 28073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1115d5b02e774b14935e3f48488f82ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.329533 28221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8381961442846ab97965dc11b100d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.354665 28221 tablet_service.cc:1511] Processing CreateTablet for tablet d65f83feb5bb4b468e71fc5b9b708de4 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:46.356823 28221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d65f83feb5bb4b468e71fc5b9b708de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.378942 28276 tablet_bootstrap.cc:492] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:46.382473 28072 tablet_service.cc:1511] Processing CreateTablet for tablet a8381961442846ab97965dc11b100d75 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:10:46.384140 28072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8381961442846ab97965dc11b100d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.386543 28220 tablet_service.cc:1511] Processing CreateTablet for tablet 8ada0581aa09483e91a300220a129859 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:46.370157 28224 tablet_service.cc:1511] Processing CreateTablet for tablet 7c0743024f324c14b4c6fc08f70600d2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:46.401015 28224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0743024f324c14b4c6fc08f70600d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.360425 28216 tablet_service.cc:1511] Processing CreateTablet for tablet b99bb21dbacb4d63839f1403ef9b08b9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:46.409324 28276 tablet_bootstrap.cc:654] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.409679 28275 tablet_bootstrap.cc:492] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:46.410343 28223 tablet_service.cc:1511] Processing CreateTablet for tablet 8addb79d6158437f8d7a4822a75661ab (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:46.411959 28075 tablet_service.cc:1511] Processing CreateTablet for tablet d65f83feb5bb4b468e71fc5b9b708de4 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:10:46.417915 28067 tablet_service.cc:1511] Processing CreateTablet for tablet b99bb21dbacb4d63839f1403ef9b08b9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:46.419514 28067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99bb21dbacb4d63839f1403ef9b08b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.426489 28071 tablet_service.cc:1511] Processing CreateTablet for tablet 8addb79d6158437f8d7a4822a75661ab (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:46.428407 28071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8addb79d6158437f8d7a4822a75661ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.434497 28275 tablet_bootstrap.cc:654] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.420012 28076 tablet_service.cc:1511] Processing CreateTablet for tablet 7c0743024f324c14b4c6fc08f70600d2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:46.445729 28145 tablet_service.cc:1511] Processing CreateTablet for tablet b99bb21dbacb4d63839f1403ef9b08b9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:10:46.447266 28145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99bb21dbacb4d63839f1403ef9b08b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.447531 28150 tablet_service.cc:1511] Processing CreateTablet for tablet 7c0743024f324c14b4c6fc08f70600d2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:10:46.449038 28150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0743024f324c14b4c6fc08f70600d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.451531 28222 tablet_service.cc:1511] Processing CreateTablet for tablet c71e4401306a4e84a111e5443be7652c (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:46.454450 28217 tablet_service.cc:1511] Processing CreateTablet for tablet 6b9a6ec9a3a1422b8723b26150db560f (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:46.463356 28220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ada0581aa09483e91a300220a129859. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:46.470026 28032 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:46.471376 28216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99bb21dbacb4d63839f1403ef9b08b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.424018 28070 tablet_service.cc:1511] Processing CreateTablet for tablet 8ada0581aa09483e91a300220a129859 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:46.473621 28070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ada0581aa09483e91a300220a129859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.474491 28222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c71e4401306a4e84a111e5443be7652c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.464187 28069 tablet_service.cc:1511] Processing CreateTablet for tablet c71e4401306a4e84a111e5443be7652c (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:46.465631 28225 tablet_service.cc:1511] Processing CreateTablet for tablet 2929b1e07bb74c67863e4c504d8d09a6 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:46.467500 28277 tablet_bootstrap.cc:492] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:46.478029 28075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d65f83feb5bb4b468e71fc5b9b708de4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.481778 28068 tablet_service.cc:1511] Processing CreateTablet for tablet 6b9a6ec9a3a1422b8723b26150db560f (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:46.492848 28223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8addb79d6158437f8d7a4822a75661ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.493754 28225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2929b1e07bb74c67863e4c504d8d09a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.494834 28151 tablet_service.cc:1511] Processing CreateTablet for tablet 8ada0581aa09483e91a300220a129859 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:10:46.496129 28151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ada0581aa09483e91a300220a129859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.497192 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0743024f324c14b4c6fc08f70600d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.497756 28217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b9a6ec9a3a1422b8723b26150db560f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.501477 28068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b9a6ec9a3a1422b8723b26150db560f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.502538 28142 tablet_service.cc:1511] Processing CreateTablet for tablet 2929b1e07bb74c67863e4c504d8d09a6 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:46.483003 28218 tablet_service.cc:1511] Processing CreateTablet for tablet 6a669edb54e3494387ede68c8acd8422 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:46.487571 28148 tablet_service.cc:1511] Processing CreateTablet for tablet 8addb79d6158437f8d7a4822a75661ab (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:10:46.488468 28147 tablet_service.cc:1511] Processing CreateTablet for tablet 6b9a6ec9a3a1422b8723b26150db560f (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:10:46.506804 28069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c71e4401306a4e84a111e5443be7652c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.507838 28218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a669edb54e3494387ede68c8acd8422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.509162 28276 tablet_bootstrap.cc:492] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:46.533128 28276 ts_tablet_manager.cc:1403] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.155s	user 0.009s	sys 0.021s
I20260709 13:10:46.505744 28142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2929b1e07bb74c67863e4c504d8d09a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.510004 28073 tablet_service.cc:1511] Processing CreateTablet for tablet 2929b1e07bb74c67863e4c504d8d09a6 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:10:46.510339 28221 tablet_service.cc:1511] Processing CreateTablet for tablet 798c63c7bb2e4373af20f55bbe872cf2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:46.510984 28149 tablet_service.cc:1511] Processing CreateTablet for tablet 798c63c7bb2e4373af20f55bbe872cf2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:46.511863 28144 tablet_service.cc:1511] Processing CreateTablet for tablet 6a669edb54e3494387ede68c8acd8422 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:46.529433 28146 tablet_service.cc:1511] Processing CreateTablet for tablet 030bc6df8e7e4f2da3b18613d20d9bdc (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:46.541795 28221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 798c63c7bb2e4373af20f55bbe872cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.543190 28219 tablet_service.cc:1511] Processing CreateTablet for tablet 030bc6df8e7e4f2da3b18613d20d9bdc (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:46.550349 28219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 030bc6df8e7e4f2da3b18613d20d9bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.550542 28148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8addb79d6158437f8d7a4822a75661ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.551268 28144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a669edb54e3494387ede68c8acd8422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.553458 28149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 798c63c7bb2e4373af20f55bbe872cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.555420 28146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 030bc6df8e7e4f2da3b18613d20d9bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.536543 28276 raft_consensus.cc:359] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:46.561025 28276 raft_consensus.cc:385] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.561439 28276 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.562039 28071 tablet_service.cc:1511] Processing CreateTablet for tablet 6a669edb54e3494387ede68c8acd8422 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:10:46.562319 28276 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:46.564038 28147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b9a6ec9a3a1422b8723b26150db560f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.545486 28073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2929b1e07bb74c67863e4c504d8d09a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.567416 28277 tablet_bootstrap.cc:654] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.549247 28275 tablet_bootstrap.cc:492] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:46.569222 28275 ts_tablet_manager.cc:1403] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.160s	user 0.008s	sys 0.007s
I20260709 13:10:46.570014 28071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a669edb54e3494387ede68c8acd8422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.545723 28143 tablet_service.cc:1511] Processing CreateTablet for tablet c71e4401306a4e84a111e5443be7652c (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:10:46.574463 28072 tablet_service.cc:1511] Processing CreateTablet for tablet 798c63c7bb2e4373af20f55bbe872cf2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:10:46.575297 28143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c71e4401306a4e84a111e5443be7652c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.575822 28072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 798c63c7bb2e4373af20f55bbe872cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.578635 28074 tablet_service.cc:1511] Processing CreateTablet for tablet 030bc6df8e7e4f2da3b18613d20d9bdc (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:10:46.593990 28074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 030bc6df8e7e4f2da3b18613d20d9bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.572576 28275 raft_consensus.cc:359] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:46.584161 28075 tablet_service.cc:1511] Processing CreateTablet for tablet 34353a7e5a8346119cf9f0836165a1b7 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:46.601934 28275 raft_consensus.cc:385] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.616392 28150 tablet_service.cc:1511] Processing CreateTablet for tablet 34353a7e5a8346119cf9f0836165a1b7 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:46.617810 28150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34353a7e5a8346119cf9f0836165a1b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.619885 28275 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.644976 28075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34353a7e5a8346119cf9f0836165a1b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.655651 28260 heartbeater.cc:499] Master 127.26.83.254:42913 was elected leader, sending a full tablet report...
I20260709 13:10:46.649875 28224 tablet_service.cc:1511] Processing CreateTablet for tablet 34353a7e5a8346119cf9f0836165a1b7 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:10:46.672561 28067 tablet_service.cc:1511] Processing CreateTablet for tablet e53fd092d3dd48c492e04080969cb666 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:46.674448 28111 heartbeater.cc:499] Master 127.26.83.254:42913 was elected leader, sending a full tablet report...
I20260709 13:10:46.674057 28067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e53fd092d3dd48c492e04080969cb666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.674700 28145 tablet_service.cc:1511] Processing CreateTablet for tablet e53fd092d3dd48c492e04080969cb666 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:46.678802 28145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e53fd092d3dd48c492e04080969cb666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.650590 28275 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:46.692655 28216 tablet_service.cc:1511] Processing CreateTablet for tablet e53fd092d3dd48c492e04080969cb666 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:10:46.694998 28222 tablet_service.cc:1511] Processing CreateTablet for tablet c5b0e8aadf244e7eb41e7e732bc27cf9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:46.695075 28220 tablet_service.cc:1511] Processing CreateTablet for tablet 2d0ccc53d55d4d73baf657119bf89f1c (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:46.694103 28276 ts_tablet_manager.cc:1434] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.161s	user 0.012s	sys 0.013s
I20260709 13:10:46.698624 28076 tablet_service.cc:1511] Processing CreateTablet for tablet 2d0ccc53d55d4d73baf657119bf89f1c (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:46.700907 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d0ccc53d55d4d73baf657119bf89f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.703344 28147 tablet_service.cc:1511] Processing CreateTablet for tablet c5b0e8aadf244e7eb41e7e732bc27cf9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:46.704982 28147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b0e8aadf244e7eb41e7e732bc27cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.668977 28069 tablet_service.cc:1511] Processing CreateTablet for tablet c5b0e8aadf244e7eb41e7e732bc27cf9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:10:46.701480 28224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34353a7e5a8346119cf9f0836165a1b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.713866 28222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b0e8aadf244e7eb41e7e732bc27cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.717095 28220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d0ccc53d55d4d73baf657119bf89f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.717947 28068 tablet_service.cc:1511] Processing CreateTablet for tablet 847d15cbd80f44b18e547291eed25617 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:46.737376 28276 tablet_bootstrap.cc:492] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:46.737469 28144 tablet_service.cc:1511] Processing CreateTablet for tablet 2d0ccc53d55d4d73baf657119bf89f1c (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:10:46.739169 28144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d0ccc53d55d4d73baf657119bf89f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.742913 28277 tablet_bootstrap.cc:492] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:46.743402 28216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e53fd092d3dd48c492e04080969cb666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.743479 28277 ts_tablet_manager.cc:1403] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.277s	user 0.015s	sys 0.023s
I20260709 13:10:46.752435 28146 tablet_service.cc:1511] Processing CreateTablet for tablet 847d15cbd80f44b18e547291eed25617 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:46.743584 28275 ts_tablet_manager.cc:1434] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.174s	user 0.004s	sys 0.013s
I20260709 13:10:46.744185 28068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847d15cbd80f44b18e547291eed25617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.754459 28146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847d15cbd80f44b18e547291eed25617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.761128 28186 heartbeater.cc:499] Master 127.26.83.254:42913 was elected leader, sending a full tablet report...
I20260709 13:10:46.761896 28275 tablet_bootstrap.cc:492] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:46.761103 28069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b0e8aadf244e7eb41e7e732bc27cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.777776 28280 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.778599 28280 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:46.781431 28280 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:46.798043 28147 tablet_service.cc:1511] Processing CreateTablet for tablet 33b72fb1e20942d9a3907c50252930bb (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:46.799538 28147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b72fb1e20942d9a3907c50252930bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.779536 28277 raft_consensus.cc:359] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:46.809864 28277 raft_consensus.cc:385] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:46.810417 28277 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:46.811816 28282 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.811775 28277 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:46.812495 28282 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:46.816052 28070 tablet_service.cc:1511] Processing CreateTablet for tablet 33b72fb1e20942d9a3907c50252930bb (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:46.821045 28275 tablet_bootstrap.cc:654] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.820801 28149 tablet_service.cc:1511] Processing CreateTablet for tablet 8d3461aa17604e69b08928a9bc0f8d18 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:46.829881 28149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d3461aa17604e69b08928a9bc0f8d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.822703 28072 tablet_service.cc:1511] Processing CreateTablet for tablet 8d3461aa17604e69b08928a9bc0f8d18 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:46.831074 28072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d3461aa17604e69b08928a9bc0f8d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.842902 28276 tablet_bootstrap.cc:654] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:46.825026 28282 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:46.868333 28073 tablet_service.cc:1511] Processing CreateTablet for tablet 25d46d07733d450e897e3a72d765e511 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:46.870927 28073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25d46d07733d450e897e3a72d765e511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.871245 28145 tablet_service.cc:1511] Processing CreateTablet for tablet 25d46d07733d450e897e3a72d765e511 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:46.872664 28145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25d46d07733d450e897e3a72d765e511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.879776 28277 ts_tablet_manager.cc:1434] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.126s	user 0.008s	sys 0.037s
I20260709 13:10:46.893625 28070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b72fb1e20942d9a3907c50252930bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.918244 28277 tablet_bootstrap.cc:492] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:46.924978 28219 tablet_service.cc:1511] Processing CreateTablet for tablet 847d15cbd80f44b18e547291eed25617 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:10:46.926574 28219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847d15cbd80f44b18e547291eed25617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.946972 28151 tablet_service.cc:1511] Processing CreateTablet for tablet c0a4a73cb5754c40af548cd4519da5b4 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:46.949994 28150 tablet_service.cc:1511] Processing CreateTablet for tablet e09b3a74da19442187d6c447c95018a9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:46.948592 28151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a4a73cb5754c40af548cd4519da5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.951759 28275 tablet_bootstrap.cc:492] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:46.952510 28275 ts_tablet_manager.cc:1403] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.191s	user 0.020s	sys 0.000s
I20260709 13:10:46.962206 28076 tablet_service.cc:1511] Processing CreateTablet for tablet e09b3a74da19442187d6c447c95018a9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:46.963765 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e09b3a74da19442187d6c447c95018a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.966046 28067 tablet_service.cc:1511] Processing CreateTablet for tablet c0a4a73cb5754c40af548cd4519da5b4 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:46.973874 28074 tablet_service.cc:1511] Processing CreateTablet for tablet 888e64754b8d47aebb2c7c3aba7b0ee2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:46.975458 28074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888e64754b8d47aebb2c7c3aba7b0ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.999352 28150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e09b3a74da19442187d6c447c95018a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.001955 28067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a4a73cb5754c40af548cd4519da5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.009680 28218 tablet_service.cc:1511] Processing CreateTablet for tablet 25d46d07733d450e897e3a72d765e511 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:10:47.011215 28218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25d46d07733d450e897e3a72d765e511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.014350 28144 tablet_service.cc:1511] Processing CreateTablet for tablet 888e64754b8d47aebb2c7c3aba7b0ee2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:47.014663 28224 tablet_service.cc:1511] Processing CreateTablet for tablet 8d3461aa17604e69b08928a9bc0f8d18 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:10:47.015805 28144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888e64754b8d47aebb2c7c3aba7b0ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.015959 28224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d3461aa17604e69b08928a9bc0f8d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:46.962751 28275 raft_consensus.cc:359] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.018198 28275 raft_consensus.cc:385] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.018589 28275 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.021863 28275 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.024495 28275 ts_tablet_manager.cc:1434] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.072s	user 0.002s	sys 0.005s
I20260709 13:10:47.025563 28275 tablet_bootstrap.cc:492] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:47.019057 28276 tablet_bootstrap.cc:492] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:47.029125 28276 ts_tablet_manager.cc:1403] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.292s	user 0.029s	sys 0.010s
I20260709 13:10:47.031643 28222 tablet_service.cc:1511] Processing CreateTablet for tablet e09b3a74da19442187d6c447c95018a9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:10:47.028473 28220 tablet_service.cc:1511] Processing CreateTablet for tablet 33b72fb1e20942d9a3907c50252930bb (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:10:46.997967 28277 tablet_bootstrap.cc:654] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.034106 28286 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.033958 28216 tablet_service.cc:1511] Processing CreateTablet for tablet c0a4a73cb5754c40af548cd4519da5b4 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:10:47.035583 28276 raft_consensus.cc:359] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.038879 28076 tablet_service.cc:1511] Processing CreateTablet for tablet e6b1c27f68974f4a9878c7c155cb43d2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:47.039633 28275 tablet_bootstrap.cc:654] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.040964 28222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e09b3a74da19442187d6c447c95018a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.053869 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:47.041281 28147 tablet_service.cc:1511] Processing CreateTablet for tablet e6b1c27f68974f4a9878c7c155cb43d2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:47.055511 28147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6b1c27f68974f4a9878c7c155cb43d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.043562 28073 tablet_service.cc:1511] Processing CreateTablet for tablet fab37ead976b4378b1298941cec677a8 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:47.045989 28146 tablet_service.cc:1511] Processing CreateTablet for tablet fab37ead976b4378b1298941cec677a8 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:47.049597 28276 raft_consensus.cc:385] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.050422 28286 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.051849 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6b1c27f68974f4a9878c7c155cb43d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.057085 28216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a4a73cb5754c40af548cd4519da5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.057085 28225 tablet_service.cc:1511] Processing CreateTablet for tablet e6b1c27f68974f4a9878c7c155cb43d2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:10:47.062227 28146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab37ead976b4378b1298941cec677a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.062862 28286 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:47.064941 28220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b72fb1e20942d9a3907c50252930bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.059554 28276 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.065826 28073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab37ead976b4378b1298941cec677a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.066144 28276 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.066323 28145 tablet_service.cc:1511] Processing CreateTablet for tablet aa289960fe9a4d2c8e61deb822de8412 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:47.067384 28225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6b1c27f68974f4a9878c7c155cb43d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.067763 28145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa289960fe9a4d2c8e61deb822de8412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.068894 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:47.074399 28276 ts_tablet_manager.cc:1434] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.002s
I20260709 13:10:47.059024 28221 tablet_service.cc:1511] Processing CreateTablet for tablet fab37ead976b4378b1298941cec677a8 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:10:47.090216 28067 tablet_service.cc:1511] Processing CreateTablet for tablet aa289960fe9a4d2c8e61deb822de8412 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:47.078771 28221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab37ead976b4378b1298941cec677a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.105386 28067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa289960fe9a4d2c8e61deb822de8412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.106580 28276 tablet_bootstrap.cc:492] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:47.119027 28282 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.119887 28282 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.137494 28282 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:10:47.140883 28199 leader_election.cc:343] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:47.141546 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65117.8us
I20260709 13:10:47.141384 28151 tablet_service.cc:1511] Processing CreateTablet for tablet b7ee90e83c2f49509f54dc87ea4d50e6 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:47.142340 28144 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce6e6dfcbee43d5b0550ce7fc634c38 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:47.145612 28217 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce6e6dfcbee43d5b0550ce7fc634c38 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:47.146996 28217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce6e6dfcbee43d5b0550ce7fc634c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.090977 28219 tablet_service.cc:1511] Processing CreateTablet for tablet 888e64754b8d47aebb2c7c3aba7b0ee2 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:10:47.162567 28219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888e64754b8d47aebb2c7c3aba7b0ee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.163000 28223 tablet_service.cc:1511] Processing CreateTablet for tablet b7ee90e83c2f49509f54dc87ea4d50e6 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:47.164386 28223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ee90e83c2f49509f54dc87ea4d50e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.160418 28151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ee90e83c2f49509f54dc87ea4d50e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.182308 28216 tablet_service.cc:1511] Processing CreateTablet for tablet aa289960fe9a4d2c8e61deb822de8412 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:10:47.183096 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:47.183650 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:47.184957 28216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa289960fe9a4d2c8e61deb822de8412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.188934 28143 tablet_service.cc:1511] Processing CreateTablet for tablet 7e272db0632a4262a90eaacd6e6d4378 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 13:10:47.184307 28199 leader_election.cc:343] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.194766 28144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce6e6dfcbee43d5b0550ce7fc634c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.196671 28143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e272db0632a4262a90eaacd6e6d4378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.200397 28276 tablet_bootstrap.cc:654] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.197007 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:47.205637 28280 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:47.206408 28127 leader_election.cc:343] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:47.207226 28127 leader_election.cc:343] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.213864 28076 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce6e6dfcbee43d5b0550ce7fc634c38 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:10:47.215378 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce6e6dfcbee43d5b0550ce7fc634c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.218691 28222 tablet_service.cc:1511] Processing CreateTablet for tablet 7e272db0632a4262a90eaacd6e6d4378 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:47.220250 28222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e272db0632a4262a90eaacd6e6d4378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.223526 28074 tablet_service.cc:1511] Processing CreateTablet for tablet b7ee90e83c2f49509f54dc87ea4d50e6 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:10:47.224995 28074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ee90e83c2f49509f54dc87ea4d50e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.226090 28072 tablet_service.cc:1511] Processing CreateTablet for tablet 7e272db0632a4262a90eaacd6e6d4378 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:10:47.227326 28072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e272db0632a4262a90eaacd6e6d4378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.228145 28070 tablet_service.cc:1511] Processing CreateTablet for tablet a732ff5937ea47eaaef6755cac5ad60e (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:47.233438 28070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a732ff5937ea47eaaef6755cac5ad60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.238763 28275 tablet_bootstrap.cc:492] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:47.241086 28275 ts_tablet_manager.cc:1403] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.216s	user 0.009s	sys 0.007s
I20260709 13:10:47.241662 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:47.243399 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:47.247418 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:47.248703 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:47.250473 28125 leader_election.cc:343] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.250972 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:47.250375 28147 tablet_service.cc:1511] Processing CreateTablet for tablet a732ff5937ea47eaaef6755cac5ad60e (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 13:10:47.252135 28125 leader_election.cc:343] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.252264 28147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a732ff5937ea47eaaef6755cac5ad60e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:47.253132 28050 leader_election.cc:343] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.253782 28282 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:47.252609 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:47.254834 28275 raft_consensus.cc:359] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.255470 28282 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:47.255877 28275 raft_consensus.cc:385] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.256265 28275 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.260581 28218 tablet_service.cc:1511] Processing CreateTablet for tablet a732ff5937ea47eaaef6755cac5ad60e (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:10:47.261984 28218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a732ff5937ea47eaaef6755cac5ad60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.270785 28275 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.271690 28277 tablet_bootstrap.cc:492] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
W20260709 13:10:47.274572 28052 leader_election.cc:343] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.274420 28149 tablet_service.cc:1511] Processing CreateTablet for tablet fc1588fd3df04c2c8354baa741ddeca9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:47.275173 28067 tablet_service.cc:1511] Processing CreateTablet for tablet fc1588fd3df04c2c8354baa741ddeca9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:47.296070 28149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1588fd3df04c2c8354baa741ddeca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.296931 28144 tablet_service.cc:1511] Processing CreateTablet for tablet 6810464df4a245f49df908fc877020d7 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:47.297963 28144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6810464df4a245f49df908fc877020d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.296092 28067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1588fd3df04c2c8354baa741ddeca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.289496 28225 tablet_service.cc:1511] Processing CreateTablet for tablet fc1588fd3df04c2c8354baa741ddeca9 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:10:47.292665 28282 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.301427 28282 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.301994 28225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1588fd3df04c2c8354baa741ddeca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.307016 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:47.308506 28125 leader_election.cc:343] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.308969 28277 ts_tablet_manager.cc:1403] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.399s	user 0.012s	sys 0.024s
I20260709 13:10:47.310635 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:47.311985 28277 raft_consensus.cc:359] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.313405 28150 tablet_service.cc:1511] Processing CreateTablet for tablet d952e9d07d9e4c5ba47d0f8a9601dc73 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:47.314766 28150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d952e9d07d9e4c5ba47d0f8a9601dc73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.296113 28052 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:47.317625 28286 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:47.317824 28127 leader_election.cc:343] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.318308 28127 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:47.313023 28277 raft_consensus.cc:385] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.319197 28282 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:47.320963 28219 tablet_service.cc:1511] Processing CreateTablet for tablet d952e9d07d9e4c5ba47d0f8a9601dc73 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:47.321735 28282 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:47.322399 28219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d952e9d07d9e4c5ba47d0f8a9601dc73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.323371 28072 tablet_service.cc:1511] Processing CreateTablet for tablet d952e9d07d9e4c5ba47d0f8a9601dc73 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:10:47.324618 28072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d952e9d07d9e4c5ba47d0f8a9601dc73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.329931 28073 tablet_service.cc:1511] Processing CreateTablet for tablet 60a7d81169af4acc91d8f3b342ac4885 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:47.331393 28073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a7d81169af4acc91d8f3b342ac4885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.319151 28277 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.332717 28277 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.334491 28276 tablet_bootstrap.cc:492] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:47.334914 28277 ts_tablet_manager.cc:1434] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 13:10:47.336081 28277 tablet_bootstrap.cc:492] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:47.334985 28276 ts_tablet_manager.cc:1403] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.253s	user 0.011s	sys 0.017s
I20260709 13:10:47.340960 28074 tablet_service.cc:1511] Processing CreateTablet for tablet 6810464df4a245f49df908fc877020d7 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:47.342303 28074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6810464df4a245f49df908fc877020d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.357056 28276 raft_consensus.cc:359] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.357882 28276 raft_consensus.cc:385] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.358186 28276 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.358966 28276 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:47.362664 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 44872.2us
I20260709 13:10:47.363960 28217 tablet_service.cc:1511] Processing CreateTablet for tablet 6810464df4a245f49df908fc877020d7 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:10:47.366542 28222 tablet_service.cc:1511] Processing CreateTablet for tablet 60a7d81169af4acc91d8f3b342ac4885 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:47.371457 28222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a7d81169af4acc91d8f3b342ac4885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.375910 28275 ts_tablet_manager.cc:1434] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.134s	user 0.005s	sys 0.011s
I20260709 13:10:47.379205 28151 tablet_service.cc:1511] Processing CreateTablet for tablet 9e12858e91dd456a9f422bca6d2d6ac3 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:47.380587 28151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e12858e91dd456a9f422bca6d2d6ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.380527 28076 tablet_service.cc:1511] Processing CreateTablet for tablet cd81a684ef6b4e0f9cc48bc3bcc280e8 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:47.381587 28275 tablet_bootstrap.cc:492] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:47.381990 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd81a684ef6b4e0f9cc48bc3bcc280e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.384881 28219 tablet_service.cc:1511] Processing CreateTablet for tablet 9e12858e91dd456a9f422bca6d2d6ac3 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:47.387995 28275 tablet_bootstrap.cc:654] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.392149 28147 tablet_service.cc:1511] Processing CreateTablet for tablet 60a7d81169af4acc91d8f3b342ac4885 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:10:47.393466 28147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a7d81169af4acc91d8f3b342ac4885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.394310 28276 ts_tablet_manager.cc:1434] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.000s
I20260709 13:10:47.395385 28276 tablet_bootstrap.cc:492] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:47.396721 28070 tablet_service.cc:1511] Processing CreateTablet for tablet 9e12858e91dd456a9f422bca6d2d6ac3 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:10:47.398046 28070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e12858e91dd456a9f422bca6d2d6ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.400851 28217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6810464df4a245f49df908fc877020d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.402223 28219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e12858e91dd456a9f422bca6d2d6ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.402920 28143 tablet_service.cc:1511] Processing CreateTablet for tablet cd81a684ef6b4e0f9cc48bc3bcc280e8 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:47.404215 28143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd81a684ef6b4e0f9cc48bc3bcc280e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.409340 28223 tablet_service.cc:1511] Processing CreateTablet for tablet cd81a684ef6b4e0f9cc48bc3bcc280e8 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:10:47.410730 28223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd81a684ef6b4e0f9cc48bc3bcc280e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.410859 28069 tablet_service.cc:1511] Processing CreateTablet for tablet 17fbe81a71fd4f8c83a65145cf2f0682 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:47.412196 28069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17fbe81a71fd4f8c83a65145cf2f0682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.413708 28218 tablet_service.cc:1511] Processing CreateTablet for tablet 17fbe81a71fd4f8c83a65145cf2f0682 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:47.415154 28218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17fbe81a71fd4f8c83a65145cf2f0682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.415619 28224 tablet_service.cc:1511] Processing CreateTablet for tablet 9f16469a2e44408daf3969b51ae89aa6 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:47.416222 28145 tablet_service.cc:1511] Processing CreateTablet for tablet 6ef726aecabb47089dcd217bfb8bfefc (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:47.416849 28224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f16469a2e44408daf3969b51ae89aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.417900 28146 tablet_service.cc:1511] Processing CreateTablet for tablet 17fbe81a71fd4f8c83a65145cf2f0682 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:10:47.413763 28073 tablet_service.cc:1511] Processing CreateTablet for tablet 6ef726aecabb47089dcd217bfb8bfefc (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:47.421465 28068 tablet_service.cc:1511] Processing CreateTablet for tablet 9f16469a2e44408daf3969b51ae89aa6 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:10:47.422367 28073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ef726aecabb47089dcd217bfb8bfefc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.423367 28068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f16469a2e44408daf3969b51ae89aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.428761 28280 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.429373 28277 tablet_bootstrap.cc:654] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.430694 28071 tablet_service.cc:1511] Processing CreateTablet for tablet f703ac7a390a45efae315d629c1dad89 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:47.429409 28280 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.434423 28280 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:47.436450 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:47.436942 28148 tablet_service.cc:1511] Processing CreateTablet for tablet 9f16469a2e44408daf3969b51ae89aa6 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:10:47.438766 28199 leader_election.cc:343] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.445053 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:10:47.446915 28199 leader_election.cc:343] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.447387 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:47.448570 28280 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:47.449759 28216 tablet_service.cc:1511] Processing CreateTablet for tablet 6ef726aecabb47089dcd217bfb8bfefc (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:10:47.452180 28148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f16469a2e44408daf3969b51ae89aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.452946 28216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ef726aecabb47089dcd217bfb8bfefc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.455976 28145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ef726aecabb47089dcd217bfb8bfefc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.460165 28276 tablet_bootstrap.cc:654] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.461364 28146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17fbe81a71fd4f8c83a65145cf2f0682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.469888 28147 tablet_service.cc:1511] Processing CreateTablet for tablet f703ac7a390a45efae315d629c1dad89 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:47.471433 28147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f703ac7a390a45efae315d629c1dad89. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:10:47.472394 28032 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:47.476014 28225 tablet_service.cc:1511] Processing CreateTablet for tablet f703ac7a390a45efae315d629c1dad89 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:10:47.478044 28225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f703ac7a390a45efae315d629c1dad89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.480851 28071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f703ac7a390a45efae315d629c1dad89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.508038 28147 tablet_service.cc:1511] Processing CreateTablet for tablet 552d81eee50848758aae23da5ed8927b (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:47.509982 28067 tablet_service.cc:1511] Processing CreateTablet for tablet c20222bc64224781bd97e8b75f2ddfa1 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:47.511005 28147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552d81eee50848758aae23da5ed8927b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.511314 28067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20222bc64224781bd97e8b75f2ddfa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.509982 28220 tablet_service.cc:1511] Processing CreateTablet for tablet 552d81eee50848758aae23da5ed8927b (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:47.514034 28220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552d81eee50848758aae23da5ed8927b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.515903 28150 tablet_service.cc:1511] Processing CreateTablet for tablet c20222bc64224781bd97e8b75f2ddfa1 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:47.538722 28076 tablet_service.cc:1511] Processing CreateTablet for tablet 552d81eee50848758aae23da5ed8927b (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:10:47.540777 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552d81eee50848758aae23da5ed8927b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.528002 28150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20222bc64224781bd97e8b75f2ddfa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.555083 28218 tablet_service.cc:1511] Processing CreateTablet for tablet c20222bc64224781bd97e8b75f2ddfa1 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:10:47.556463 28218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20222bc64224781bd97e8b75f2ddfa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.557600 28147 tablet_service.cc:1511] Processing CreateTablet for tablet 0720cab120de4047b423cb89ed85154a (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:47.558866 28147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0720cab120de4047b423cb89ed85154a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.572120 28223 tablet_service.cc:1511] Processing CreateTablet for tablet 0720cab120de4047b423cb89ed85154a (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:47.579035 28275 tablet_bootstrap.cc:492] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:47.579623 28275 ts_tablet_manager.cc:1403] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.198s	user 0.013s	sys 0.013s
I20260709 13:10:47.582528 28275 raft_consensus.cc:359] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.583225 28275 raft_consensus.cc:385] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.583479 28275 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.584079 28275 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.585625 28069 tablet_service.cc:1511] Processing CreateTablet for tablet 0720cab120de4047b423cb89ed85154a (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:10:47.586113 28275 ts_tablet_manager.cc:1434] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:10:47.586704 28069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0720cab120de4047b423cb89ed85154a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.588227 28275 tablet_bootstrap.cc:492] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:47.591812 28076 tablet_service.cc:1511] Processing CreateTablet for tablet ceb515ffdfa04f2f977a4c5aa1aecbee (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:47.593084 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceb515ffdfa04f2f977a4c5aa1aecbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.594167 28275 tablet_bootstrap.cc:654] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.597215 28286 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.597685 28286 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.599817 28286 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:47.601251 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:47.601915 28161 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:47.603225 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:47.604102 28286 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:47.604472 28286 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.601758 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:47.611305 28277 tablet_bootstrap.cc:492] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:47.611815 28277 ts_tablet_manager.cc:1403] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.276s	user 0.023s	sys 0.029s
I20260709 13:10:47.612210 28286 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:47.614519 28052 leader_election.cc:343] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.618000 28275 tablet_bootstrap.cc:492] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:47.618503 28275 ts_tablet_manager.cc:1403] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.011s
I20260709 13:10:47.620064 28277 raft_consensus.cc:359] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.621452 28275 raft_consensus.cc:359] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.625061 28223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0720cab120de4047b423cb89ed85154a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.625160 28286 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.625770 28148 tablet_service.cc:1511] Processing CreateTablet for tablet ceb515ffdfa04f2f977a4c5aa1aecbee (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:47.633980 28275 raft_consensus.cc:385] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.634330 28275 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.634994 28148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceb515ffdfa04f2f977a4c5aa1aecbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.635682 28276 tablet_bootstrap.cc:492] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:47.635197 28275 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.636158 28276 ts_tablet_manager.cc:1403] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.241s	user 0.023s	sys 0.018s
I20260709 13:10:47.639397 28276 raft_consensus.cc:359] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.633786 28277 raft_consensus.cc:385] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.640096 28276 raft_consensus.cc:385] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.640456 28276 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.640265 28277 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.643930 28277 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.643016 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:47.645962 28161 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:47.641234 28276 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.645247 28275 ts_tablet_manager.cc:1434] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:10:47.645325 28286 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:47.646533 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:10:47.651546 28052 leader_election.cc:343] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.658294 28275 tablet_bootstrap.cc:492] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:47.662081 28161 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:47.663391 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:47.664289 28286 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:47.664786 28286 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:47.665956 28276 ts_tablet_manager.cc:1434] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.004s
I20260709 13:10:47.665607 28286 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.667060 28276 tablet_bootstrap.cc:492] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:47.672183 28073 tablet_service.cc:1511] Processing CreateTablet for tablet 6fbb28a09242477abe442ecc8a938738 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:47.673627 28073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fbb28a09242477abe442ecc8a938738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.676249 28275 tablet_bootstrap.cc:654] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.677064 28222 tablet_service.cc:1511] Processing CreateTablet for tablet ceb515ffdfa04f2f977a4c5aa1aecbee (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:10:47.682564 28277 ts_tablet_manager.cc:1434] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.069s	user 0.005s	sys 0.009s
I20260709 13:10:47.684166 28277 tablet_bootstrap.cc:492] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:47.689388 28222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceb515ffdfa04f2f977a4c5aa1aecbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.693480 28148 tablet_service.cc:1511] Processing CreateTablet for tablet 6fbb28a09242477abe442ecc8a938738 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:47.695803 28148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fbb28a09242477abe442ecc8a938738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.708664 28150 tablet_service.cc:1511] Processing CreateTablet for tablet 66540b865ca24721910c667ccd790b77 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:47.709642 28276 tablet_bootstrap.cc:654] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.709909 28150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66540b865ca24721910c667ccd790b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.710695 28076 tablet_service.cc:1511] Processing CreateTablet for tablet 6f5f1be456de4894af76dffdad9ea6a3 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:47.711891 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f5f1be456de4894af76dffdad9ea6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.708694 28067 tablet_service.cc:1511] Processing CreateTablet for tablet 66540b865ca24721910c667ccd790b77 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:47.717360 28067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66540b865ca24721910c667ccd790b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.710613 28220 tablet_service.cc:1511] Processing CreateTablet for tablet 66540b865ca24721910c667ccd790b77 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:10:47.720284 27983 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:47.723132 28220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66540b865ca24721910c667ccd790b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.728488 28146 tablet_service.cc:1511] Processing CreateTablet for tablet 6f5f1be456de4894af76dffdad9ea6a3 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:47.729840 28146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f5f1be456de4894af76dffdad9ea6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.744524 28223 tablet_service.cc:1511] Processing CreateTablet for tablet 6fbb28a09242477abe442ecc8a938738 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:10:47.746513 28223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fbb28a09242477abe442ecc8a938738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.747270 28301 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.747768 28301 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.748272 28150 tablet_service.cc:1511] Processing CreateTablet for tablet 61f69a3a4f9a410e9fc9d6e9c38de9f1 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:47.749475 28150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f69a3a4f9a410e9fc9d6e9c38de9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.750989 28216 tablet_service.cc:1511] Processing CreateTablet for tablet 6f5f1be456de4894af76dffdad9ea6a3 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:10:47.752666 28216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f5f1be456de4894af76dffdad9ea6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.756335 28076 tablet_service.cc:1511] Processing CreateTablet for tablet 61f69a3a4f9a410e9fc9d6e9c38de9f1 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:47.757503 28076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f69a3a4f9a410e9fc9d6e9c38de9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.758221 28301 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:47.759918 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:47.736985 28277 tablet_bootstrap.cc:654] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:47.761438 28127 leader_election.cc:343] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.762041 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:47.763041 28275 tablet_bootstrap.cc:492] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:47.763525 28275 ts_tablet_manager.cc:1403] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.111s	user 0.017s	sys 0.016s
W20260709 13:10:47.763582 28125 leader_election.cc:343] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.764088 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:47.766551 28301 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:47.774255 28275 raft_consensus.cc:359] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.775342 28275 raft_consensus.cc:385] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.775732 28275 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.790673 28222 tablet_service.cc:1511] Processing CreateTablet for tablet 61f69a3a4f9a410e9fc9d6e9c38de9f1 (DEFAULT_TABLE table=test-workload [id=33cb0926ab7040f8a1735610acb16879]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:10:47.792426 28222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f69a3a4f9a410e9fc9d6e9c38de9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:47.796833 28275 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.799266 28276 tablet_bootstrap.cc:492] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:47.799775 28276 ts_tablet_manager.cc:1403] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.133s	user 0.013s	sys 0.001s
I20260709 13:10:47.803071 28276 raft_consensus.cc:359] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.803795 28276 raft_consensus.cc:385] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.804085 28276 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.805711 28275 ts_tablet_manager.cc:1434] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.003s
I20260709 13:10:47.808401 28275 tablet_bootstrap.cc:492] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:47.808923 28280 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.809448 28280 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.811460 28280 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:47.813025 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:47.814473 28199 leader_election.cc:343] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.814781 28301 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.815008 28275 tablet_bootstrap.cc:654] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.815328 28301 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.817543 28276 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.818679 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:47.819361 28161 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:47.820195 28277 tablet_bootstrap.cc:492] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:47.820364 28276 ts_tablet_manager.cc:1434] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:10:47.820688 28277 ts_tablet_manager.cc:1403] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.137s	user 0.023s	sys 0.012s
I20260709 13:10:47.821002 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:47.822238 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:47.818404 28301 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:47.829108 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:10:47.835548 28125 leader_election.cc:343] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.837646 28280 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.839020 28280 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.840596 28280 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:47.840986 28276 tablet_bootstrap.cc:492] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:47.848186 28276 tablet_bootstrap.cc:654] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.850039 28277 raft_consensus.cc:359] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.853945 28277 raft_consensus.cc:385] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.855379 28277 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.853302 28314 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:47.857112 28286 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.858891 28286 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.861107 28286 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:47.863381 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:47.863906 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:47.864275 28234 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:47.858891 28314 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.866150 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:47.866748 28314 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:47.867507 28286 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:47.868060 28286 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:47.868860 28050 leader_election.cc:343] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.868994 28277 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.868477 28286 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:47.873274 28277 ts_tablet_manager.cc:1434] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.001s
I20260709 13:10:47.874840 28277 tablet_bootstrap.cc:492] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
W20260709 13:10:47.879130 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 57094.3us
W20260709 13:10:47.880642 28127 leader_election.cc:343] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.881399 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:47.882395 28301 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:47.884582 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:47.887024 28085 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:47.889384 28275 tablet_bootstrap.cc:492] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:47.889751 28276 tablet_bootstrap.cc:492] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:47.889917 28275 ts_tablet_manager.cc:1403] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.082s	user 0.007s	sys 0.017s
I20260709 13:10:47.891335 28276 ts_tablet_manager.cc:1403] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.026s
I20260709 13:10:47.892998 28275 raft_consensus.cc:359] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.893805 28275 raft_consensus.cc:385] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.893626 28286 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.893656 28276 raft_consensus.cc:359] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.894237 28275 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.894631 28276 raft_consensus.cc:385] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.894961 28276 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.895208 28275 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.895707 28276 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.896855 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:47.898353 28276 ts_tablet_manager.cc:1434] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.005s
I20260709 13:10:47.899436 28276 tablet_bootstrap.cc:492] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:47.900506 28314 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:47.898371 28050 leader_election.cc:343] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.902616 28314 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:47.902994 28275 ts_tablet_manager.cc:1434] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:10:47.904048 28275 tablet_bootstrap.cc:492] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:47.904621 28286 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:47.904879 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:47.905473 28234 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:47.905977 28276 tablet_bootstrap.cc:654] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.909688 28275 tablet_bootstrap.cc:654] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.910439 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:47.913911 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:47.914556 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:47.915155 28161 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:47.915428 28277 tablet_bootstrap.cc:654] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.917063 28276 tablet_bootstrap.cc:492] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:47.917640 28276 ts_tablet_manager.cc:1403] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.007s
W20260709 13:10:47.918920 28199 leader_election.cc:343] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.919782 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:47.924970 28314 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:47.925477 28314 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.925794 28314 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:47.926510 28277 tablet_bootstrap.cc:492] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:47.927032 28277 ts_tablet_manager.cc:1403] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.000s
I20260709 13:10:47.926966 28276 raft_consensus.cc:359] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.927841 28276 raft_consensus.cc:385] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.928135 28276 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.928444 28234 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:47.928900 28276 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.929934 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:47.930845 28286 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:47.931447 28286 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:47.932569 28276 ts_tablet_manager.cc:1434] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
W20260709 13:10:47.933379 28199 leader_election.cc:343] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.932699 28286 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.933820 28276 tablet_bootstrap.cc:492] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:47.930565 28277 raft_consensus.cc:359] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.939646 28277 raft_consensus.cc:385] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.940131 28277 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.941510 28276 tablet_bootstrap.cc:654] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.942855 28277 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.945174 28314 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.949250 28277 ts_tablet_manager.cc:1434] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 13:10:47.949396 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:47.950554 28085 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:47.951217 28277 tablet_bootstrap.cc:492] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:47.957870 28275 tablet_bootstrap.cc:492] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:47.958595 28275 ts_tablet_manager.cc:1403] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.019s
I20260709 13:10:47.924373 28161 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:47.962167 28314 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:47.961958 28275 raft_consensus.cc:359] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.963014 28275 raft_consensus.cc:385] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:47.963383 28275 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:47.964097 28275 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.964571 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:47.966492 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:47.970013 28085 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
W20260709 13:10:47.968474 28199 leader_election.cc:343] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:47.972801 28314 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:47.973493 28314 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:47.974347 28314 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.976908 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:47.978192 28317 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:47.978695 28317 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:47.978951 28275 ts_tablet_manager.cc:1434] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:10:47.979696 28317 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:47.971534 28277 tablet_bootstrap.cc:654] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.985908 28275 tablet_bootstrap.cc:492] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:47.986001 27983 catalog_manager.cc:5697] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:47.989413 28276 tablet_bootstrap.cc:492] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:47.991747 28276 ts_tablet_manager.cc:1403] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.058s	user 0.018s	sys 0.026s
I20260709 13:10:47.993003 28275 tablet_bootstrap.cc:654] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:47.997170 28276 raft_consensus.cc:359] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.001622 28276 raft_consensus.cc:385] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.002738 28276 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.004714 28276 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.006153 28275 tablet_bootstrap.cc:492] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.009052 28275 ts_tablet_manager.cc:1403] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.006s
I20260709 13:10:48.012233 28275 raft_consensus.cc:359] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.013039 28275 raft_consensus.cc:385] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.013386 28275 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.014205 28275 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.016088 28276 ts_tablet_manager.cc:1434] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:10:48.016659 28275 ts_tablet_manager.cc:1434] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:48.017064 28276 tablet_bootstrap.cc:492] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.016615 27983 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:48.018968 28275 tablet_bootstrap.cc:492] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:48.024252 28276 tablet_bootstrap.cc:654] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.025463 28317 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.025617 28275 tablet_bootstrap.cc:654] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.025985 28317 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.028147 28317 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:48.029976 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:48.030458 28277 tablet_bootstrap.cc:492] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:48.030772 28161 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:48.030989 28277 ts_tablet_manager.cc:1403] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.080s	user 0.019s	sys 0.006s
I20260709 13:10:48.031697 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.033831 28277 raft_consensus.cc:359] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.034267 28199 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:48.034637 28277 raft_consensus.cc:385] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.035022 28277 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.035430 28317 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.035902 28277 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.037328 27983 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:48.038652 28277 ts_tablet_manager.cc:1434] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:48.039749 28277 tablet_bootstrap.cc:492] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:48.047053 28277 tablet_bootstrap.cc:654] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.049256 28317 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:48.058207 28275 tablet_bootstrap.cc:492] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.058702 28275 ts_tablet_manager.cc:1403] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.001s
I20260709 13:10:48.059003 28276 tablet_bootstrap.cc:492] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.059474 28276 ts_tablet_manager.cc:1403] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.043s	user 0.024s	sys 0.003s
I20260709 13:10:48.061564 28275 raft_consensus.cc:359] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.062320 28277 tablet_bootstrap.cc:492] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:48.062366 28275 raft_consensus.cc:385] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.062209 28276 raft_consensus.cc:359] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.062865 28275 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.062961 28276 raft_consensus.cc:385] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.062930 28277 ts_tablet_manager.cc:1403] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.000s
I20260709 13:10:48.063396 28276 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.063894 28275 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.064239 28276 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.066531 28276 ts_tablet_manager.cc:1434] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:48.066260 28277 raft_consensus.cc:359] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.066936 28277 raft_consensus.cc:385] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.067128 28277 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.067983 28276 tablet_bootstrap.cc:492] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.067799 28277 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.070084 28277 ts_tablet_manager.cc:1434] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:48.071656 28275 ts_tablet_manager.cc:1434] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:10:48.072616 28275 tablet_bootstrap.cc:492] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:48.074191 28276 tablet_bootstrap.cc:654] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.079663 28275 tablet_bootstrap.cc:654] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.083858 28301 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.084435 28301 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.086444 28276 tablet_bootstrap.cc:492] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.086759 28301 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.087347 28301 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.087891 28301 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.088443 28276 ts_tablet_manager.cc:1403] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.002s
I20260709 13:10:48.089056 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.089742 28234 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:48.090265 28301 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.092015 28277 tablet_bootstrap.cc:492] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:48.092764 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.093134 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.092526 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:10:48.095011 28125 leader_election.cc:343] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.095279 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.096081 28234 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
W20260709 13:10:48.096261 28125 leader_election.cc:343] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.098178 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:48.098309 28276 raft_consensus.cc:359] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.099301 28276 raft_consensus.cc:385] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.099406 28277 tablet_bootstrap.cc:654] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.099527 28317 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.099736 28276 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.100095 28317 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.101722 28333 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:48.102267 28333 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.102680 28333 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.102725 28317 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:48.105123 28276 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.113013 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:48.113655 28276 ts_tablet_manager.cc:1434] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.002s
I20260709 13:10:48.114123 28161 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:48.115049 28276 tablet_bootstrap.cc:492] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.117655 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.118654 28086 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:48.119081 28275 tablet_bootstrap.cc:492] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.119632 28275 ts_tablet_manager.cc:1403] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.047s	user 0.022s	sys 0.021s
I20260709 13:10:48.122567 28276 tablet_bootstrap.cc:654] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.122341 28275 raft_consensus.cc:359] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.123032 28275 raft_consensus.cc:385] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.123333 28275 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.124125 28275 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.126751 28301 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:48.128451 28301 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.131592 28301 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.128654 28333 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.145931 28286 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:48.147437 28333 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.148423 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:48.127614 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:48.135087 28275 ts_tablet_manager.cc:1434] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.001s
W20260709 13:10:48.150007 28125 leader_election.cc:343] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.150736 28275 tablet_bootstrap.cc:492] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:48.151115 28277 tablet_bootstrap.cc:492] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:48.151703 28277 ts_tablet_manager.cc:1403] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.060s	user 0.030s	sys 0.029s
I20260709 13:10:48.154968 28277 raft_consensus.cc:359] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.155930 28277 raft_consensus.cc:385] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.156287 28277 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.157132 28317 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:48.157526 28317 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.157943 28317 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.157214 28277 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.158703 28301 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.160353 28277 ts_tablet_manager.cc:1434] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:10:48.160980 28301 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.162039 28277 tablet_bootstrap.cc:492] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:48.143373 28276 tablet_bootstrap.cc:492] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.163152 28276 ts_tablet_manager.cc:1403] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.048s	user 0.007s	sys 0.016s
I20260709 13:10:48.163893 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:48.164546 28234 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.165237 28317 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.165967 28276 raft_consensus.cc:359] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.166625 28276 raft_consensus.cc:385] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.166893 28276 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.167395 28317 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:48.167639 28275 tablet_bootstrap.cc:654] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.169684 28277 tablet_bootstrap.cc:654] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.169592 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:48.170338 28160 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.170415 28276 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.170871 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:48.175205 28235 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.175450 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:48.176506 28276 ts_tablet_manager.cc:1434] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
W20260709 13:10:48.176857 28125 leader_election.cc:343] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.178969 28160 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:48.184005 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:48.184754 28086 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.185158 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:48.186197 28314 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:48.186679 28314 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:48.187502 28314 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.190858 28276 tablet_bootstrap.cc:492] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.198580 28235 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:48.199870 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:48.200776 28301 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:48.201464 28275 tablet_bootstrap.cc:492] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.201951 28301 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:48.202895 28276 tablet_bootstrap.cc:654] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.203302 28301 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.201962 28275 ts_tablet_manager.cc:1403] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.004s
I20260709 13:10:48.201463 28234 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:48.206430 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:48.208022 28333 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:48.208621 28333 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:48.209484 28333 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.224805 28275 raft_consensus.cc:359] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.226706 28275 raft_consensus.cc:385] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.227116 28275 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.236697 28286 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:48.229579 28275 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.242347 28275 ts_tablet_manager.cc:1434] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.003s
I20260709 13:10:48.243381 28275 tablet_bootstrap.cc:492] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:48.253819 28277 tablet_bootstrap.cc:492] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:48.256479 28277 ts_tablet_manager.cc:1403] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.095s	user 0.038s	sys 0.024s
I20260709 13:10:48.251545 28333 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.260769 28277 raft_consensus.cc:359] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.261974 28277 raft_consensus.cc:385] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.261405 28333 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.262832 28277 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.264719 28086 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:48.271006 28333 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.278784 28286 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.279372 28286 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.286485 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:48.287524 28161 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 1: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:10:48.270848 28277 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.272279 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.289497 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.289920 28234 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:48.291342 28277 ts_tablet_manager.cc:1434] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:10:48.276360 28276 tablet_bootstrap.cc:492] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.291481 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:48.292019 28276 ts_tablet_manager.cc:1403] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.101s	user 0.024s	sys 0.018s
I20260709 13:10:48.292415 28277 tablet_bootstrap.cc:492] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:48.294935 28276 raft_consensus.cc:359] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.295665 28276 raft_consensus.cc:385] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.295928 28276 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.276899 28275 tablet_bootstrap.cc:654] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.296903 28286 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.297408 28306 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.296666 28276 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.275523 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.301707 28276 ts_tablet_manager.cc:1434] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:10:48.301970 28125 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:48.302662 28276 tablet_bootstrap.cc:492] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.302796 28301 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.307395 27983 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:48.311674 28301 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:48.311975 28277 tablet_bootstrap.cc:654] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.314311 28276 tablet_bootstrap.cc:654] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.307425 28306 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:48.324496 28276 tablet_bootstrap.cc:492] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.325320 28276 ts_tablet_manager.cc:1403] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.002s
I20260709 13:10:48.328118 28276 raft_consensus.cc:359] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.328629 28275 tablet_bootstrap.cc:492] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.328914 28276 raft_consensus.cc:385] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.329362 28275 ts_tablet_manager.cc:1403] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.086s	user 0.023s	sys 0.004s
I20260709 13:10:48.329366 28276 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.328589 27984 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.330618 28276 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.332119 28275 raft_consensus.cc:359] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.332826 28275 raft_consensus.cc:385] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.333089 28275 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.333189 28301 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.333516 28276 ts_tablet_manager.cc:1434] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:10:48.334558 28276 tablet_bootstrap.cc:492] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.333930 28275 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.333920 28301 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.331431 27984 catalog_manager.cc:5697] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.337232 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.337515 28301 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.338353 28306 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.338758 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.338958 28306 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.339534 28235 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:48.341290 28306 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.341463 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:48.342386 28333 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:48.342815 28333 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.343199 28333 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.343731 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:48.344420 28160 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:48.345921 28275 ts_tablet_manager.cc:1434] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.005s
I20260709 13:10:48.345844 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:48.346951 28275 tablet_bootstrap.cc:492] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
W20260709 13:10:48.347143 28125 leader_election.cc:343] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.348652 28347 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:48.349081 28347 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.349398 28347 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.352931 28306 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:48.356148 28275 tablet_bootstrap.cc:654] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.359584 28333 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.364223 28314 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:48.371376 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 79990.7us
I20260709 13:10:48.373215 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.373755 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:48.374001 28235 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
W20260709 13:10:48.375660 28125 leader_election.cc:343] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.379161 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:48.380020 28235 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.389580 28314 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.390498 28314 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.397073 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:48.399107 28199 leader_election.cc:343] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.399787 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:48.400378 28314 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:48.400583 28160 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:48.401684 28349 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.402801 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:48.405357 28314 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:48.405838 28314 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.406263 28314 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.408849 28333 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.358027 28347 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.409394 28306 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:48.399768 28235 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:48.411826 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:48.410419 28349 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.421057 28301 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.421614 28301 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.423679 28301 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.424124 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.424856 28233 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:48.426222 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:48.427341 28356 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:48.427747 28356 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.428061 28356 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.433430 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:48.434221 28160 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:48.435964 28356 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.436889 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.439623 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:48.439623 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:48.441077 28233 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.442589 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:48.443184 28234 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.445513 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.447199 28125 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:48.448359 28301 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:48.450290 28234 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:48.451614 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:48.452549 28286 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:48.453323 28286 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:48.453552 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:48.454123 28161 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.454275 28286 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.440346 28160 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.413506 28333 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:48.414417 28277 tablet_bootstrap.cc:492] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:48.464987 28347 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.465224 28277 ts_tablet_manager.cc:1403] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.173s	user 0.024s	sys 0.019s
I20260709 13:10:48.465663 28349 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.466027 28333 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:48.466917 28333 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.468341 28161 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:48.468382 28277 raft_consensus.cc:359] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.469544 28277 raft_consensus.cc:385] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.469952 28277 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.470862 28277 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.473587 28277 ts_tablet_manager.cc:1434] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:10:48.474519 28277 tablet_bootstrap.cc:492] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
W20260709 13:10:48.474820 28032 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:48.416479 28276 tablet_bootstrap.cc:654] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.477771 28161 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.484652 28317 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:48.485911 28233 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:48.487159 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:48.488914 28306 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:48.489365 28306 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:48.490165 28306 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.495033 28317 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:48.496134 28276 tablet_bootstrap.cc:492] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.496719 28276 ts_tablet_manager.cc:1403] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.162s	user 0.009s	sys 0.009s
I20260709 13:10:48.499893 28276 raft_consensus.cc:359] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.507071 28356 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.500653 28276 raft_consensus.cc:385] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.510555 28276 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.510979 28277 tablet_bootstrap.cc:654] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.512187 28276 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.511933 27984 catalog_manager.cc:5697] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.514132 28160 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:48.516309 28333 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.516793 28276 ts_tablet_manager.cc:1434] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:10:48.516812 28333 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.517820 28276 tablet_bootstrap.cc:492] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.520394 28333 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.523976 28276 tablet_bootstrap.cc:654] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.526878 27983 catalog_manager.cc:5697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.534742 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.535475 28085 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:48.536916 28125 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:48.538097 28333 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:48.538563 28333 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.538993 28333 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.546101 28317 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:48.550526 28286 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.551038 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.551245 28286 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.551789 28235 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:48.555380 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.556218 28235 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:48.557961 28052 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:48.560432 28356 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:48.560344 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:48.561565 28161 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:48.563511 28356 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.575174 28286 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.575438 28356 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.578760 28276 tablet_bootstrap.cc:492] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.578725 27984 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.582185 28276 ts_tablet_manager.cc:1403] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.065s	user 0.018s	sys 0.013s
I20260709 13:10:48.584913 28276 raft_consensus.cc:359] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.585836 28276 raft_consensus.cc:385] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.586149 28276 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.464152 28314 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.588241 28314 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:48.564793 28333 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.592265 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:48.592867 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:48.592555 28276 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.593395 28161 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.593487 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:48.594164 28086 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.594970 28276 ts_tablet_manager.cc:1434] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
W20260709 13:10:48.595469 28199 leader_election.cc:343] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.595885 28276 tablet_bootstrap.cc:492] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.592213 28277 tablet_bootstrap.cc:492] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:48.597254 28277 ts_tablet_manager.cc:1403] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.123s	user 0.027s	sys 0.024s
I20260709 13:10:48.602130 28276 tablet_bootstrap.cc:654] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.592216 28314 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:10:48.604430 28277 raft_consensus.cc:359] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.605151 28277 raft_consensus.cc:385] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.605419 28277 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.606031 28277 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.608395 28275 tablet_bootstrap.cc:492] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.608891 28275 ts_tablet_manager.cc:1403] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.262s	user 0.064s	sys 0.090s
I20260709 13:10:48.608961 28277 ts_tablet_manager.cc:1434] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:10:48.609830 28161 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:48.610203 28277 tablet_bootstrap.cc:492] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:48.611259 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:48.611647 28275 raft_consensus.cc:359] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.612367 28275 raft_consensus.cc:385] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.612612 28275 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.612572 28314 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:48.613621 28314 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:48.614676 28333 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.614465 28314 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.617682 28277 tablet_bootstrap.cc:654] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.618191 28275 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.620318 28275 ts_tablet_manager.cc:1434] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:10:48.622761 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:48.623354 28235 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.624825 28275 tablet_bootstrap.cc:492] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:48.622251 28086 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:48.636132 28235 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:48.637573 28125 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:48.639236 28275 tablet_bootstrap.cc:654] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.640671 28333 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:48.641185 28333 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:48.644358 28333 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.638662 28356 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.645957 28276 tablet_bootstrap.cc:492] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.646557 28276 ts_tablet_manager.cc:1403] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.008s
I20260709 13:10:48.648252 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:48.650601 28286 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.650337 28356 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.651351 28286 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.653425 28301 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:48.653824 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:48.654405 28235 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.653532 28161 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.656091 28286 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.657997 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:48.658895 28233 raft_consensus.cc:2393] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 1: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:10:48.665355 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:48.670171 28276 raft_consensus.cc:359] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.662329 27982 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:48.670908 28276 raft_consensus.cc:385] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.671236 28276 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.671993 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:48.674103 28286 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.671955 28276 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.660604 28277 tablet_bootstrap.cc:492] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:48.675654 28277 ts_tablet_manager.cc:1403] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.066s	user 0.015s	sys 0.005s
I20260709 13:10:48.679852 28277 raft_consensus.cc:359] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.680722 28277 raft_consensus.cc:385] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.681053 28277 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.681856 28161 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:48.689695 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:48.691517 28356 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:48.682013 28277 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.682942 28301 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:48.693194 28235 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:48.694563 28359 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:48.696859 28356 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:48.697861 28356 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.700888 28277 ts_tablet_manager.cc:1434] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:10:48.702513 28277 tablet_bootstrap.cc:492] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:48.704964 28275 tablet_bootstrap.cc:492] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.705503 28275 ts_tablet_manager.cc:1403] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.000s
I20260709 13:10:48.688670 28276 ts_tablet_manager.cc:1434] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.002s
I20260709 13:10:48.687875 28286 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:48.707185 28276 tablet_bootstrap.cc:492] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
W20260709 13:10:48.726200 28125 consensus_peers.cc:597] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:48.728269 28314 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.729238 28314 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.730943 28314 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:48.731760 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.740312 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:10:48.750480 28199 leader_election.cc:343] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:48.751598 28199 leader_election.cc:343] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.752151 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:48.753170 28314 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:48.758059 27983 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.766283 27984 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.769503 28275 raft_consensus.cc:359] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.770314 28275 raft_consensus.cc:385] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.770612 28275 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.771404 28275 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.769948 28277 tablet_bootstrap.cc:654] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.776027 28275 ts_tablet_manager.cc:1434] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.070s	user 0.010s	sys 0.014s
I20260709 13:10:48.777302 28275 tablet_bootstrap.cc:492] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:48.783015 28276 tablet_bootstrap.cc:654] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.784065 28275 tablet_bootstrap.cc:654] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.785341 28301 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:48.799537 28277 tablet_bootstrap.cc:492] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:48.800016 28277 ts_tablet_manager.cc:1403] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.098s	user 0.015s	sys 0.000s
I20260709 13:10:48.812011 28275 tablet_bootstrap.cc:492] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.812546 28275 ts_tablet_manager.cc:1403] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.000s
I20260709 13:10:48.812937 28277 raft_consensus.cc:359] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.813670 28277 raft_consensus.cc:385] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.813953 28277 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.814816 28277 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.816906 28277 ts_tablet_manager.cc:1434] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:10:48.817469 28276 tablet_bootstrap.cc:492] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.817333 28275 raft_consensus.cc:359] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.818694 28276 ts_tablet_manager.cc:1403] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.112s	user 0.014s	sys 0.000s
I20260709 13:10:48.818765 28275 raft_consensus.cc:385] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.819149 28275 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.817771 28277 tablet_bootstrap.cc:492] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:48.821008 28275 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.821533 28276 raft_consensus.cc:359] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.822245 28276 raft_consensus.cc:385] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.822932 28276 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.823740 28276 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.826257 28276 ts_tablet_manager.cc:1434] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:48.830590 28277 tablet_bootstrap.cc:654] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.831920 28275 ts_tablet_manager.cc:1434] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:10:48.832908 28276 tablet_bootstrap.cc:492] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.832960 28275 tablet_bootstrap.cc:492] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:48.840464 28276 tablet_bootstrap.cc:654] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.855314 28276 tablet_bootstrap.cc:492] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.857124 28286 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.858035 28275 tablet_bootstrap.cc:654] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.857846 28286 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.855885 28276 ts_tablet_manager.cc:1403] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.026s	user 0.003s	sys 0.018s
I20260709 13:10:48.863080 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:10:48.865237 28050 leader_election.cc:343] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.860713 28286 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:48.866870 28276 raft_consensus.cc:359] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.867647 28276 raft_consensus.cc:385] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.868000 28276 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.868392 28277 tablet_bootstrap.cc:492] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:48.868871 28277 ts_tablet_manager.cc:1403] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.022s
I20260709 13:10:48.868866 28276 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.870419 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:10:48.872789 28052 leader_election.cc:343] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.873445 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:48.875365 28286 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:48.877799 28276 ts_tablet_manager.cc:1434] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:10:48.878654 28276 tablet_bootstrap.cc:492] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.880159 28277 raft_consensus.cc:359] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.881021 28277 raft_consensus.cc:385] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.881381 28277 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.882422 28275 tablet_bootstrap.cc:492] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.882130 28277 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.882879 28275 ts_tablet_manager.cc:1403] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.006s
I20260709 13:10:48.884688 28276 tablet_bootstrap.cc:654] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.887436 28277 ts_tablet_manager.cc:1434] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:10:48.888582 28277 tablet_bootstrap.cc:492] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:48.885795 28275 raft_consensus.cc:359] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.891942 28275 raft_consensus.cc:385] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.892288 28275 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.892657 28314 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.895478 28277 tablet_bootstrap.cc:654] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.895644 28314 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.898067 28314 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:48.899250 28276 tablet_bootstrap.cc:492] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.899281 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.899984 28276 ts_tablet_manager.cc:1403] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.007s
I20260709 13:10:48.899660 28286 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:10:48.902864 28199 leader_election.cc:343] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.899605 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:48.903996 28161 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:48.905362 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:48.893107 28275 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.909142 28275 ts_tablet_manager.cc:1434] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.004s
I20260709 13:10:48.910056 28314 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:48.910504 28314 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.910899 28314 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.912053 28276 raft_consensus.cc:359] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.910104 28275 tablet_bootstrap.cc:492] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:48.913590 28277 tablet_bootstrap.cc:492] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:48.914273 28277 ts_tablet_manager.cc:1403] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.016s
I20260709 13:10:48.912792 28276 raft_consensus.cc:385] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.915405 28276 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.917439 28277 raft_consensus.cc:359] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.917587 28314 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.919390 28275 tablet_bootstrap.cc:654] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.918232 28277 raft_consensus.cc:385] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.919742 28314 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:48.920246 28277 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.921203 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:48.920964 28277 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.921758 28161 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.922004 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:48.922312 28276 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:48.925254 28199 leader_election.cc:343] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:48.926043 28378 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:48.927192 28277 ts_tablet_manager.cc:1434] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:10:48.928109 28277 tablet_bootstrap.cc:492] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:48.929617 28276 ts_tablet_manager.cc:1434] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:10:48.932267 28276 tablet_bootstrap.cc:492] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.954773 28275 tablet_bootstrap.cc:492] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.955422 28275 ts_tablet_manager.cc:1403] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.020s
I20260709 13:10:48.956477 28086 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:48.957598 28276 tablet_bootstrap.cc:654] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.958226 28275 raft_consensus.cc:359] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.958956 28275 raft_consensus.cc:385] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.959259 28275 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.959931 28275 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.961974 28275 ts_tablet_manager.cc:1434] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:48.962908 28275 tablet_bootstrap.cc:492] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:48.965458 28276 tablet_bootstrap.cc:492] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:48.966024 28276 ts_tablet_manager.cc:1403] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.000s
I20260709 13:10:48.969072 28275 tablet_bootstrap.cc:654] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.968858 28276 raft_consensus.cc:359] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.969718 28276 raft_consensus.cc:385] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.970103 28276 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.970973 28276 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.971937 28359 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.972458 28359 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.973388 28306 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:48.974550 28359 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:48.976619 28276 ts_tablet_manager.cc:1434] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:10:48.984486 28276 tablet_bootstrap.cc:492] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:48.939697 28161 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:48.985345 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:48.990556 28159 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:48.985625 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:48.991405 28275 tablet_bootstrap.cc:492] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:48.991552 28085 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:48.991866 28275 ts_tablet_manager.cc:1403] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.006s
I20260709 13:10:48.986366 28306 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:48.992873 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:48.988336 28333 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:48.944042 28277 tablet_bootstrap.cc:654] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.994578 28275 raft_consensus.cc:359] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.995304 28275 raft_consensus.cc:385] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:48.995568 28275 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:48.996207 28275 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:48.997311 28276 tablet_bootstrap.cc:654] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:48.998421 28275 ts_tablet_manager.cc:1434] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:10:48.999416 28275 tablet_bootstrap.cc:492] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:49.007112 28275 tablet_bootstrap.cc:654] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.020237 28314 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:49.020658 28314 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:49.021063 28314 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.025321 28333 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.025856 28333 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.028847 28333 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.030282 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.031131 28234 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:49.032298 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:49.034355 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:49.036213 28125 leader_election.cc:343] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.037592 28317 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:49.038244 28317 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:49.039242 28317 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.042817 28306 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.043617 28127 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:49.043531 28306 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.045423 28333 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:49.045828 28333 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:49.046236 28333 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.048882 28306 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.050032 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.051710 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:49.052640 28161 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 1: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:10:49.053864 28050 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:49.055081 28306 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.069962 28356 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:49.078814 28359 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.079330 28359 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.082324 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.082547 28359 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:10:49.084360 28199 leader_election.cc:343] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.084609 28306 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:49.082324 28277 tablet_bootstrap.cc:492] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:49.085678 28277 ts_tablet_manager.cc:1403] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.158s	user 0.019s	sys 0.006s
I20260709 13:10:49.085983 28276 tablet_bootstrap.cc:492] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:49.086229 28391 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.086464 28276 ts_tablet_manager.cc:1403] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.102s	user 0.018s	sys 0.037s
I20260709 13:10:49.086714 28391 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.088637 28277 raft_consensus.cc:359] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.086233 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:49.089018 28276 raft_consensus.cc:359] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.089438 28277 raft_consensus.cc:385] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.089600 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.089921 28277 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
W20260709 13:10:49.091811 28199 leader_election.cc:343] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.092401 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:49.093444 28359 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:10:49.094446 28125 leader_election.cc:343] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.089829 28276 raft_consensus.cc:385] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.095338 28276 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.097748 28333 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.097851 28276 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.099776 28301 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:49.100198 28276 ts_tablet_manager.cc:1434] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:10:49.100536 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:49.101176 28276 tablet_bootstrap.cc:492] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:49.102442 27984 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:49.107054 28314 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.108239 28317 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:49.110061 28276 tablet_bootstrap.cc:654] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.110121 28333 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.111090 28314 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.111280 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:49.111856 28233 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.115882 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:49.116539 28161 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.118700 28391 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.118835 28233 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:49.119117 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.119820 28275 tablet_bootstrap.cc:492] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:49.120317 28275 ts_tablet_manager.cc:1403] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.121s	user 0.022s	sys 0.038s
I20260709 13:10:49.120901 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:49.121556 28085 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.123170 28127 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:49.123898 28275 raft_consensus.cc:359] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.127780 28275 raft_consensus.cc:385] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.128193 28275 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
W20260709 13:10:49.128473 28127 leader_election.cc:343] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.129091 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:49.129889 28333 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:49.129086 28275 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.130369 28333 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:49.130504 28301 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:49.131245 28333 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.133419 28275 ts_tablet_manager.cc:1434] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
W20260709 13:10:49.133615 28125 leader_election.cc:343] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.134380 28275 tablet_bootstrap.cc:492] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:49.134930 28378 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:49.139598 28276 tablet_bootstrap.cc:492] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:49.140174 28276 ts_tablet_manager.cc:1403] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.005s
I20260709 13:10:49.141911 28275 tablet_bootstrap.cc:654] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.142236 28356 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:49.148016 28277 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.142899 28276 raft_consensus.cc:359] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.151141 28277 ts_tablet_manager.cc:1434] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.000s
I20260709 13:10:49.151325 28276 raft_consensus.cc:385] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.151719 28276 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.152434 28276 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.154070 28276 ts_tablet_manager.cc:1434] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:10:49.154665 28161 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:49.156579 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:49.159564 28280 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:49.160112 28276 tablet_bootstrap.cc:492] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:49.160071 28280 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:49.160406 28317 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.161070 28317 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.161036 28280 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.165339 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.166281 28084 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:49.167793 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:49.168977 28359 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:49.170277 28317 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.171847 28277 tablet_bootstrap.cc:492] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:49.170526 28359 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:49.172909 28359 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.177157 28306 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.156577 28085 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:49.178553 28306 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.182811 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:49.186175 28359 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.188402 28276 tablet_bootstrap.cc:654] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.192234 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:10:49.193424 28199 leader_election.cc:343] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.194621 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:49.189756 28306 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.215351 28359 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.217949 28314 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.218631 28314 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.221832 28356 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:49.229744 28314 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.231611 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:10:49.234963 28199 leader_election.cc:343] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.240489 28084 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.240520 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:49.254232 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.252693 28050 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:10:49.266359 28199 leader_election.cc:343] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:49.267388 28199 leader_election.cc:343] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.267923 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:49.269047 28314 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:49.272485 28277 tablet_bootstrap.cc:654] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.260066 27984 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:49.277539 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:49.278151 28085 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.279680 28314 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:49.286758 28275 tablet_bootstrap.cc:492] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:49.287479 28275 ts_tablet_manager.cc:1403] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.153s	user 0.053s	sys 0.048s
I20260709 13:10:49.299261 28276 tablet_bootstrap.cc:492] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:49.299171 28333 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.299906 28276 ts_tablet_manager.cc:1403] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.140s	user 0.010s	sys 0.001s
I20260709 13:10:49.299965 28333 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.301265 27981 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:49.302831 28276 raft_consensus.cc:359] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.303532 28276 raft_consensus.cc:385] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.303740 28333 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.305119 28275 raft_consensus.cc:359] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.305907 28275 raft_consensus.cc:385] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.306233 28275 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.306030 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.306563 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.296972 28359 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:49.307731 28125 leader_election.cc:343] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.307986 28359 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.309953 28234 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:49.311358 28306 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:49.311900 28275 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.313474 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:49.314065 28275 ts_tablet_manager.cc:1434] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:10:49.314621 28333 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:49.315002 28275 tablet_bootstrap.cc:492] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:49.315045 28333 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:49.318392 28333 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.319607 28085 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:49.303824 28276 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.321549 28276 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.321045 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:49.323621 28314 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:49.324108 28314 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:49.325815 28275 tablet_bootstrap.cc:654] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.325136 28314 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.324118 28276 ts_tablet_manager.cc:1434] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:10:49.330611 28276 tablet_bootstrap.cc:492] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:49.342515 28277 tablet_bootstrap.cc:492] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:49.343010 28277 ts_tablet_manager.cc:1403] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.172s	user 0.015s	sys 0.015s
I20260709 13:10:49.345647 28277 raft_consensus.cc:359] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.346333 28277 raft_consensus.cc:385] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.346613 28277 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.347235 28277 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.348701 28275 tablet_bootstrap.cc:492] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:49.349182 28275 ts_tablet_manager.cc:1403] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.006s
I20260709 13:10:49.351465 28276 tablet_bootstrap.cc:654] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.351759 28275 raft_consensus.cc:359] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.354995 28275 raft_consensus.cc:385] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.355301 28275 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.355985 28275 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.357849 27981 catalog_manager.cc:5697] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:49.359280 28275 ts_tablet_manager.cc:1434] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.004s
I20260709 13:10:49.360556 28275 tablet_bootstrap.cc:492] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:49.350904 28333 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.366643 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:49.367280 28275 tablet_bootstrap.cc:654] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.367775 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:49.368229 28125 leader_election.cc:343] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.353372 28277 ts_tablet_manager.cc:1434] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:10:49.368903 28391 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.369711 28277 tablet_bootstrap.cc:492] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:49.369395 28391 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.372076 28391 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.375000 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.376459 28314 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:49.376641 28125 leader_election.cc:343] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.376940 28314 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.379050 28314 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.382180 28359 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.384001 28275 tablet_bootstrap.cc:492] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:49.384660 28275 ts_tablet_manager.cc:1403] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.003s
I20260709 13:10:49.384897 28333 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.384052 28276 tablet_bootstrap.cc:492] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:49.385672 28276 ts_tablet_manager.cc:1403] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.005s
W20260709 13:10:49.386847 28199 leader_election.cc:343] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.387684 28275 raft_consensus.cc:359] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.388005 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.388453 28275 raft_consensus.cc:385] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.388361 28276 raft_consensus.cc:359] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.388978 28275 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.389178 28276 raft_consensus.cc:385] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.389485 28276 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.389883 28275 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.390053 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:49.390112 28276 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.390841 28235 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.392247 28275 ts_tablet_manager.cc:1434] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:10:49.392711 28276 ts_tablet_manager.cc:1434] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:10:49.393599 28276 tablet_bootstrap.cc:492] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:49.395103 28275 tablet_bootstrap.cc:492] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:49.396888 28235 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:49.399609 28276 tablet_bootstrap.cc:654] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.399544 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:49.401757 28127 leader_election.cc:343] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.402248 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:49.402596 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:49.402956 28275 tablet_bootstrap.cc:654] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.403394 28160 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:10:49.403520 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:49.404096 28161 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:49.404248 28333 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:49.406332 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:10:49.406488 28199 leader_election.cc:343] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.409924 28276 tablet_bootstrap.cc:492] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:49.410373 28276 ts_tablet_manager.cc:1403] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.017s	user 0.002s	sys 0.012s
I20260709 13:10:49.411043 28277 tablet_bootstrap.cc:654] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.411155 28333 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:49.411697 28333 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:49.412537 28333 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.412959 28276 raft_consensus.cc:359] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.413647 28276 raft_consensus.cc:385] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.413921 28276 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.414845 28276 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.416895 28276 ts_tablet_manager.cc:1434] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:10:49.419467 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:49.420583 28359 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:49.421840 28276 tablet_bootstrap.cc:492] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:49.421458 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:49.424226 28359 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:49.424623 28359 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:49.424993 28359 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.429917 28275 tablet_bootstrap.cc:492] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:49.430526 28275 ts_tablet_manager.cc:1403] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.003s
I20260709 13:10:49.434077 28275 raft_consensus.cc:359] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.435006 28275 raft_consensus.cc:385] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.435431 28275 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.436370 28275 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.438076 28277 tablet_bootstrap.cc:492] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:49.438536 28277 ts_tablet_manager.cc:1403] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.069s	user 0.006s	sys 0.011s
I20260709 13:10:49.438627 28391 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.439324 28275 ts_tablet_manager.cc:1434] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:10:49.439121 28391 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.440209 28275 tablet_bootstrap.cc:492] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:49.434725 28359 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.444585 28391 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.444834 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:10:49.446265 28199 leader_election.cc:343] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.447542 28314 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:49.449404 28276 tablet_bootstrap.cc:654] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.449342 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.450394 28314 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.451293 28275 tablet_bootstrap.cc:654] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.450884 28277 raft_consensus.cc:359] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.451769 28277 raft_consensus.cc:385] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.450855 28314 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.452126 28277 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
W20260709 13:10:49.452203 28127 leader_election.cc:343] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.452917 28277 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.453522 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.442054 28333 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.454329 28314 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.457777 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:49.461674 28125 leader_election.cc:343] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.461861 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:49.462182 28125 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:49.462729 28160 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:49.464646 27982 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:49.480610 28277 ts_tablet_manager.cc:1434] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:10:49.482028 28277 tablet_bootstrap.cc:492] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:49.470345 28391 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:49.503062 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:49.477756 28333 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.526089 28314 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:49.518934 28359 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.547935 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:49.548879 28084 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.558880 28333 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.559206 28086 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.561841 28314 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:49.563900 28199 leader_election.cc:343] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.564559 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.571552 28160 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.571496 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.577870 28349 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.573398 28314 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.575558 28235 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:49.589377 28275 tablet_bootstrap.cc:492] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:49.589908 28275 ts_tablet_manager.cc:1403] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.150s	user 0.028s	sys 0.046s
W20260709 13:10:49.595139 28125 leader_election.cc:343] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.584690 28349 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.599123 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.599370 28349 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.599543 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:49.599886 28234 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:49.601231 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:49.601157 28275 raft_consensus.cc:359] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.601950 28275 raft_consensus.cc:385] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:10:49.601999 28050 leader_election.cc:343] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.602223 28349 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:49.602280 28275 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.602782 28349 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:49.603222 28349 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.603276 28275 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:49.609364 28032 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:49.611490 28333 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.618058 28333 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.614400 28160 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:49.615072 28275 ts_tablet_manager.cc:1434] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:10:49.615379 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:49.620108 28333 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.620631 28333 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:49.621011 28333 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:49.621326 28333 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.622149 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.622148 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:49.622917 28235 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:49.623136 28356 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.623591 28359 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:49.624037 28275 tablet_bootstrap.cc:492] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:49.623610 28356 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.624782 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.627148 28359 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:49.630316 28277 tablet_bootstrap.cc:654] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.632104 28356 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.632699 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.633167 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:49.633492 28235 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:49.634886 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:10:49.635553 28125 leader_election.cc:343] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.632764 28359 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.639302 28356 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:49.641322 28356 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:49.641793 28356 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:49.643707 28276 tablet_bootstrap.cc:492] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:49.644354 28276 ts_tablet_manager.cc:1403] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.223s	user 0.018s	sys 0.015s
I20260709 13:10:49.644659 28333 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.646823 28333 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.648533 28276 raft_consensus.cc:359] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.649307 28276 raft_consensus.cc:385] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.650036 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:10:49.651757 28125 leader_election.cc:343] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.654567 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:49.652529 28276 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.655745 28378 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:49.656101 28378 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:49.656610 28378 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.656034 28276 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.658995 28349 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.661005 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:49.661597 28235 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.662220 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:10:49.664007 28050 leader_election.cc:343] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.664779 28276 ts_tablet_manager.cc:1434] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:10:49.664922 28356 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.666056 28276 tablet_bootstrap.cc:492] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:49.668907 28391 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:49.670454 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:49.671214 28275 tablet_bootstrap.cc:654] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.682034 28301 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.671244 28234 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:49.682610 28301 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.685303 28301 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.675534 28356 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.690922 28235 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:49.686129 28349 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.694121 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:49.694691 28280 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:49.705866 28410 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:49.707357 27982 catalog_manager.cc:5697] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:49.708530 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:49.709427 28391 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:49.709780 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.709975 28391 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:49.711026 28391 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.699769 28233 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.699048 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:49.703292 28314 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.729928 28084 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:10:49.731627 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.732537 28085 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 1: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:10:49.702526 28234 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:10:49.736306 28314 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.739164 28301 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.739856 28301 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.741132 28232 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:49.741825 28275 tablet_bootstrap.cc:492] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:49.741971 28301 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.743878 28275 ts_tablet_manager.cc:1403] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.008s
I20260709 13:10:49.746335 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:49.743249 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:49.749972 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:49.747448 28230 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
W20260709 13:10:49.754184 28199 leader_election.cc:343] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.754868 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:49.756098 28301 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:49.756469 28301 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:49.756848 28301 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.756709 28333 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:49.749540 28378 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.759562 28276 tablet_bootstrap.cc:654] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.765374 28275 raft_consensus.cc:359] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.766177 28275 raft_consensus.cc:385] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.766477 28275 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.768481 28277 tablet_bootstrap.cc:492] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:49.769399 28277 ts_tablet_manager.cc:1403] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.288s	user 0.015s	sys 0.014s
I20260709 13:10:49.770362 28275 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.773955 28359 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:49.770439 28233 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:49.778085 28378 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.778697 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:49.785598 28275 ts_tablet_manager.cc:1434] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.002s
I20260709 13:10:49.787083 28275 tablet_bootstrap.cc:492] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:49.795385 28275 tablet_bootstrap.cc:654] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.798578 28432 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:49.799172 28432 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:49.801687 28430 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:49.802206 28430 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:49.802682 28432 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.803036 28430 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.809975 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:49.810698 28233 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.809325 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:49.887075 28277 raft_consensus.cc:359] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.887859 28277 raft_consensus.cc:385] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.873772 28359 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.888161 28277 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.890847 28277 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.893702 28277 ts_tablet_manager.cc:1434] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.124s	user 0.003s	sys 0.005s
I20260709 13:10:49.894627 28277 tablet_bootstrap.cc:492] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:49.900113 28277 tablet_bootstrap.cc:654] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:49.876471 28378 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:49.881685 28161 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.902019 28359 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.902923 28378 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:49.924933 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:49.925544 28083 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.914026 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:49.921690 28301 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.935978 28359 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.942751 28301 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:49.944995 28125 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:49.953220 28391 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:49.957360 28083 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:49.967610 28276 tablet_bootstrap.cc:492] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:49.968824 28276 ts_tablet_manager.cc:1403] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.303s	user 0.035s	sys 0.023s
I20260709 13:10:49.977537 28233 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:49.964810 28378 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.974570 28359 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.981927 28436 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:49.983119 28359 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.985039 28359 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:49.988718 28276 raft_consensus.cc:359] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.989462 28276 raft_consensus.cc:385] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.989763 28275 tablet_bootstrap.cc:492] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:49.989778 28276 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.990435 28275 ts_tablet_manager.cc:1403] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.204s	user 0.013s	sys 0.030s
W20260709 13:10:49.990470 28125 leader_election.cc:343] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:49.990660 28391 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:49.990808 28276 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.991645 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:49.992468 28391 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:49.993073 28276 ts_tablet_manager.cc:1434] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:10:49.993207 28391 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:49.993530 28275 raft_consensus.cc:359] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.994014 28276 tablet_bootstrap.cc:492] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:49.994227 28275 raft_consensus.cc:385] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:49.994621 28275 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:49.994297 28391 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.995297 28275 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:49.996872 28275 ts_tablet_manager.cc:1434] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:49.997931 28275 tablet_bootstrap.cc:492] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:50.005671 28317 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:50.009057 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:50.010460 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:50.011104 28233 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.012698 28277 tablet_bootstrap.cc:492] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:50.013190 28277 ts_tablet_manager.cc:1403] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.119s	user 0.009s	sys 0.010s
I20260709 13:10:50.013476 28378 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:50.017257 28233 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:50.018307 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:10:50.022244 28199 leader_election.cc:343] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.024140 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:50.034258 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:50.035022 28158 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:50.035858 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:50.036509 28160 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:50.021219 28161 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:50.024358 28439 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:50.039144 28276 tablet_bootstrap.cc:654] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.033036 28301 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.042482 28301 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:50.043022 28277 raft_consensus.cc:359] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.043843 28277 raft_consensus.cc:385] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.044163 28277 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.043642 28301 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.045401 28277 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.047817 28277 ts_tablet_manager.cc:1434] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
W20260709 13:10:50.041352 28125 leader_election.cc:343] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.050108 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:50.051533 28199 leader_election.cc:343] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:50.052358 28125 consensus_peers.cc:597] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.053659 28277 tablet_bootstrap.cc:492] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:50.061683 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:50.063421 28314 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.063840 28314 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.064035 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:50.064210 28314 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.065563 28436 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.066045 28436 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.070349 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:50.072486 28317 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.075155 28317 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:50.076069 28317 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.082427 27982 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:50.071718 28359 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.090996 28359 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.091423 28359 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.089946 28280 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.069486 28410 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:10:50.094596 28280 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.061059 27981 catalog_manager.cc:5697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:50.098974 28280 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:50.102268 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:50.103147 28233 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:50.104002 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:50.106164 28199 leader_election.cc:343] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.095882 28436 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:50.116132 28359 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.118203 28359 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:50.118422 28314 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.118872 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:50.123184 28314 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:50.101182 27981 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:50.127130 28199 leader_election.cc:343] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.137642 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:50.141322 28378 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.142122 28275 tablet_bootstrap.cc:654] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.142879 28276 tablet_bootstrap.cc:492] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:50.143393 28276 ts_tablet_manager.cc:1403] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.150s	user 0.007s	sys 0.006s
I20260709 13:10:50.127831 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:50.147209 28277 tablet_bootstrap.cc:654] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:50.147141 28199 leader_election.cc:343] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.152426 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:50.154374 28159 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:50.155584 28125 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:50.154404 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:50.156399 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:50.157110 28161 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.157512 28436 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.156488 28157 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.182662 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:50.185690 28451 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:50.188063 28314 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.197285 28161 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:50.198366 28436 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.199724 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:50.188930 28314 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.200721 28314 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.200812 28451 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.201529 28451 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:50.201637 27983 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:50.202677 28451 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.209127 28276 raft_consensus.cc:359] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.210279 28276 raft_consensus.cc:385] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.210718 28276 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.211900 28276 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.219686 28276 ts_tablet_manager.cc:1434] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.076s	user 0.010s	sys 0.001s
I20260709 13:10:50.220638 28276 tablet_bootstrap.cc:492] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:50.229252 28157 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:50.231772 28314 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.232820 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:50.234639 28359 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.235136 28359 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:50.235954 28359 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.247864 28275 tablet_bootstrap.cc:492] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:50.248381 28275 ts_tablet_manager.cc:1403] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.251s	user 0.033s	sys 0.013s
I20260709 13:10:50.249280 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:10:50.250689 28199 leader_election.cc:343] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.252125 28314 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:50.253216 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:50.253763 28160 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.265344 28086 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.267437 28276 tablet_bootstrap.cc:654] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.269369 28277 tablet_bootstrap.cc:492] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:50.270093 28277 ts_tablet_manager.cc:1403] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.217s	user 0.013s	sys 0.007s
I20260709 13:10:50.252472 27981 catalog_manager.cc:5697] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:50.278832 28160 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:50.279745 27981 catalog_manager.cc:5697] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:50.287825 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:50.291497 28411 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.292236 28411 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:50.293396 28391 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:50.294267 28275 raft_consensus.cc:359] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.295009 28275 raft_consensus.cc:385] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.295316 28275 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.296132 28275 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.299721 28158 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:10:50.300269 28158 pending_rounds.cc:85] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 0
I20260709 13:10:50.300668 28158 pending_rounds.cc:107] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:10:50.301067 28158 raft_consensus.cc:2889] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:50.293344 28411 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.303701 28275 ts_tablet_manager.cc:1434] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.000s
I20260709 13:10:50.300964 28277 raft_consensus.cc:359] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:50.304690 28199 consensus_peers.cc:597] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.304982 28277 raft_consensus.cc:385] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.305223 28275 tablet_bootstrap.cc:492] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:50.305311 28277 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.307327 28432 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:50.312561 28277 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.315654 28277 ts_tablet_manager.cc:1434] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:10:50.317104 28277 tablet_bootstrap.cc:492] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:50.327862 27983 catalog_manager.cc:5697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:50.332995 27983 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:50.357043 28432 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:10:50.396044 28233 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:50.397682 28432 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.405967 28276 tablet_bootstrap.cc:492] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:50.406149 28275 tablet_bootstrap.cc:654] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.406504 28276 ts_tablet_manager.cc:1403] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.186s	user 0.012s	sys 0.008s
I20260709 13:10:50.419708 28276 raft_consensus.cc:359] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.420210 28276 raft_consensus.cc:385] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.420380 28276 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.422817 28349 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.424623 28349 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.426867 28277 tablet_bootstrap.cc:654] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.428342 28349 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:50.429996 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:50.434227 27983 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:50.424412 28276 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.438877 28276 ts_tablet_manager.cc:1434] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.003s
I20260709 13:10:50.439865 28276 tablet_bootstrap.cc:492] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:50.445205 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:50.446086 28158 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 1: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:10:50.447690 28050 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:50.448323 28356 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.447983 28277 tablet_bootstrap.cc:492] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:50.450744 28430 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.451057 28277 ts_tablet_manager.cc:1403] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.134s	user 0.021s	sys 0.000s
I20260709 13:10:50.454427 28277 raft_consensus.cc:359] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.455371 28277 raft_consensus.cc:385] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.455711 28277 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.456549 28277 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.459071 28430 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.459663 28277 ts_tablet_manager.cc:1434] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:50.461184 28277 tablet_bootstrap.cc:492] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:50.463793 28275 tablet_bootstrap.cc:492] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:50.464326 28275 ts_tablet_manager.cc:1403] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.159s	user 0.013s	sys 0.007s
I20260709 13:10:50.467036 28275 raft_consensus.cc:359] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.467754 28275 raft_consensus.cc:385] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.468101 28277 tablet_bootstrap.cc:654] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.445528 28276 tablet_bootstrap.cc:654] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.474176 28391 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.468051 28275 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.492107 28275 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.495004 28275 ts_tablet_manager.cc:1434] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:10:50.495824 28275 tablet_bootstrap.cc:492] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:50.500768 28275 tablet_bootstrap.cc:654] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.542840 28314 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.545907 28356 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.543644 28314 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.548092 28314 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:50.548663 28276 tablet_bootstrap.cc:492] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:50.549369 28276 ts_tablet_manager.cc:1403] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.110s	user 0.013s	sys 0.000s
I20260709 13:10:50.549566 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:50.551115 28199 leader_election.cc:343] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.546474 28356 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.552634 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:50.553241 28277 tablet_bootstrap.cc:492] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:50.553596 28356 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:50.553707 28277 ts_tablet_manager.cc:1403] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.093s	user 0.015s	sys 0.018s
I20260709 13:10:50.554664 28276 raft_consensus.cc:359] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.561383 28276 raft_consensus.cc:385] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.561707 28276 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.562471 28276 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.555574 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:50.564670 28276 ts_tablet_manager.cc:1434] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:10:50.556877 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:50.566890 28276 tablet_bootstrap.cc:492] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
W20260709 13:10:50.558166 28199 leader_election.cc:343] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.568969 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:50.571220 28451 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.568457 28050 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:50.579177 28468 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.580250 28276 tablet_bootstrap.cc:654] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.582547 28439 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.581461 28301 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:50.583251 28439 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.585542 28439 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:50.588243 28086 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.594760 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:50.595355 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:50.581425 28277 raft_consensus.cc:359] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.595559 28232 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:50.597992 28127 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:50.599249 28301 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:50.599637 28301 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.599980 28301 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.596458 28277 raft_consensus.cc:385] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.601613 28277 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.602684 28277 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.603658 28439 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.604210 28439 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.606067 28277 ts_tablet_manager.cc:1434] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:10:50.607386 28277 tablet_bootstrap.cc:492] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:50.607427 28439 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:10:50.610456 28125 leader_election.cc:343] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.613108 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:10:50.615338 28127 leader_election.cc:343] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.618696 28277 tablet_bootstrap.cc:654] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.621060 28301 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.624920 28439 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.627171 28275 tablet_bootstrap.cc:492] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:50.627426 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:50.627822 28275 ts_tablet_manager.cc:1403] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.132s	user 0.047s	sys 0.052s
I20260709 13:10:50.631438 28439 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.631450 28275 raft_consensus.cc:359] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.635035 28275 raft_consensus.cc:385] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.635322 28275 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.635923 28275 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.638005 28275 ts_tablet_manager.cc:1434] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
W20260709 13:10:50.644150 28125 leader_election.cc:343] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.644770 28275 tablet_bootstrap.cc:492] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:50.644632 28125 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:50.650871 28439 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.663239 28301 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:50.664501 28276 tablet_bootstrap.cc:492] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:50.665027 28276 ts_tablet_manager.cc:1403] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.098s	user 0.013s	sys 0.005s
I20260709 13:10:50.673274 28277 tablet_bootstrap.cc:492] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:50.673877 28277 ts_tablet_manager.cc:1403] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.067s	user 0.018s	sys 0.017s
I20260709 13:10:50.674381 28314 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.674844 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:50.675112 28083 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.677361 28277 raft_consensus.cc:359] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.678046 28277 raft_consensus.cc:385] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.678399 28277 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.679306 28277 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.681846 28277 ts_tablet_manager.cc:1434] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:10:50.682811 28277 tablet_bootstrap.cc:492] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:50.697024 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:50.697649 28233 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:50.715474 28032 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:50.739400 28276 raft_consensus.cc:359] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.740284 28276 raft_consensus.cc:385] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.740605 28276 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.743620 28276 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.745445 28276 ts_tablet_manager.cc:1434] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.080s	user 0.013s	sys 0.005s
I20260709 13:10:50.746574 28276 tablet_bootstrap.cc:492] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
W20260709 13:10:50.753280 28125 leader_election.cc:343] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.755285 28378 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.755417 28233 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:50.755868 28378 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.760778 28275 tablet_bootstrap.cc:654] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.757253 28127 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:50.768026 28301 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:50.769680 28276 tablet_bootstrap.cc:654] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.774739 28378 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:50.776190 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:50.768607 28301 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:50.777303 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:50.781085 28127 leader_election.cc:343] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.781679 28301 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:50.782492 28125 leader_election.cc:343] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.783020 28125 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:50.784406 28451 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.784428 28378 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.785001 28451 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.788933 28356 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:10:50.789331 28451 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:50.789558 28356 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.793172 28277 tablet_bootstrap.cc:654] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.795859 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:10:50.797727 28199 leader_election.cc:343] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.801153 28356 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:50.802281 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:50.805570 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:50.807461 28199 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:50.808200 28199 consensus_peers.cc:597] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:10:50.809324 28199 leader_election.cc:343] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:50.809754 28199 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:50.812244 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:50.815368 28314 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.816802 28314 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.818018 28314 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:50.822417 28050 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:50.823400 28356 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:50.823223 28411 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.836256 28275 tablet_bootstrap.cc:492] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:50.837119 28275 ts_tablet_manager.cc:1403] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.193s	user 0.009s	sys 0.024s
I20260709 13:10:50.848505 28276 tablet_bootstrap.cc:492] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:50.848454 28275 raft_consensus.cc:359] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.849076 28276 ts_tablet_manager.cc:1403] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.103s	user 0.009s	sys 0.004s
I20260709 13:10:50.849252 28275 raft_consensus.cc:385] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.849539 28275 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.850389 28275 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.851979 28276 raft_consensus.cc:359] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.852655 28276 raft_consensus.cc:385] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.852985 28276 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.852398 28275 ts_tablet_manager.cc:1434] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.000s
I20260709 13:10:50.853660 28276 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.854590 28275 tablet_bootstrap.cc:492] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:50.858040 28277 tablet_bootstrap.cc:492] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:50.858819 28277 ts_tablet_manager.cc:1403] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.176s	user 0.008s	sys 0.007s
I20260709 13:10:50.861351 28277 raft_consensus.cc:359] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.862006 28277 raft_consensus.cc:385] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.862269 28277 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.862892 28277 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.855883 28276 ts_tablet_manager.cc:1434] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:50.865330 28275 tablet_bootstrap.cc:654] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.866132 28277 ts_tablet_manager.cc:1434] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:50.867105 28277 tablet_bootstrap.cc:492] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:50.873307 28276 tablet_bootstrap.cc:492] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:50.873979 28451 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:50.883319 28277 tablet_bootstrap.cc:654] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.886487 28083 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:50.895131 28276 tablet_bootstrap.cc:654] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.894454 27982 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:50.903412 28275 tablet_bootstrap.cc:492] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:50.904153 28275 ts_tablet_manager.cc:1403] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.050s	user 0.007s	sys 0.013s
W20260709 13:10:50.887694 28199 consensus_peers.cc:597] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:50.907644 28275 raft_consensus.cc:359] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.908681 28275 raft_consensus.cc:385] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.909085 28275 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.918867 28275 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.925304 28276 tablet_bootstrap.cc:492] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:50.925387 28275 ts_tablet_manager.cc:1434] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.021s	user 0.002s	sys 0.009s
I20260709 13:10:50.926400 28276 ts_tablet_manager.cc:1403] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.020s
I20260709 13:10:50.926528 28275 tablet_bootstrap.cc:492] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:50.929059 28276 raft_consensus.cc:359] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.929888 28276 raft_consensus.cc:385] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.930223 28276 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.930940 28276 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.933925 28276 ts_tablet_manager.cc:1434] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:10:50.934803 28275 tablet_bootstrap.cc:654] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.942623 28275 tablet_bootstrap.cc:492] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:50.943096 28275 ts_tablet_manager.cc:1403] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 13:10:50.944406 28277 tablet_bootstrap.cc:492] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:50.945660 28275 raft_consensus.cc:359] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.946308 28275 raft_consensus.cc:385] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.944967 28277 ts_tablet_manager.cc:1403] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.078s	user 0.020s	sys 0.016s
I20260709 13:10:50.946579 28275 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.947336 28275 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.949234 28275 ts_tablet_manager.cc:1434] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:10:50.950119 28275 tablet_bootstrap.cc:492] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:50.949858 28277 raft_consensus.cc:359] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.950678 28277 raft_consensus.cc:385] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.950960 28277 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.951411 28276 tablet_bootstrap.cc:492] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:50.951634 28277 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.953837 28277 ts_tablet_manager.cc:1434] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:50.956959 28275 tablet_bootstrap.cc:654] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.959501 28277 tablet_bootstrap.cc:492] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:50.959760 28276 tablet_bootstrap.cc:654] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.966998 28277 tablet_bootstrap.cc:654] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.968746 28275 tablet_bootstrap.cc:492] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:50.969285 28275 ts_tablet_manager.cc:1403] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.009s
I20260709 13:10:50.972342 28275 raft_consensus.cc:359] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.972694 28430 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:10:50.973075 28275 raft_consensus.cc:385] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.973371 28275 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.973224 28430 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.974051 28275 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.977142 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:50.978266 28275 ts_tablet_manager.cc:1434] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:10:50.978662 28430 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:50.979244 28277 tablet_bootstrap.cc:492] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:50.979275 28275 tablet_bootstrap.cc:492] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:50.979939 28277 ts_tablet_manager.cc:1403] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.009s
I20260709 13:10:50.981519 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:50.983547 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:50.984187 28277 raft_consensus.cc:359] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.985302 28277 raft_consensus.cc:385] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:50.985644 28468 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:50.985842 28275 tablet_bootstrap.cc:654] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:50.985638 28277 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:50.989130 28277 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:50.991878 28277 ts_tablet_manager.cc:1434] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:10:50.993479 28277 tablet_bootstrap.cc:492] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:50.997097 28275 tablet_bootstrap.cc:492] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:51.002627 28277 tablet_bootstrap.cc:654] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.009738 28468 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.010258 28468 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.013787 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:51.014604 28232 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 1: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:10:51.015641 28277 tablet_bootstrap.cc:492] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.016139 28277 ts_tablet_manager.cc:1403] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.009s
I20260709 13:10:51.016062 28468 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.020051 28277 raft_consensus.cc:359] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.021062 28277 raft_consensus.cc:385] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.021453 28277 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.022750 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:51.022559 28277 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.024945 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:51.025763 28277 ts_tablet_manager.cc:1434] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 13:10:51.026309 28468 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.027275 28277 tablet_bootstrap.cc:492] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.030050 28275 ts_tablet_manager.cc:1403] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.004s
I20260709 13:10:51.032855 28275 raft_consensus.cc:359] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.033516 28275 raft_consensus.cc:385] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.033805 28275 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.035187 28430 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.035681 28430 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.038652 28277 tablet_bootstrap.cc:654] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.038403 28468 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.043159 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:51.045460 28430 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.051581 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:51.052876 28277 tablet_bootstrap.cc:492] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.053639 28277 ts_tablet_manager.cc:1403] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.003s
I20260709 13:10:51.054109 28050 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:51.055630 28430 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.057644 28277 raft_consensus.cc:359] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.058650 28277 raft_consensus.cc:385] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.059037 28277 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.059984 28277 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.063138 28277 ts_tablet_manager.cc:1434] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:10:51.064496 28277 tablet_bootstrap.cc:492] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.034559 28275 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.070317 28301 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.070811 28301 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.073011 28301 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.078574 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:51.078758 28277 tablet_bootstrap.cc:654] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:51.080302 28125 leader_election.cc:343] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.091686 28275 ts_tablet_manager.cc:1434] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.000s
I20260709 13:10:51.092442 28277 tablet_bootstrap.cc:492] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.092777 28275 tablet_bootstrap.cc:492] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:51.093174 28277 ts_tablet_manager.cc:1403] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.011s
I20260709 13:10:51.097060 28277 raft_consensus.cc:359] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.098028 28277 raft_consensus.cc:385] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.098397 28277 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.098554 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:51.099498 28277 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:51.100996 28127 leader_election.cc:343] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.101475 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:51.102595 28301 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.102572 28277 ts_tablet_manager.cc:1434] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:10:51.104656 28277 tablet_bootstrap.cc:492] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.108984 28275 tablet_bootstrap.cc:654] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.109180 28276 tablet_bootstrap.cc:492] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:51.109635 28276 ts_tablet_manager.cc:1403] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.158s	user 0.030s	sys 0.084s
I20260709 13:10:51.112306 28276 raft_consensus.cc:359] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.112995 28276 raft_consensus.cc:385] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.113153 28277 tablet_bootstrap.cc:654] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.113276 28276 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.114106 28276 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.116781 28276 ts_tablet_manager.cc:1434] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:51.118016 28276 tablet_bootstrap.cc:492] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:51.124606 28277 tablet_bootstrap.cc:492] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.124922 28276 tablet_bootstrap.cc:654] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.125125 28277 ts_tablet_manager.cc:1403] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.006s
I20260709 13:10:51.128464 28277 raft_consensus.cc:359] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.129179 28277 raft_consensus.cc:385] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.129451 28277 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.130038 28277 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.131973 28277 ts_tablet_manager.cc:1434] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:51.144230 28277 tablet_bootstrap.cc:492] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.148926 28489 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.149469 28489 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.151746 28489 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:51.152716 28277 tablet_bootstrap.cc:654] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.156458 28275 tablet_bootstrap.cc:492] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:51.156950 28275 ts_tablet_manager.cc:1403] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.064s	user 0.025s	sys 0.019s
I20260709 13:10:51.157033 28276 tablet_bootstrap.cc:492] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:51.157522 28276 ts_tablet_manager.cc:1403] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.015s
I20260709 13:10:51.158735 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:51.159392 28157 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:51.159796 28275 raft_consensus.cc:359] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.155081 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:51.160503 28275 raft_consensus.cc:385] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.160849 28275 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.160609 28276 raft_consensus.cc:359] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.161579 28276 raft_consensus.cc:385] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.161964 28276 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
W20260709 13:10:51.162976 28199 leader_election.cc:343] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.163280 28276 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.166237 28276 ts_tablet_manager.cc:1434] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:10:51.166329 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:51.167474 28276 tablet_bootstrap.cc:492] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:51.170305 28489 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.170719 28489 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.171048 28489 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.170957 28275 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.173316 28275 ts_tablet_manager.cc:1434] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 13:10:51.173877 28439 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:51.174486 28276 tablet_bootstrap.cc:654] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.175915 28275 tablet_bootstrap.cc:492] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:51.178436 28277 tablet_bootstrap.cc:492] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.179019 28277 ts_tablet_manager.cc:1403] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.011s
I20260709 13:10:51.181751 28277 raft_consensus.cc:359] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.182412 28277 raft_consensus.cc:385] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.182675 28277 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.183305 28277 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.183776 28275 tablet_bootstrap.cc:654] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.185441 28277 ts_tablet_manager.cc:1434] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:10:51.186322 28277 tablet_bootstrap.cc:492] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.193122 28277 tablet_bootstrap.cc:654] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.207612 28277 tablet_bootstrap.cc:492] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.208437 28277 ts_tablet_manager.cc:1403] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.001s
I20260709 13:10:51.196213 28489 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.212697 28277 raft_consensus.cc:359] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.214344 28277 raft_consensus.cc:385] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.215217 28277 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.216171 28277 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.219285 28277 ts_tablet_manager.cc:1434] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 13:10:51.220686 28277 tablet_bootstrap.cc:492] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.222472 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:10:51.225623 28199 leader_election.cc:343] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.229935 28277 tablet_bootstrap.cc:654] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.229714 28086 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.240752 28359 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.241297 28359 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.243249 28359 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:51.249660 28277 tablet_bootstrap.cc:492] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.251739 28277 ts_tablet_manager.cc:1403] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.011s
I20260709 13:10:51.253022 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:51.255822 28359 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.256507 28277 raft_consensus.cc:359] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.256563 28359 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.257508 28277 raft_consensus.cc:385] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.257465 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:51.258277 28277 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.258751 28157 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:51.259183 28277 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.262444 28277 ts_tablet_manager.cc:1434] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:10:51.263652 28277 tablet_bootstrap.cc:492] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.265038 28378 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.265705 28378 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.269140 28378 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.272147 28277 tablet_bootstrap.cc:654] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.273488 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:51.274178 28082 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:51.279989 28125 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:51.283980 28277 tablet_bootstrap.cc:492] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.284809 28277 ts_tablet_manager.cc:1403] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.002s
I20260709 13:10:51.284943 28378 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.285564 28378 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.288141 28082 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.289831 28277 raft_consensus.cc:359] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.293447 28277 raft_consensus.cc:385] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.293990 28277 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.294683 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:51.295674 28085 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:51.295212 28277 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.299054 28277 ts_tablet_manager.cc:1434] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:10:51.299440 28378 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.300665 28277 tablet_bootstrap.cc:492] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.308593 28277 tablet_bootstrap.cc:654] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.312315 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:51.313336 28157 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:51.258965 28359 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:51.322000 28277 tablet_bootstrap.cc:492] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.322889 28277 ts_tablet_manager.cc:1403] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.010s
I20260709 13:10:51.329603 28489 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:10:51.330909 28199 leader_election.cc:343] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.336305 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:51.331557 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:51.338848 28489 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.338943 28232 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:51.339368 28489 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.339980 28276 tablet_bootstrap.cc:492] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:51.340533 28276 ts_tablet_manager.cc:1403] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.173s	user 0.047s	sys 0.073s
I20260709 13:10:51.343142 28276 raft_consensus.cc:359] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.343778 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:51.345860 28314 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.339962 28489 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.346203 28314 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.346551 28314 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.352550 28314 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.353766 28277 raft_consensus.cc:359] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.354387 28314 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:51.354477 28277 raft_consensus.cc:385] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.355075 28277 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.355779 28277 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.357335 28277 ts_tablet_manager.cc:1434] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:10:51.358207 28277 tablet_bootstrap.cc:492] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.360522 28275 tablet_bootstrap.cc:492] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:51.361035 28275 ts_tablet_manager.cc:1403] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.185s	user 0.065s	sys 0.073s
I20260709 13:10:51.363916 28277 tablet_bootstrap.cc:654] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.363654 28275 raft_consensus.cc:359] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.364424 28275 raft_consensus.cc:385] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.363224 28378 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.343807 28276 raft_consensus.cc:385] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.370697 28276 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.371877 28378 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.371580 28276 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.364714 28275 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.374431 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:51.374640 28314 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.374897 28275 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.372831 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:51.375255 28314 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.375996 28158 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.377326 28275 ts_tablet_manager.cc:1434] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 13:10:51.377396 28314 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:51.374172 28276 ts_tablet_manager.cc:1434] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:10:51.384404 28158 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:51.384872 28276 tablet_bootstrap.cc:492] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:51.374892 28489 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.375068 28160 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.387892 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:51.388643 28232 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.392000 28160 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:51.367945 28083 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.378567 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:51.393044 28082 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.397290 28430 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.397886 28430 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.369122 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:51.392194 28276 tablet_bootstrap.cc:654] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.400095 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:51.400398 28085 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.378350 28275 tablet_bootstrap.cc:492] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:51.400864 28081 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:51.402941 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:51.403748 28233 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:51.405427 28052 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:51.406275 28468 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.406631 28468 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.406960 28468 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.408345 28232 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:51.410732 28439 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:51.412065 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:51.415494 28439 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.415918 28359 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.416313 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:51.414426 28468 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.416668 28359 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:51.417876 28359 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.419883 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:51.400125 28430 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.420478 28233 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.422829 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:51.420500 28514 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.424257 28514 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.426476 28514 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:10:51.427600 28050 leader_election.cc:343] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.428342 28233 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
W20260709 13:10:51.431033 28050 leader_election.cc:343] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.431540 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:51.433182 28081 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:51.433661 28052 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:51.434536 28430 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.434998 28430 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:51.435864 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:51.435806 28430 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.436575 28161 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:51.432050 28468 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.416020 28439 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.443915 28439 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.448253 28277 tablet_bootstrap.cc:492] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.449019 28277 ts_tablet_manager.cc:1403] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.091s	user 0.016s	sys 0.003s
I20260709 13:10:51.449991 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:51.450784 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:51.452052 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:51.452765 28314 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.453279 28314 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.453733 28314 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.454452 28378 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.454938 28378 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.455297 28378 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.455632 28468 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.456110 28468 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.458930 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:51.459649 28157 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:51.460474 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:51.462277 28514 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.462294 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:51.462702 28085 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:51.463289 28356 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.463792 28356 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.463222 28514 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:51.464175 28356 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.464715 28514 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.463768 28082 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:51.469062 28275 tablet_bootstrap.cc:654] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.470803 28523 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.471284 28523 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.471679 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:51.472359 28233 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:51.473773 28277 raft_consensus.cc:359] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.472355 28356 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.500806 28356 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.502045 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:51.502692 28234 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.474906 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:51.504083 28277 raft_consensus.cc:385] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.504446 28277 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.507114 28277 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.467908 28468 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.476517 28489 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:51.477557 28276 tablet_bootstrap.cc:492] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:51.517313 28276 ts_tablet_manager.cc:1403] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.133s	user 0.014s	sys 0.028s
I20260709 13:10:51.519711 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:51.486764 28301 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.522152 28301 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.523365 28378 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.525874 28301 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.487052 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:51.490803 27981 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:10:51.507498 28125 leader_election.cc:343] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.475677 28523 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:10:51.510376 28052 leader_election.cc:343] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.511966 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:51.527320 28157 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.513823 28234 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:51.515165 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:51.515724 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:51.520263 28276 raft_consensus.cc:359] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.528875 28276 raft_consensus.cc:385] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.529186 28276 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.529258 28052 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:51.530068 28523 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.529842 28276 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.530531 28523 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:51.531306 28523 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.531994 28276 ts_tablet_manager.cc:1434] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 13:10:51.534317 28157 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:51.537158 28232 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:51.538479 28276 tablet_bootstrap.cc:492] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:51.540169 28275 tablet_bootstrap.cc:492] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:51.540917 28230 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:51.546149 28160 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:51.478819 28314 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.547669 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:51.548592 28314 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:51.549412 28275 ts_tablet_manager.cc:1403] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.171s	user 0.017s	sys 0.011s
I20260709 13:10:51.550387 28276 tablet_bootstrap.cc:654] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.550915 28430 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.551386 28430 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.551775 28430 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.554816 28275 raft_consensus.cc:359] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.555423 28275 raft_consensus.cc:385] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.555721 28275 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.556363 28275 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.559511 28277 ts_tablet_manager.cc:1434] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.110s	user 0.007s	sys 0.000s
I20260709 13:10:51.560398 28489 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.561818 28489 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:51.562745 28489 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.554636 28378 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.565303 28277 tablet_bootstrap.cc:492] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.560766 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:51.572904 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:51.573675 28275 ts_tablet_manager.cc:1434] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 13:10:51.575268 28275 tablet_bootstrap.cc:492] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:51.577019 28082 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.584673 28277 tablet_bootstrap.cc:654] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.643893 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:51.646164 28086 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.576393 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:51.670667 28160 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.677234 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:51.678853 28157 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.681681 28276 tablet_bootstrap.cc:492] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:51.685143 28082 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:51.689131 28127 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:51.707820 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:51.712839 28083 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.713387 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:51.714629 28439 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.718047 28160 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:51.719565 28276 ts_tablet_manager.cc:1403] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.181s	user 0.009s	sys 0.010s
I20260709 13:10:51.721930 28277 tablet_bootstrap.cc:492] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:51.721789 28439 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:51.722133 28157 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
W20260709 13:10:51.723841 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 55395.9us
I20260709 13:10:51.724696 28439 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.727555 28301 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.728049 28301 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.728440 28301 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.732512 28275 tablet_bootstrap.cc:654] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.739509 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:51.740515 28230 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.753127 28086 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:51.751380 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:51.760581 28277 ts_tablet_manager.cc:1403] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.195s	user 0.010s	sys 0.010s
I20260709 13:10:51.761788 28430 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.766758 28083 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:51.770016 28301 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.772356 28085 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:51.774430 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:51.781003 28276 raft_consensus.cc:359] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.781872 28430 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.781872 28276 raft_consensus.cc:385] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.783623 28276 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.784415 28276 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.787418 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:10:51.791826 28052 leader_election.cc:343] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.793421 28230 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:51.796597 28275 tablet_bootstrap.cc:492] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:51.797806 28359 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.798591 28275 ts_tablet_manager.cc:1403] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.224s	user 0.016s	sys 0.008s
I20260709 13:10:51.798584 28359 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:51.799438 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:51.794212 28301 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.800122 28359 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.801275 28489 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:51.788717 28276 ts_tablet_manager.cc:1434] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.001s
I20260709 13:10:51.804802 27984 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.794466 28277 raft_consensus.cc:359] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.806803 28489 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:51.809542 28489 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.808575 28277 raft_consensus.cc:385] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.811823 28277 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:51.820704 28277 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.830097 28276 tablet_bootstrap.cc:492] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:51.814646 28275 raft_consensus.cc:359] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.831476 28275 raft_consensus.cc:385] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:51.831874 28275 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
W20260709 13:10:51.821818 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46314.8us
I20260709 13:10:51.846585 28275 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:51.853847 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44277.8us
I20260709 13:10:51.856626 28277 ts_tablet_manager.cc:1434] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.095s	user 0.009s	sys 0.000s
I20260709 13:10:51.857339 28275 ts_tablet_manager.cc:1434] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.001s
W20260709 13:10:51.864324 28127 leader_election.cc:343] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:51.880896 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:51.881889 28157 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.892602 28275 tablet_bootstrap.cc:492] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:51.894876 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:51.900100 28489 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.906179 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:51.900794 28489 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.909443 28277 tablet_bootstrap.cc:492] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:51.912632 28489 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:51.929315 28276 tablet_bootstrap.cc:654] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.932296 28430 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:51.932899 28430 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:51.935068 28430 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:51.937983 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:51.938797 28161 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
W20260709 13:10:51.939312 28032 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:51.941818 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:51.942956 28525 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:51.943531 28527 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:51.943943 28527 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.944303 28527 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:51.953212 28275 tablet_bootstrap.cc:654] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:51.955601 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 60803.9us
I20260709 13:10:51.960449 28277 tablet_bootstrap.cc:654] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:51.964170 28157 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:51.973207 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:51.973351 28514 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:51.966594 27984 catalog_manager.cc:5697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:51.973914 28233 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.003509 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:52.004714 28086 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.006593 27984 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:52.058012 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54597.6us
I20260709 13:10:52.009274 28356 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.008862 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:52.061455 28356 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:52.064999 28356 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.069871 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:52.070693 28156 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:52.059949 28233 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:52.081492 28154 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:52.097767 28125 leader_election.cc:343] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.101704 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:52.102847 28235 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:52.088757 28527 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.079010 28489 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:10:52.106585 28527 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:52.110167 28314 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:52.110206 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:52.112447 28451 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.112874 28451 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.113296 28275 tablet_bootstrap.cc:492] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:52.113304 28451 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.113924 28275 ts_tablet_manager.cc:1403] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.222s	user 0.020s	sys 0.005s
I20260709 13:10:52.117901 28275 raft_consensus.cc:359] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.123414 28275 raft_consensus.cc:385] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.123873 28275 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.122076 28127 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:52.127594 28514 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.124104 28082 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:52.129827 28514 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:52.130167 28356 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.130862 28514 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.135751 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:52.136598 28356 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.137284 27981 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.151648 28275 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.160634 28314 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.161937 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:52.162562 28155 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.183790 28276 tablet_bootstrap.cc:492] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:52.184521 28276 ts_tablet_manager.cc:1403] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.355s	user 0.019s	sys 0.006s
I20260709 13:10:52.187151 28277 tablet_bootstrap.cc:492] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:52.187675 28277 ts_tablet_manager.cc:1403] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.290s	user 0.016s	sys 0.011s
I20260709 13:10:52.194453 28276 raft_consensus.cc:359] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.195492 28276 raft_consensus.cc:385] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.195822 28276 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.200403 28541 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.206305 27984 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.209291 28276 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.201046 28541 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.216216 28275 ts_tablet_manager.cc:1434] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.102s	user 0.009s	sys 0.001s
I20260709 13:10:52.228950 28277 raft_consensus.cc:359] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.230005 28277 raft_consensus.cc:385] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.230355 28277 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.231365 28468 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.231746 28277 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.233209 28277 ts_tablet_manager.cc:1434] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:10:52.241914 28276 ts_tablet_manager.cc:1434] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.001s
I20260709 13:10:52.242726 28275 tablet_bootstrap.cc:492] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:52.243265 28301 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:10:52.241154 28514 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:10:52.268478 28276 tablet_bootstrap.cc:492] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:52.256850 28451 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.273891 28235 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 1: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:10:52.270002 28541 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:52.281728 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:52.282651 28080 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:52.289563 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:52.298840 28514 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.276661 28451 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:52.305230 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:52.288645 28155 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:52.312772 28277 tablet_bootstrap.cc:492] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:52.300766 28541 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:10:52.321097 28489 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.325461 28127 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:52.329161 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:52.336392 28527 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.331418 28301 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:52.338124 28527 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:52.347090 28527 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.338855 28550 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:10:52.379137 27983 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:52.388212 28276 tablet_bootstrap.cc:654] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.385337 27983 catalog_manager.cc:5697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:52.390604 27983 catalog_manager.cc:5697] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:52.391770 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:52.392619 28158 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:52.415587 28359 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.426117 28489 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.427349 28550 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.445276 28275 tablet_bootstrap.cc:654] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.464138 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:52.470383 28546 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.467485 28439 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.474395 28439 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.477727 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:52.479171 28125 leader_election.cc:343] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.476526 28439 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:52.487608 28277 tablet_bootstrap.cc:654] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.501307 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:10:52.511700 28127 leader_election.cc:343] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.512197 28127 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:52.520576 28541 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.540807 28527 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.546121 27982 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.546367 28527 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.573997 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:52.577493 28546 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.578061 28546 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.579015 28527 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:52.581178 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:10:52.583237 28052 leader_election.cc:343] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.589468 28546 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:52.594233 28232 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:52.594269 28050 leader_election.cc:343] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.595115 28050 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:52.597237 28356 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.601770 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:52.602653 28154 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:52.604225 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:52.622249 28527 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.623749 28275 tablet_bootstrap.cc:492] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:52.624488 28275 ts_tablet_manager.cc:1403] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.384s	user 0.013s	sys 0.008s
I20260709 13:10:52.625808 28199 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:52.623749 28276 tablet_bootstrap.cc:492] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:52.627085 28276 ts_tablet_manager.cc:1403] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.359s	user 0.040s	sys 0.041s
I20260709 13:10:52.635358 28275 raft_consensus.cc:359] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.636121 28275 raft_consensus.cc:385] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.636401 28275 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.637511 28546 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.639918 28275 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.643126 28276 raft_consensus.cc:359] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.644019 28276 raft_consensus.cc:385] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.644291 28276 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.644408 28275 ts_tablet_manager.cc:1434] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:10:52.645453 28276 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.638998 28527 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.649039 28276 ts_tablet_manager.cc:1434] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 13:10:52.649510 28277 tablet_bootstrap.cc:492] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:52.652509 28277 ts_tablet_manager.cc:1403] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.340s	user 0.027s	sys 0.036s
I20260709 13:10:52.652870 28276 tablet_bootstrap.cc:492] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:52.659982 28275 tablet_bootstrap.cc:492] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
W20260709 13:10:52.661924 28125 consensus_peers.cc:597] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:52.671568 28275 tablet_bootstrap.cc:654] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.674973 28277 raft_consensus.cc:359] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.675772 28277 raft_consensus.cc:385] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.676242 28277 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.676985 28277 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.682230 28277 ts_tablet_manager.cc:1434] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 13:10:52.679628 28276 tablet_bootstrap.cc:654] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.692075 28276 tablet_bootstrap.cc:492] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:52.692643 28276 ts_tablet_manager.cc:1403] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.001s
I20260709 13:10:52.693599 28277 tablet_bootstrap.cc:492] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:52.695820 28276 raft_consensus.cc:359] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.698154 28276 raft_consensus.cc:385] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.698565 28276 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.699383 28276 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.702486 28276 ts_tablet_manager.cc:1434] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:10:52.704520 28276 tablet_bootstrap.cc:492] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:52.707370 28541 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.709332 28277 tablet_bootstrap.cc:654] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.715858 28275 tablet_bootstrap.cc:492] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:52.715960 28276 tablet_bootstrap.cc:654] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.716409 28275 ts_tablet_manager.cc:1403] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.006s
I20260709 13:10:52.722319 28275 raft_consensus.cc:359] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.723465 28275 raft_consensus.cc:385] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.723858 28275 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.725757 28275 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.728262 27984 catalog_manager.cc:5697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.740788 28277 tablet_bootstrap.cc:492] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:52.741653 28277 ts_tablet_manager.cc:1403] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.001s
I20260709 13:10:52.745352 28277 raft_consensus.cc:359] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.746279 28277 raft_consensus.cc:385] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.746625 28277 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.748708 28275 ts_tablet_manager.cc:1434] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.001s
I20260709 13:10:52.750597 28275 tablet_bootstrap.cc:492] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:52.751678 28276 tablet_bootstrap.cc:492] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:52.752144 28276 ts_tablet_manager.cc:1403] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.023s
I20260709 13:10:52.747617 28277 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.754863 28276 raft_consensus.cc:359] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.755792 28276 raft_consensus.cc:385] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.756053 28276 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.756316 28277 ts_tablet_manager.cc:1434] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 13:10:52.756675 28276 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.757412 28277 tablet_bootstrap.cc:492] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:52.758260 28276 ts_tablet_manager.cc:1434] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:10:52.759243 28276 tablet_bootstrap.cc:492] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:52.762337 28275 tablet_bootstrap.cc:654] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.763623 28550 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.764981 28276 tablet_bootstrap.cc:654] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.764112 28550 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.768325 28550 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:52.768951 28277 tablet_bootstrap.cc:654] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.778354 28277 tablet_bootstrap.cc:492] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:52.778853 28277 ts_tablet_manager.cc:1403] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.006s
I20260709 13:10:52.780891 28276 tablet_bootstrap.cc:492] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:52.781351 28276 ts_tablet_manager.cc:1403] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.006s
I20260709 13:10:52.781507 28277 raft_consensus.cc:359] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.782239 28277 raft_consensus.cc:385] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.785341 28276 raft_consensus.cc:359] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.786311 28276 raft_consensus.cc:385] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.786657 28276 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.787323 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:52.782505 28277 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.787611 28276 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.788002 28161 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:52.789836 28276 ts_tablet_manager.cc:1434] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:10:52.788347 28277 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.791035 28276 tablet_bootstrap.cc:492] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:52.792322 28277 ts_tablet_manager.cc:1434] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:10:52.793330 28277 tablet_bootstrap.cc:492] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:52.796789 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:52.797609 28276 tablet_bootstrap.cc:654] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.797564 28083 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:52.798836 28275 tablet_bootstrap.cc:492] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:52.799321 28275 ts_tablet_manager.cc:1403] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.001s
I20260709 13:10:52.802470 28275 raft_consensus.cc:359] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.803505 28275 raft_consensus.cc:385] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.803927 28275 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.804765 28275 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.807876 28451 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.808171 28275 ts_tablet_manager.cc:1434] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:10:52.808481 28451 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.809216 28275 tablet_bootstrap.cc:492] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:52.809804 28276 tablet_bootstrap.cc:492] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:52.810226 28276 ts_tablet_manager.cc:1403] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 13:10:52.810765 28451 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:52.815064 28275 tablet_bootstrap.cc:654] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.815552 28546 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.816015 28546 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.818723 28277 tablet_bootstrap.cc:654] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.813063 28276 raft_consensus.cc:359] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.821314 28276 raft_consensus.cc:385] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.821713 28276 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.824569 28546 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:52.824343 28276 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.826435 28276 ts_tablet_manager.cc:1434] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 13:10:52.827637 28276 tablet_bootstrap.cc:492] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:52.832633 28430 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:52.835628 28277 tablet_bootstrap.cc:492] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:52.836091 28276 tablet_bootstrap.cc:654] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.836480 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:52.837271 28161 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:52.836091 28277 ts_tablet_manager.cc:1403] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.002s
I20260709 13:10:52.837505 28275 tablet_bootstrap.cc:492] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:52.838783 28275 ts_tablet_manager.cc:1403] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.004s
I20260709 13:10:52.841763 28277 raft_consensus.cc:359] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.842706 28277 raft_consensus.cc:385] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.842969 28277 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.843794 28277 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.846634 28277 ts_tablet_manager.cc:1434] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:10:52.848006 28277 tablet_bootstrap.cc:492] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:52.852064 28275 raft_consensus.cc:359] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.853057 28275 raft_consensus.cc:385] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.853574 28275 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.853948 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:52.852989 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:52.855928 28359 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.856338 28359 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.856817 28359 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.854528 28275 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.870863 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:52.878409 28543 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.878942 28543 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.879383 28543 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.881106 28430 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.881675 28275 ts_tablet_manager.cc:1434] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 13:10:52.882277 28525 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.882383 28527 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:52.882871 28525 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.885366 28525 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:52.888374 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:52.889287 28161 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:52.871846 28359 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.890619 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:52.891511 28359 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:52.892361 28356 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:52.892274 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:52.892848 28356 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.893056 28232 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:52.893282 28356 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.895947 28276 tablet_bootstrap.cc:492] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:52.896431 28276 ts_tablet_manager.cc:1403] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.069s	user 0.021s	sys 0.038s
I20260709 13:10:52.881675 28430 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.899056 28276 raft_consensus.cc:359] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.899761 28276 raft_consensus.cc:385] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:52.900067 28276 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:52.900257 28356 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.901520 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:52.902009 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:52.902338 28543 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.903656 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:52.904271 28230 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.904353 28430 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:52.904778 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:52.905352 28160 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.913116 28230 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:52.914510 28052 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:52.915380 28430 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.915961 28430 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:10:52.916988 28199 leader_election.cc:343] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.917708 28275 tablet_bootstrap.cc:492] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:52.918133 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:52.915548 28277 tablet_bootstrap.cc:654] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.918756 28154 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.917636 28430 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.921916 28543 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:52.922581 28356 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:52.922520 28276 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.924005 28160 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:52.925596 28276 ts_tablet_manager.cc:1434] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:10:52.929816 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:52.934058 28153 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.933996 28276 tablet_bootstrap.cc:492] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:52.917018 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:52.926143 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:52.936208 28359 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.956938 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:52.957623 28082 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:52.959396 28154 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:52.960345 28538 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.942988 28430 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.961416 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:52.946141 28155 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:52.970029 28153 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:52.971488 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:52.975574 28538 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.977480 28275 tablet_bootstrap.cc:654] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:52.975697 28359 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:52.984470 28199 leader_election.cc:343] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:52.986160 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:52.988006 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:52.990494 28411 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:52.992004 28550 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:52.975596 28082 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:52.993650 28411 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:52.994493 28411 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:52.994427 27983 catalog_manager.cc:5697] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:52.996210 28538 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.000039 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:53.000810 28230 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:53.003072 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:53.004751 28538 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.005154 28538 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.005506 28538 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.009097 28276 tablet_bootstrap.cc:654] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.012830 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:53.016837 28359 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:53.017379 28277 tablet_bootstrap.cc:492] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:53.017889 28277 ts_tablet_manager.cc:1403] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.170s	user 0.034s	sys 0.026s
I20260709 13:10:53.017940 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:10:53.021377 28199 leader_election.cc:343] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.023978 28538 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.029428 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:53.029341 28277 raft_consensus.cc:359] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.030206 28277 raft_consensus.cc:385] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.030565 28277 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.031515 28356 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.030206 28230 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.031230 28277 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.032094 28356 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.039505 28277 ts_tablet_manager.cc:1434] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:10:53.035097 28538 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.058141 28277 tablet_bootstrap.cc:492] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:53.043004 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:53.059697 28451 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.060220 28451 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:53.061249 28451 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.043499 28230 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:53.045128 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:53.064868 28086 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.064962 28277 tablet_bootstrap.cc:654] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:53.067022 28199 leader_election.cc:343] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.047627 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:53.048486 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:53.050118 28080 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.067677 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:10:53.073539 28052 leader_election.cc:343] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.074498 28550 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.075389 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:53.076220 28514 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.076691 28514 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:53.077109 28277 tablet_bootstrap.cc:492] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:53.077554 28514 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.055248 28276 tablet_bootstrap.cc:492] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:53.037180 28356 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.094310 28276 ts_tablet_manager.cc:1403] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.165s	user 0.016s	sys 0.000s
I20260709 13:10:53.055231 28275 tablet_bootstrap.cc:492] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:53.095635 28275 ts_tablet_manager.cc:1403] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.178s	user 0.016s	sys 0.003s
I20260709 13:10:53.077581 28277 ts_tablet_manager.cc:1403] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.005s
I20260709 13:10:53.098084 28275 raft_consensus.cc:359] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.098742 28275 raft_consensus.cc:385] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.099000 28275 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.099597 28086 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:53.099793 28277 raft_consensus.cc:359] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.100504 28277 raft_consensus.cc:385] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.100808 28277 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
W20260709 13:10:53.101775 28032 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:53.052914 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:53.105468 28275 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.106521 28153 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:53.103111 28276 raft_consensus.cc:359] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.109221 28050 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:53.111212 28430 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.111608 28430 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.111963 28430 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.109467 28276 raft_consensus.cc:385] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.113204 28276 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.113842 28276 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.101437 28277 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.122426 28276 ts_tablet_manager.cc:1434] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:10:53.122416 28430 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.124044 28276 tablet_bootstrap.cc:492] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:53.125202 28430 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.126932 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:53.127590 28161 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.132661 27983 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:53.134959 28277 ts_tablet_manager.cc:1434] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.037s	user 0.003s	sys 0.002s
I20260709 13:10:53.134346 27983 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:53.135922 28277 tablet_bootstrap.cc:492] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:53.137377 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:53.138657 27982 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.141754 28161 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
W20260709 13:10:53.142145 28052 leader_election.cc:343] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.145668 28050 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:53.147650 28275 ts_tablet_manager.cc:1434] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 13:10:53.147727 28430 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.149259 28430 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:53.149410 28275 tablet_bootstrap.cc:492] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:53.150197 28430 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:53.155627 28601 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:10:53.162402 28277 tablet_bootstrap.cc:654] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.175056 28451 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.175663 28451 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.177949 28451 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:53.178844 28277 tablet_bootstrap.cc:492] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:53.179569 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:53.179838 28277 ts_tablet_manager.cc:1403] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.007s
I20260709 13:10:53.180256 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:10:53.181092 28199 leader_election.cc:343] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:53.183359 28199 leader_election.cc:343] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.183586 28277 raft_consensus.cc:359] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.183845 28199 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:53.184321 28277 raft_consensus.cc:385] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.184711 28277 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.184839 28359 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.185668 28277 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.187441 28277 ts_tablet_manager.cc:1434] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:10:53.188372 28277 tablet_bootstrap.cc:492] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:53.192550 28275 tablet_bootstrap.cc:654] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.193270 28276 tablet_bootstrap.cc:654] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.197669 28277 tablet_bootstrap.cc:654] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.208459 28275 tablet_bootstrap.cc:492] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:53.208978 28275 ts_tablet_manager.cc:1403] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.003s
I20260709 13:10:53.212232 28277 tablet_bootstrap.cc:492] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:53.212792 28277 ts_tablet_manager.cc:1403] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.005s
I20260709 13:10:53.216049 28275 raft_consensus.cc:359] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.220937 28277 raft_consensus.cc:359] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.221743 28277 raft_consensus.cc:385] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.222051 28277 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.223246 28276 tablet_bootstrap.cc:492] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:53.216868 28275 raft_consensus.cc:385] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.223623 28275 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.223744 28276 ts_tablet_manager.cc:1403] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.100s	user 0.018s	sys 0.011s
I20260709 13:10:53.224401 28275 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.226785 28275 ts_tablet_manager.cc:1434] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:10:53.227900 28275 tablet_bootstrap.cc:492] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:53.228513 28527 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.229022 28527 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.230702 28527 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.222802 28277 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.232016 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:53.232692 28153 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:53.233309 28277 ts_tablet_manager.cc:1434] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:10:53.234079 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:53.234314 28275 tablet_bootstrap.cc:654] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.235044 28525 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.235417 28525 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.235736 28525 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.236303 28277 tablet_bootstrap.cc:492] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:53.214845 27983 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.243808 28525 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.243860 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:53.227510 28276 raft_consensus.cc:359] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.249212 28276 raft_consensus.cc:385] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.249545 28276 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.250288 28276 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.252489 28276 ts_tablet_manager.cc:1434] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.000s
I20260709 13:10:53.265203 28276 tablet_bootstrap.cc:492] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:53.256717 28230 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:53.266852 28525 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.267534 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:53.268376 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:53.294150 28153 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.295661 28231 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.328559 28430 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.332502 28430 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.341964 28275 tablet_bootstrap.cc:492] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:53.342242 28276 tablet_bootstrap.cc:654] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.342468 28275 ts_tablet_manager.cc:1403] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.115s	user 0.016s	sys 0.023s
I20260709 13:10:53.345510 28277 tablet_bootstrap.cc:654] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.352466 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:53.352465 28430 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.354002 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:53.355877 28232 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
W20260709 13:10:53.354815 28050 leader_election.cc:343] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.357491 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:53.360062 28430 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.360523 28430 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.360972 28430 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.362038 28231 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:53.367336 28277 tablet_bootstrap.cc:492] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:53.367805 28277 ts_tablet_manager.cc:1403] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.132s	user 0.011s	sys 0.001s
I20260709 13:10:53.371693 28277 raft_consensus.cc:359] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.372527 28277 raft_consensus.cc:385] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.373266 28525 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.373919 28277 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.374758 28052 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:53.374615 28277 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.375813 28527 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.373778 28525 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.376358 28527 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:53.376785 28277 ts_tablet_manager.cc:1434] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:10:53.377797 28525 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.377385 28527 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.378170 28277 tablet_bootstrap.cc:492] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:53.378407 28276 tablet_bootstrap.cc:492] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:53.378947 28276 ts_tablet_manager.cc:1403] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.118s	user 0.017s	sys 0.007s
I20260709 13:10:53.380651 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:53.381315 28161 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:53.381345 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:53.384091 28430 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:53.385609 28052 leader_election.cc:343] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.386433 28275 raft_consensus.cc:359] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.387737 28276 raft_consensus.cc:359] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.390044 28275 raft_consensus.cc:385] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.390127 28276 raft_consensus.cc:385] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.390587 28275 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.391772 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:53.390666 28276 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.394927 28275 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.393980 28356 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.394114 28153 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:53.400544 28276 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.394518 28525 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:10:53.403507 28275 ts_tablet_manager.cc:1434] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.003s
I20260709 13:10:53.403618 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:53.401214 28356 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.404855 28356 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:53.406314 28050 leader_election.cc:343] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.420241 28356 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.430285 28276 ts_tablet_manager.cc:1434] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.001s
I20260709 13:10:53.430370 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:53.431169 28161 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.433821 28430 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.435256 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:53.435940 28231 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.436252 28277 tablet_bootstrap.cc:654] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.439371 28161 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:53.440277 28356 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.440538 28275 tablet_bootstrap.cc:492] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:53.442386 28276 tablet_bootstrap.cc:492] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:53.442479 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:10:53.443805 28052 leader_election.cc:343] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.449010 28231 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:53.450515 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:53.451289 28356 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.451778 28356 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:53.452543 28356 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.455225 27982 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.461917 28276 tablet_bootstrap.cc:654] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.456012 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:53.465140 28525 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.466261 28525 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:53.467188 28525 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.474357 28527 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:53.520181 27982 catalog_manager.cc:5697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.525130 27982 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.527820 28356 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.528337 28356 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.537938 28550 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:10:53.541401 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:53.542078 28161 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:53.545154 28301 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:53.537451 28451 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:10:53.546975 28275 tablet_bootstrap.cc:654] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.548090 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:53.557935 28356 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.561793 28277 tablet_bootstrap.cc:492] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:53.562237 28277 ts_tablet_manager.cc:1403] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.184s	user 0.013s	sys 0.006s
I20260709 13:10:53.555387 28527 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.564682 28277 raft_consensus.cc:359] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.566671 28277 raft_consensus.cc:385] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.567072 28277 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.567687 28277 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:53.567559 28616 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:10:53.566249 28527 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.573742 28277 ts_tablet_manager.cc:1434] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:10:53.573866 28356 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.574939 28356 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.573319 28527 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.579371 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:53.579799 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:53.580380 28232 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:53.592052 28356 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.593456 28276 tablet_bootstrap.cc:492] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:53.594061 28276 ts_tablet_manager.cc:1403] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.152s	user 0.028s	sys 0.051s
I20260709 13:10:53.594120 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:53.598079 28277 tablet_bootstrap.cc:492] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:53.599921 28538 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:53.622531 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:53.624452 28525 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.631332 28276 raft_consensus.cc:359] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.640961 28356 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:53.651413 28276 raft_consensus.cc:385] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.651751 28276 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.651705 28527 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.653128 28276 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.654708 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:53.655308 28160 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.656002 28276 ts_tablet_manager.cc:1434] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.003s
I20260709 13:10:53.643038 28275 tablet_bootstrap.cc:492] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:53.657274 28275 ts_tablet_manager.cc:1403] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.229s	user 0.015s	sys 0.030s
I20260709 13:10:53.657471 28276 tablet_bootstrap.cc:492] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:53.660105 28275 raft_consensus.cc:359] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.660917 28275 raft_consensus.cc:385] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.661231 28275 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.648634 28277 tablet_bootstrap.cc:654] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:53.648983 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49346.6us
I20260709 13:10:53.649955 28082 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.661940 28275 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.665398 28276 tablet_bootstrap.cc:654] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.667143 28275 ts_tablet_manager.cc:1434] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:10:53.672051 28527 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.674813 28160 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:53.676594 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:53.675611 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:53.684590 28275 tablet_bootstrap.cc:492] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:53.683355 28565 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:10:53.684815 28230 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.685487 28527 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:53.707564 28527 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:53.710536 28527 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.722059 28276 tablet_bootstrap.cc:492] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:53.723330 28276 ts_tablet_manager.cc:1403] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.066s	user 0.014s	sys 0.009s
W20260709 13:10:53.785391 28052 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:53.811894 28628 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.812453 28628 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.753544 28276 raft_consensus.cc:359] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.815240 28276 raft_consensus.cc:385] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.815564 28276 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.816274 28276 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.777662 28565 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.821547 28565 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.823764 28565 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:53.825627 28628 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.833622 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:53.834415 28082 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:53.834862 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:53.836778 28275 tablet_bootstrap.cc:654] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.837066 28565 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:53.838793 28276 ts_tablet_manager.cc:1434] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.115s	user 0.007s	sys 0.011s
I20260709 13:10:53.840801 27982 catalog_manager.cc:5697] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:53.839864 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:53.845023 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:53.846405 28230 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
W20260709 13:10:53.846825 28050 leader_election.cc:343] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.849224 28277 tablet_bootstrap.cc:492] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
W20260709 13:10:53.849582 28199 leader_election.cc:343] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.849730 28277 ts_tablet_manager.cc:1403] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.255s	user 0.017s	sys 0.040s
I20260709 13:10:53.855191 28276 tablet_bootstrap.cc:492] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:53.857251 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:53.845607 28231 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:53.865487 28451 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.866099 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:53.867769 28277 raft_consensus.cc:359] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.868789 28277 raft_consensus.cc:385] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.868949 28451 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.869163 28277 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.871790 28277 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.869402 28451 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.868798 28628 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:53.877166 28277 ts_tablet_manager.cc:1434] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:10:53.878496 28277 tablet_bootstrap.cc:492] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:53.877481 28628 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.884541 28628 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.892798 28451 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.895740 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:53.897563 28628 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.897854 28356 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:53.899724 28628 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.901022 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:53.901274 28086 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:53.901630 28231 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 1: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:10:53.912315 28451 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:53.913641 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:53.915055 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:10:53.916890 28050 leader_election.cc:343] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:53.917297 28199 leader_election.cc:343] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.917730 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:53.921123 28275 tablet_bootstrap.cc:492] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:53.921622 28275 ts_tablet_manager.cc:1403] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.238s	user 0.006s	sys 0.019s
I20260709 13:10:53.920401 28451 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.924427 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:53.924656 28275 raft_consensus.cc:359] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.925390 28275 raft_consensus.cc:385] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.925647 28275 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.925932 28356 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:53.926443 28277 tablet_bootstrap.cc:654] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.926393 28275 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.928887 28275 ts_tablet_manager.cc:1434] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:53.933491 28275 tablet_bootstrap.cc:492] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:53.939352 28276 tablet_bootstrap.cc:654] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.945811 28628 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:53.949152 28430 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:53.947897 28275 tablet_bootstrap.cc:654] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.951761 28277 tablet_bootstrap.cc:492] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:53.952232 28277 ts_tablet_manager.cc:1403] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.008s
I20260709 13:10:53.955219 28277 raft_consensus.cc:359] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.955971 28277 raft_consensus.cc:385] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.956223 28277 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:53.956827 28277 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.959033 28277 ts_tablet_manager.cc:1434] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:10:53.959980 28277 tablet_bootstrap.cc:492] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:53.966060 28277 tablet_bootstrap.cc:654] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:53.969834 28161 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:53.971002 28451 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.971508 28451 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.973572 28451 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:53.974383 28430 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.974828 28430 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.977459 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:53.980214 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:53.982185 28050 leader_election.cc:343] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:53.983155 28525 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:53.984525 28430 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:53.994415 28275 tablet_bootstrap.cc:492] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:53.985136 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:53.988842 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:53.995005 28275 ts_tablet_manager.cc:1403] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.062s	user 0.007s	sys 0.005s
I20260709 13:10:53.995404 28230 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:53.997575 28275 raft_consensus.cc:359] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:53.998303 28275 raft_consensus.cc:385] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:53.998404 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:53.999998 28525 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.000365 28525 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.000788 28525 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.001011 28161 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:54.011683 28525 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.014945 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:10:54.016165 28050 leader_election.cc:343] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.019385 28277 tablet_bootstrap.cc:492] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:54.019905 28277 ts_tablet_manager.cc:1403] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.026s
I20260709 13:10:54.021837 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:54.021983 28276 tablet_bootstrap.cc:492] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:54.022748 28231 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.023090 28276 ts_tablet_manager.cc:1403] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.172s	user 0.017s	sys 0.000s
I20260709 13:10:54.024397 28230 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.026646 28276 raft_consensus.cc:359] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.027347 28276 raft_consensus.cc:385] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.021863 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:54.029872 28231 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:54.030455 28550 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.030795 28525 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.030853 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:53.998587 28275 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.024346 28277 raft_consensus.cc:359] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.031872 28275 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.032601 28277 raft_consensus.cc:385] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.032936 28277 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.033608 28277 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.033824 28525 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.034765 28525 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:54.035666 28525 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.036834 28277 ts_tablet_manager.cc:1434] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:10:54.037122 28430 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.037891 28277 tablet_bootstrap.cc:492] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:54.042133 28275 ts_tablet_manager.cc:1434] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.000s
I20260709 13:10:54.027618 28276 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.044039 28276 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.044574 28275 tablet_bootstrap.cc:492] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:54.046227 28538 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.048544 28277 tablet_bootstrap.cc:654] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.049724 28538 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.052086 28538 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.052404 28276 ts_tablet_manager.cc:1434] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:10:54.054389 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:54.058475 28125 leader_election.cc:343] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.059110 28301 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.059572 28301 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.061928 28301 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.063988 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:54.064575 28080 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:54.065796 28125 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:54.067103 28275 tablet_bootstrap.cc:654] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.068164 28624 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.068549 28624 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.070293 28624 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.070426 28276 tablet_bootstrap.cc:492] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:54.073185 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:10:54.075995 28127 leader_election.cc:343] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.076383 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:54.077732 28356 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.078238 28514 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.085250 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:54.096854 28624 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.105038 28624 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.107437 28277 tablet_bootstrap.cc:492] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:54.107812 28275 tablet_bootstrap.cc:492] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:54.107997 28277 ts_tablet_manager.cc:1403] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.070s	user 0.008s	sys 0.005s
I20260709 13:10:54.108348 28275 ts_tablet_manager.cc:1403] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.005s
I20260709 13:10:54.109660 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:54.110268 28080 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.111001 28277 raft_consensus.cc:359] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.111189 28275 raft_consensus.cc:359] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.112636 28356 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.118860 28277 raft_consensus.cc:385] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.119009 28275 raft_consensus.cc:385] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.119428 28275 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.119182 28356 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.121506 28356 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.118130 28080 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:54.123539 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:10:54.124863 28050 leader_election.cc:343] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.125281 28125 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:54.121135 28275 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.128393 28275 ts_tablet_manager.cc:1434] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 13:10:54.128887 28514 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.129387 28514 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:10:54.130482 28275 tablet_bootstrap.cc:492] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:54.130152 28514 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.131805 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:54.137295 28275 tablet_bootstrap.cc:654] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:54.139596 28127 leader_election.cc:343] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:10:54.140920 28052 leader_election.cc:343] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.141361 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:54.144140 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:54.119306 28277 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.146629 28525 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.145747 28277 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:54.149245 28127 leader_election.cc:343] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.152315 28277 ts_tablet_manager.cc:1434] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.044s	user 0.002s	sys 0.005s
I20260709 13:10:54.153908 28277 tablet_bootstrap.cc:492] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:54.115052 28276 tablet_bootstrap.cc:654] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.160065 27981 catalog_manager.cc:5697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.163954 27983 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:54.180814 28525 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.190151 28565 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.184557 28277 tablet_bootstrap.cc:654] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.190868 28565 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.197391 28565 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:54.200589 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:54.203866 28525 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.204759 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:54.209959 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:54.214777 28276 tablet_bootstrap.cc:492] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:54.215384 28276 ts_tablet_manager.cc:1403] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.145s	user 0.016s	sys 0.012s
I20260709 13:10:54.218935 28275 tablet_bootstrap.cc:492] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:54.218851 28550 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.219535 28275 ts_tablet_manager.cc:1403] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.090s	user 0.021s	sys 0.018s
W20260709 13:10:54.226565 28032 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:54.227494 28276 raft_consensus.cc:359] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.228682 28275 raft_consensus.cc:359] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.229319 28276 raft_consensus.cc:385] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.229818 28276 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.230526 28276 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.232626 28276 ts_tablet_manager.cc:1434] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.003s
I20260709 13:10:54.235642 28276 tablet_bootstrap.cc:492] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:54.229434 28275 raft_consensus.cc:385] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.240150 28275 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.240949 28275 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.244376 28275 ts_tablet_manager.cc:1434] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:10:54.247246 28276 tablet_bootstrap.cc:654] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:54.248350 28261 tablet.cc:2406] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:10:54.252031 28275 tablet_bootstrap.cc:492] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:54.259606 28275 tablet_bootstrap.cc:654] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.271694 28277 tablet_bootstrap.cc:492] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:54.271903 28275 tablet_bootstrap.cc:492] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:54.272230 28277 ts_tablet_manager.cc:1403] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.119s	user 0.010s	sys 0.001s
I20260709 13:10:54.272327 28275 ts_tablet_manager.cc:1403] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.007s
I20260709 13:10:54.275305 28277 raft_consensus.cc:359] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.275386 28275 raft_consensus.cc:359] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.276127 28277 raft_consensus.cc:385] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.276187 28275 raft_consensus.cc:385] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.276597 28277 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.276628 28275 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.298040 28277 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.292865 28565 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.302853 28565 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.305094 28565 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:54.306416 28275 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.309839 28276 tablet_bootstrap.cc:492] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:54.310320 28276 ts_tablet_manager.cc:1403] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.075s	user 0.028s	sys 0.003s
I20260709 13:10:54.310633 28235 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.313555 28276 raft_consensus.cc:359] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.314261 28276 raft_consensus.cc:385] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.314527 28276 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.315122 28276 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.317504 28276 ts_tablet_manager.cc:1434] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:10:54.318785 28276 tablet_bootstrap.cc:492] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:54.324106 28275 ts_tablet_manager.cc:1434] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.051s	user 0.011s	sys 0.003s
I20260709 13:10:54.325459 28276 tablet_bootstrap.cc:654] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.326819 28275 tablet_bootstrap.cc:492] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:54.337533 28277 ts_tablet_manager.cc:1434] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.004s
I20260709 13:10:54.339915 28276 tablet_bootstrap.cc:492] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:54.340493 28276 ts_tablet_manager.cc:1403] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:10:54.341218 28275 tablet_bootstrap.cc:654] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.338581 28277 tablet_bootstrap.cc:492] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:54.353142 28277 tablet_bootstrap.cc:654] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.356943 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:54.349874 28276 raft_consensus.cc:359] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.358084 28276 raft_consensus.cc:385] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.357910 28082 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:54.358398 28276 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.359084 28276 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.361166 28276 ts_tablet_manager.cc:1434] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:10:54.361495 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:54.362143 28276 tablet_bootstrap.cc:492] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:54.371176 28276 tablet_bootstrap.cc:654] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.383862 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:54.384322 28276 tablet_bootstrap.cc:492] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:54.385650 28276 ts_tablet_manager.cc:1403] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.006s
I20260709 13:10:54.386435 28633 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.387804 28633 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.388281 28633 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.396672 28633 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.402819 28633 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:54.400609 28276 raft_consensus.cc:359] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.403568 28276 raft_consensus.cc:385] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.403939 28276 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.404758 28276 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:54.406113 28199 leader_election.cc:343] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.407083 28276 ts_tablet_manager.cc:1434] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:10:54.408211 28276 tablet_bootstrap.cc:492] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:54.409459 28275 tablet_bootstrap.cc:492] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:54.409988 28275 ts_tablet_manager.cc:1403] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.083s	user 0.031s	sys 0.028s
I20260709 13:10:54.412657 28275 raft_consensus.cc:359] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.413437 28275 raft_consensus.cc:385] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.413481 28277 tablet_bootstrap.cc:492] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:54.413874 28275 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.414088 28277 ts_tablet_manager.cc:1403] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.076s	user 0.023s	sys 0.025s
I20260709 13:10:54.414814 28276 tablet_bootstrap.cc:654] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.414654 28275 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.417460 28275 ts_tablet_manager.cc:1434] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:10:54.421305 28275 tablet_bootstrap.cc:492] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:54.421080 28277 raft_consensus.cc:359] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.421811 28277 raft_consensus.cc:385] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.422196 28277 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.423171 28277 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.427350 28277 ts_tablet_manager.cc:1434] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.004s
I20260709 13:10:54.428601 28275 tablet_bootstrap.cc:654] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.428790 28277 tablet_bootstrap.cc:492] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:54.438686 28301 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.439442 28301 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.441138 28359 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.441917 28359 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.444376 28359 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:54.448166 28277 tablet_bootstrap.cc:654] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.423352 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:54.449909 28086 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.458307 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:54.459575 28235 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:54.461985 28277 tablet_bootstrap.cc:492] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:54.462555 28277 ts_tablet_manager.cc:1403] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.008s
I20260709 13:10:54.467972 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:54.469861 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:54.470672 28161 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.481026 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:54.441972 28301 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.484776 28277 raft_consensus.cc:359] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.485807 28277 raft_consensus.cc:385] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.486167 28277 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.486999 28277 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.489867 28277 ts_tablet_manager.cc:1434] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:10:54.496650 28086 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:54.500334 28277 tablet_bootstrap.cc:492] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:54.516635 28275 tablet_bootstrap.cc:492] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:54.504362 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:54.517150 28275 ts_tablet_manager.cc:1403] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.096s	user 0.041s	sys 0.050s
I20260709 13:10:54.517354 28080 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:54.517750 28276 tablet_bootstrap.cc:492] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
W20260709 13:10:54.507845 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44720us
I20260709 13:10:54.509430 28628 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.518702 28276 ts_tablet_manager.cc:1403] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.111s	user 0.050s	sys 0.040s
I20260709 13:10:54.518424 28628 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.509884 28525 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:54.520668 28359 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.521176 28359 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.521612 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:54.519950 28275 raft_consensus.cc:359] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.522421 28275 raft_consensus.cc:385] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.522727 28275 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.522893 28359 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:54.513064 28161 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:54.520551 28628 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.523376 28514 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.524874 28276 raft_consensus.cc:359] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.525560 28276 raft_consensus.cc:385] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.525810 28276 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.526348 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:54.526438 28276 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.529173 28276 ts_tablet_manager.cc:1434] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:10:54.529089 28229 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:54.530143 28276 tablet_bootstrap.cc:492] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:54.532480 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:54.534071 28628 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.534446 28628 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.534752 28628 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.536041 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:54.523375 28275 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.536651 28161 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:54.536624 28276 tablet_bootstrap.cc:654] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.538076 28277 tablet_bootstrap.cc:654] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
W20260709 13:10:54.537385 28199 leader_election.cc:343] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.541932 28275 ts_tablet_manager.cc:1434] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:10:54.543051 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:54.544422 28082 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:10:54.544623 28275 tablet_bootstrap.cc:492] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:54.542162 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:54.545951 28565 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.546311 28565 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.546634 28565 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.548609 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:54.549520 28633 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.549973 28633 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:54.550737 28633 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.552872 28624 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.553359 28624 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.541057 28514 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.554435 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:54.554605 28565 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.555397 28624 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.559267 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:54.559904 28086 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.560252 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:54.561101 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:54.562155 28275 tablet_bootstrap.cc:654] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.562362 28565 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:54.563524 28082 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:54.565279 28301 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.566876 28430 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:54.566867 28301 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.570703 28235 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 1: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:10:54.572418 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:54.573463 28082 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:54.575445 28301 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.578630 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:54.579416 28161 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.579583 28628 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.582293 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:54.582898 28155 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.590816 28155 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:54.592021 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:54.593129 28235 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.593714 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:54.596138 28430 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.596675 28430 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:54.597463 28430 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.598145 28628 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.603276 28235 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:54.566102 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:54.621798 28161 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:54.626959 28277 tablet_bootstrap.cc:492] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:54.632064 28633 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.633813 28277 ts_tablet_manager.cc:1403] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.134s	user 0.011s	sys 0.001s
I20260709 13:10:54.635185 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:54.636206 28624 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.636572 28624 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.636940 28514 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.637982 28277 raft_consensus.cc:359] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.636940 28624 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:54.631035 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65001.3us
I20260709 13:10:54.639999 28127 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:54.641103 28514 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.643258 28633 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.647099 28624 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.649961 28276 tablet_bootstrap.cc:492] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:54.638731 28277 raft_consensus.cc:385] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.653398 28624 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.655290 28277 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.656280 28277 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.659233 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:54.659832 28234 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.661064 28277 ts_tablet_manager.cc:1434] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 13:10:54.662097 28277 tablet_bootstrap.cc:492] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:54.661986 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:54.662742 28359 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.663178 28359 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:54.663843 28359 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.655006 28276 ts_tablet_manager.cc:1403] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.125s	user 0.029s	sys 0.010s
I20260709 13:10:54.650681 28514 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.629105 28275 tablet_bootstrap.cc:492] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:54.667222 28275 ts_tablet_manager.cc:1403] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.123s	user 0.014s	sys 0.003s
I20260709 13:10:54.667415 28276 raft_consensus.cc:359] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.668087 28276 raft_consensus.cc:385] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.668356 28276 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.650890 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:54.670301 28276 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.671096 28275 raft_consensus.cc:359] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.671797 28275 raft_consensus.cc:385] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.672078 28275 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.673002 28275 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.675045 28275 ts_tablet_manager.cc:1434] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:10:54.675606 28276 ts_tablet_manager.cc:1434] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:10:54.676239 28234 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:54.676610 28276 tablet_bootstrap.cc:492] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:54.679195 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:54.679831 28082 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.676328 28275 tablet_bootstrap.cc:492] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:54.683588 27982 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:54.688617 28538 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:54.688822 28082 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:10:54.775851 28359 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.688139 28086 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:54.776371 28359 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.778239 28359 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:54.777794 27983 catalog_manager.cc:5697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:54.785802 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 91787.4us
I20260709 13:10:54.773941 28275 tablet_bootstrap.cc:654] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.789947 28277 tablet_bootstrap.cc:654] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.798949 28276 tablet_bootstrap.cc:654] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.799136 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:10:54.803068 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 113947us
W20260709 13:10:54.806035 28199 leader_election.cc:343] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.811471 28127 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:54.813645 28430 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.814149 28430 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.814179 28676 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.815395 28676 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:10:54.816170 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 122470us
I20260709 13:10:54.816721 28676 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.824838 28430 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.825697 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:10:54.832058 28050 leader_election.cc:343] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.835139 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:54.864559 28624 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.867799 28624 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.839289 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:54.870492 28160 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:54.872629 28277 tablet_bootstrap.cc:492] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:54.873466 28277 ts_tablet_manager.cc:1403] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.212s	user 0.015s	sys 0.016s
W20260709 13:10:54.840883 28052 leader_election.cc:343] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:54.874541 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:10:54.875504 28276 tablet_bootstrap.cc:492] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:54.875976 28276 ts_tablet_manager.cc:1403] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.200s	user 0.014s	sys 0.011s
I20260709 13:10:54.878819 28430 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:54.887913 28624 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.893046 28276 raft_consensus.cc:359] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.893805 28276 raft_consensus.cc:385] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.893525 28277 raft_consensus.cc:359] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.894093 28276 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.894240 28277 raft_consensus.cc:385] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.894567 28277 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.894776 28276 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.896318 28276 ts_tablet_manager.cc:1434] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.002s
I20260709 13:10:54.902908 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:54.903151 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:10:54.903545 28235 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:54.904547 28359 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.904928 28359 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.905241 28359 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.908591 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:54.910552 28127 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:54.912624 28275 tablet_bootstrap.cc:492] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:54.913161 28275 ts_tablet_manager.cc:1403] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.237s	user 0.023s	sys 0.000s
I20260709 13:10:54.915167 28538 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:54.916038 28277 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.917142 28086 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 0.
I20260709 13:10:54.917805 28276 tablet_bootstrap.cc:492] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b: Bootstrap starting.
I20260709 13:10:54.919279 28359 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.915612 28538 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.921257 28538 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.925323 28277 ts_tablet_manager.cc:1434] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.052s	user 0.003s	sys 0.003s
I20260709 13:10:54.928144 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:54.930528 28080 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:54.930981 28277 tablet_bootstrap.cc:492] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Bootstrap starting.
I20260709 13:10:54.937108 28080 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:54.941618 28275 raft_consensus.cc:359] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.944594 28275 raft_consensus.cc:385] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:54.944846 28359 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:54.944918 28275 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:54.949489 28538 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.946121 28275 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.951725 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:54.958230 28161 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 1: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:10:54.959733 28275 ts_tablet_manager.cc:1434] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:10:54.953663 28277 tablet_bootstrap.cc:654] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.953848 28276 tablet_bootstrap.cc:654] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.959856 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:54.954443 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:54.961522 28565 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:54.960916 28082 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 1: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:10:54.962522 28275 tablet_bootstrap.cc:492] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:54.965571 28538 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:54.968984 28275 tablet_bootstrap.cc:654] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:54.978888 28359 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.978739 28565 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:54.980688 28275 tablet_bootstrap.cc:492] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:54.981213 28275 ts_tablet_manager.cc:1403] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.006s
I20260709 13:10:54.981043 28565 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.986428 28525 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:54.979777 28359 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.989648 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:54.990965 28359 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:54.993464 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:54.994345 28082 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:54.995671 28127 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:54.997609 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:54.998306 28161 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:54.997519 28275 raft_consensus.cc:359] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:54.998090 28676 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:10:55.006121 28275 raft_consensus.cc:385] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.006376 28277 tablet_bootstrap.cc:492] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: No bootstrap required, opened a new log
I20260709 13:10:55.006942 28277 ts_tablet_manager.cc:1403] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent bootstrapping tablet: real 0.076s	user 0.009s	sys 0.009s
I20260709 13:10:55.007293 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:55.006476 28275 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.008025 28683 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:55.008366 28683 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.008685 28683 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.019161 27981 catalog_manager.cc:5697] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:55.025007 28275 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.023577 28359 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.027820 28275 ts_tablet_manager.cc:1434] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.005s
I20260709 13:10:55.027680 28359 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.029901 28683 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.030993 28359 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:55.032006 28683 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:55.032094 28277 raft_consensus.cc:359] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.032529 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:55.032927 28277 raft_consensus.cc:385] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.033023 28275 tablet_bootstrap.cc:492] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Bootstrap starting.
I20260709 13:10:55.033293 28277 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.034008 28277 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.034242 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:55.034902 28161 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:55.035933 28082 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 0.
I20260709 13:10:55.037698 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:55.038574 28683 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.039215 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:55.039812 28082 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.040378 28628 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.041349 28550 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:55.041775 28550 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.044101 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:55.044580 28277 ts_tablet_manager.cc:1434] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:10:55.045336 28160 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.046690 28550 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.052632 28082 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:55.054239 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:55.056160 28160 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:55.056962 28359 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.057520 28359 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:55.058524 28359 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.065364 28276 tablet_bootstrap.cc:492] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b: No bootstrap required, opened a new log
I20260709 13:10:55.065930 28276 ts_tablet_manager.cc:1403] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b: Time spent bootstrapping tablet: real 0.152s	user 0.012s	sys 0.005s
I20260709 13:10:55.082204 28276 raft_consensus.cc:359] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.083097 28276 raft_consensus.cc:385] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.083446 28276 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.084153 28276 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.088317 28550 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.092953 28276 ts_tablet_manager.cc:1434] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 13:10:55.099001 28275 tablet_bootstrap.cc:654] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Neither blocks nor log segments found. Creating new log.
I20260709 13:10:55.104964 28690 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:10:55.117460 28628 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.117137 27983 catalog_manager.cc:5697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.117986 28628 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.120036 28550 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:55.123116 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:55.123975 28235 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 0.
I20260709 13:10:55.118813 27983 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.126482 27983 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.126955 28052 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:55.129072 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:55.129446 28430 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:10:55.129906 28430 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.130311 28430 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:10:55.132185 28050 leader_election.cc:343] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.120476 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:55.120051 28628 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:55.137240 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:55.146342 28084 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.160488 28430 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.162544 28430 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:55.153743 28154 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.197526 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:10:55.198200 28230 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.203608 28154 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:55.204684 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:55.208432 28275 tablet_bootstrap.cc:492] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: No bootstrap required, opened a new log
I20260709 13:10:55.214679 28275 ts_tablet_manager.cc:1403] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent bootstrapping tablet: real 0.182s	user 0.020s	sys 0.027s
W20260709 13:10:55.219357 28691 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:10:55.219667 28230 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:10:55.222038 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:10:55.226140 28052 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:55.226482 28359 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.226948 28359 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:55.220927 28084 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:55.231426 28698 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:10:55.231980 28698 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:10:55.227656 28359 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.232851 28698 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:10:55.233777 28050 leader_election.cc:343] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:10:55.245926 28275 raft_consensus.cc:359] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.246807 28275 raft_consensus.cc:385] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:10:55.247157 28275 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Initialized, Role: FOLLOWER
I20260709 13:10:55.251804 28275 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.255718 28275 ts_tablet_manager.cc:1434] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.003s
I20260709 13:10:55.273864 27981 catalog_manager.cc:5697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.277810 28538 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.286898 27983 catalog_manager.cc:5697] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:55.288903 28411 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:10:55.289907 28032 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:10:55.297858 28633 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.302615 28538 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.359941 28538 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.360394 28538 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.362110 28538 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:55.362975 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:55.364300 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:10:55.365034 28235 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 1: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:10:55.366128 28127 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:10:55.366837 28538 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:10:55.373649 28538 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:10:55.398913 28359 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.410239 28411 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.513091 28690 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.513660 28690 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.515281 28690 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:55.516197 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:10:55.516547 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:10:55.516882 28080 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:55.517354 28154 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:10:55.518934 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:55.519559 28690 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:55.519925 28690 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.520249 28690 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.526423 28690 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.528265 28690 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:55.529525 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:10:55.530243 28080 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.529417 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:10:55.533387 28154 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:55.536367 28080 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:10:55.537714 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:10:55.538456 28690 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:55.539813 28690 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:10:55.540874 28690 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:10:55.542598 28154 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:10:55.551777 27982 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 0 to 2, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 2 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.578629 28411 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:10:55.589614 28411 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.702126 28699 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.717625 28698 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.760855 28550 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:55.775853 28411 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:10:56.012950 28084 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:56.014117 28550 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:56.022715 28161 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:10:56.024802 28550 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:10:56.071817 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:10:56.103513 28718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:10:56.107148 28719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:56.108119 26959 server_base.cc:1061] running on GCE node
W20260709 13:10:56.111452 28721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:10:56.112982 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:10:56.113245 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:10:56.113471 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602656113451 us; error 0 us; skew 500 ppm
I20260709 13:10:56.114128 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:10:56.117398 26959 webserver.cc:533] Webserver started at http://127.26.83.196:43833/ using document root <none> and password file <none>
I20260709 13:10:56.118142 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:10:56.118422 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:10:56.118876 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:10:56.120595 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-3-root/instance:
uuid: "70996712c3c942468a6006602de6477f"
format_stamp: "Formatted at 2026-07-09 13:10:56 on dist-test-slave-329l"
I20260709 13:10:56.126735 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:10:56.132388 28726 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.134052 26959 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:10:56.134490 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-3-root
uuid: "70996712c3c942468a6006602de6477f"
format_stamp: "Formatted at 2026-07-09 13:10:56 on dist-test-slave-329l"
I20260709 13:10:56.135020 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:10:56.170403 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:10:56.171958 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:10:56.173586 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:10:56.175987 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:10:56.176221 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.176534 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:10:56.176767 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:10:56.273015 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.196:46855
I20260709 13:10:56.273168 28789 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.196:46855 every 8 connection(s)
I20260709 13:10:56.337004 26959 auto_leader_rebalancer-test.cc:384] add a tserver: 70996712c3c942468a6006602de6477f
I20260709 13:10:56.346150 28790 heartbeater.cc:344] Connected to a master server at 127.26.83.254:42913
I20260709 13:10:56.346594 28790 heartbeater.cc:461] Registering TS with master...
I20260709 13:10:56.347394 28790 heartbeater.cc:507] Master 127.26.83.254:42913 requested a full tablet report, sending...
I20260709 13:10:56.349551 27982 ts_manager.cc:194] Registered new tserver with Master: 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:10:56.351423 27982 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56104
I20260709 13:10:57.354126 28790 heartbeater.cc:499] Master 127.26.83.254:42913 was elected leader, sending a full tablet report...
I20260709 13:10:57.466176 28154 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:57.473811 28235 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:57.478241 28084 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:57.475915 28676 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:57.481546 28676 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:57.506111 28794 raft_consensus.cc:2955] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:10:57.509207 28124 consensus_peers.cc:597] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1ce6e6dfcbee43d5b0550ce7fc634c38. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:57.513317 28084 raft_consensus.cc:2955] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.525848 28230 raft_consensus.cc:2955] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.527348 27982 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.550426 28230 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:57.564941 28154 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:57.565526 28084 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:57.568821 28411 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:57.570637 28411 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
W20260709 13:10:57.578286 28198 consensus_peers.cc:597] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c0743024f324c14b4c6fc08f70600d2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:57.600805 28633 raft_consensus.cc:2955] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.602946 28161 raft_consensus.cc:2955] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.608850 28084 raft_consensus.cc:2955] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.631304 28230 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:57.630841 27981 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:10:57.656553 28198 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 923fa739e01e4f05a05099628c53a6c3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:57.663461 28161 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:57.666841 28082 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:57.667789 28813 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:57.670502 28633 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:57.687808 28411 raft_consensus.cc:2955] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.689961 28161 raft_consensus.cc:2955] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.689838 28082 raft_consensus.cc:2955] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.706702 28234 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:57.712478 27981 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:57.723819 28198 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e53fd092d3dd48c492e04080969cb666. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:57.727731 28161 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:57.728984 28082 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:10:57.735764 28411 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:57.736687 28633 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:57.745422 28411 raft_consensus.cc:2955] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.747219 28154 raft_consensus.cc:2955] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.749140 28082 raft_consensus.cc:2955] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.766445 27983 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:10:57.984817 28822 ts_tablet_manager.cc:933] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:10:57.987027 28822 tablet_copy_client.cc:323] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:10:58.007346 28171 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet 1ce6e6dfcbee43d5b0550ce7fc634c38 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:10:58.007979 28171 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet 1ce6e6dfcbee43d5b0550ce7fc634c38 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-1ce6e6dfcbee43d5b0550ce7fc634c38
I20260709 13:10:58.016306 28171 tablet_copy_source_session.cc:215] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:58.022542 28822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce6e6dfcbee43d5b0550ce7fc634c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.036067 28822 tablet_copy_client.cc:806] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:58.037762 28822 tablet_copy_client.cc:670] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:58.039964 28826 ts_tablet_manager.cc:933] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:58.041918 28826 tablet_copy_client.cc:323] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:10:58.044221 28822 tablet_copy_client.cc:538] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:58.055230 28822 tablet_bootstrap.cc:492] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:10:58.056051 28245 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet 7c0743024f324c14b4c6fc08f70600d2 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:10:58.056747 28245 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet 7c0743024f324c14b4c6fc08f70600d2 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-7c0743024f324c14b4c6fc08f70600d2
I20260709 13:10:58.063453 28245 tablet_copy_source_session.cc:215] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:58.067121 28826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0743024f324c14b4c6fc08f70600d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.078146 28822 tablet_bootstrap.cc:492] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.079542 28822 tablet_bootstrap.cc:492] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:10:58.080806 28822 ts_tablet_manager.cc:1403] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.005s
I20260709 13:10:58.084465 28822 raft_consensus.cc:359] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:58.085623 28822 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:10:58.086505 28822 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:58.088321 28826 tablet_copy_client.cc:806] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:58.089169 28826 tablet_copy_client.cc:670] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:58.096345 28822 ts_tablet_manager.cc:1434] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.003s
I20260709 13:10:58.097486 28826 tablet_copy_client.cc:538] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:58.109760 28826 tablet_bootstrap.cc:492] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:10:58.110543 28171 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-1ce6e6dfcbee43d5b0550ce7fc634c38 received from {username='slave'} at 127.0.0.1:43184
I20260709 13:10:58.111100 28171 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-1ce6e6dfcbee43d5b0550ce7fc634c38 on tablet 1ce6e6dfcbee43d5b0550ce7fc634c38 with peer 70996712c3c942468a6006602de6477f
I20260709 13:10:58.133900 28826 tablet_bootstrap.cc:492] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.134924 28826 tablet_bootstrap.cc:492] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:10:58.135653 28826 ts_tablet_manager.cc:1403] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.008s
I20260709 13:10:58.139032 28826 raft_consensus.cc:359] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:58.140069 28826 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:10:58.140869 28826 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:58.143397 28826 ts_tablet_manager.cc:1434] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:10:58.145304 28245 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-7c0743024f324c14b4c6fc08f70600d2 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:10:58.145776 28245 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-7c0743024f324c14b4c6fc08f70600d2 on tablet 7c0743024f324c14b4c6fc08f70600d2 with peer 70996712c3c942468a6006602de6477f
I20260709 13:10:58.191382 28826 ts_tablet_manager.cc:933] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:58.193295 28826 tablet_copy_client.cc:323] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:10:58.195037 28245 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet e53fd092d3dd48c492e04080969cb666 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:10:58.195549 28245 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet e53fd092d3dd48c492e04080969cb666 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-e53fd092d3dd48c492e04080969cb666
I20260709 13:10:58.201005 28245 tablet_copy_source_session.cc:215] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:58.203979 28826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e53fd092d3dd48c492e04080969cb666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.215265 28822 ts_tablet_manager.cc:933] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:10:58.220430 28826 tablet_copy_client.cc:806] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:58.221158 28826 tablet_copy_client.cc:670] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:58.222163 28822 tablet_copy_client.cc:323] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:10:58.224597 28244 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet 923fa739e01e4f05a05099628c53a6c3 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:10:58.225256 28244 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet 923fa739e01e4f05a05099628c53a6c3 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-923fa739e01e4f05a05099628c53a6c3
I20260709 13:10:58.228892 28826 tablet_copy_client.cc:538] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:58.238772 28826 tablet_bootstrap.cc:492] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:10:58.239601 28244 tablet_copy_source_session.cc:215] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:58.244388 28822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 923fa739e01e4f05a05099628c53a6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:58.269126 28826 tablet_bootstrap.cc:492] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.271061 28826 tablet_bootstrap.cc:492] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:10:58.271852 28826 ts_tablet_manager.cc:1403] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.001s
I20260709 13:10:58.276177 28826 raft_consensus.cc:359] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:58.277308 28826 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:10:58.279846 28826 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:58.283385 28826 ts_tablet_manager.cc:1434] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:10:58.285689 28244 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-e53fd092d3dd48c492e04080969cb666 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:10:58.286181 28244 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-e53fd092d3dd48c492e04080969cb666 on tablet e53fd092d3dd48c492e04080969cb666 with peer 70996712c3c942468a6006602de6477f
I20260709 13:10:58.287827 28822 tablet_copy_client.cc:806] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:58.288466 28822 tablet_copy_client.cc:670] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:58.294454 28822 tablet_copy_client.cc:538] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:58.314059 28822 tablet_bootstrap.cc:492] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:10:58.335220 28822 tablet_bootstrap.cc:492] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:58.336023 28822 tablet_bootstrap.cc:492] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:10:58.336683 28822 ts_tablet_manager.cc:1403] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.000s
I20260709 13:10:58.339449 28822 raft_consensus.cc:359] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:58.340411 28822 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:10:58.341090 28822 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:10:58.343029 28822 ts_tablet_manager.cc:1434] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:10:58.344940 28244 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-923fa739e01e4f05a05099628c53a6c3 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:10:58.345413 28244 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-923fa739e01e4f05a05099628c53a6c3 on tablet 923fa739e01e4f05a05099628c53a6c3 with peer 70996712c3c942468a6006602de6477f
I20260709 13:10:58.570530 28764 raft_consensus.cc:1217] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:58.576555 28764 raft_consensus.cc:1217] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:58.597723 28764 raft_consensus.cc:1217] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:58.802704 28764 raft_consensus.cc:1217] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:10:59.006266 28411 raft_consensus.cc:1064] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:10:59.008677 28411 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:10:59.017210 28080 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.020184 28633 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:59.024302 28764 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.026224 28153 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.026715 28633 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:59.029582 28811 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:59.047886 28835 raft_consensus.cc:1064] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:10:59.051528 28835 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:10:59.059041 28633 raft_consensus.cc:2955] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.064674 28080 raft_consensus.cc:2955] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.075711 28764 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.078328 28084 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.079485 28794 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:59.085600 28835 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:59.090112 27982 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.096570 28763 raft_consensus.cc:2955] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.111671 28234 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.141461 28794 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:59.146380 28842 raft_consensus.cc:1064] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:10:59.147501 28153 raft_consensus.cc:2955] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.149048 28842 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:10:59.164311 28794 raft_consensus.cc:2955] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.168093 28080 raft_consensus.cc:2955] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.169399 28764 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.171046 28842 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:59.188786 28763 raft_consensus.cc:2955] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.220155 28080 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.221380 28154 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.229032 28230 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:10:59.230901 27981 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.246564 28762 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:59.246701 28234 raft_consensus.cc:2955] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.250185 28842 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:59.260442 28763 raft_consensus.cc:2955] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.261260 28842 raft_consensus.cc:2955] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.265368 28161 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:10:59.271011 28762 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:59.273159 28676 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:59.273618 28080 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:59.276005 28854 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:59.281772 27982 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.282788 28794 raft_consensus.cc:2955] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.285565 28846 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:59.287882 28762 raft_consensus.cc:2955] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.297525 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c0743024f324c14b4c6fc08f70600d2 with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40978
I20260709 13:10:59.297802 28846 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:59.311136 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1ce6e6dfcbee43d5b0550ce7fc634c38 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:59.314774 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7c0743024f324c14b4c6fc08f70600d2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:59.328068 28842 raft_consensus.cc:2955] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.330833 28857 tablet_replica.cc:333] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:10:59.331993 28857 raft_consensus.cc:2243] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:10:59.337927 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54386.5us
I20260709 13:10:59.337953 28857 raft_consensus.cc:2272] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:59.344998 28082 raft_consensus.cc:2955] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.347553 28857 ts_tablet_manager.cc:1916] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:59.349395 28161 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:59.359179 27982 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.369966 27983 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.379071 28216 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ce6e6dfcbee43d5b0550ce7fc634c38 with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
I20260709 13:10:59.381867 28082 raft_consensus.cc:2955] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.386765 28846 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:59.403179 28843 raft_consensus.cc:1064] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:10:59.405973 28843 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:10:59.401705 28762 raft_consensus.cc:2955] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.414774 28860 tablet_replica.cc:333] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:10:59.417258 28860 raft_consensus.cc:2243] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:59.420871 28860 raft_consensus.cc:2272] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:59.419157 28153 raft_consensus.cc:2955] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.426777 28860 ts_tablet_manager.cc:1916] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:59.450372 28857 ts_tablet_manager.cc:1929] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:59.450978 28857 log.cc:1199] T 7c0743024f324c14b4c6fc08f70600d2 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/7c0743024f324c14b4c6fc08f70600d2
I20260709 13:10:59.452667 28860 ts_tablet_manager.cc:1929] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:59.453504 28860 log.cc:1199] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/1ce6e6dfcbee43d5b0550ce7fc634c38
I20260709 13:10:59.452667 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet 7c0743024f324c14b4c6fc08f70600d2 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:10:59.457593 28157 raft_consensus.cc:2955] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.473769 28762 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.474105 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet 1ce6e6dfcbee43d5b0550ce7fc634c38 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:10:59.477557 28842 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:59.485594 28080 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.489677 28153 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:10:59.494876 28842 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:59.497534 28230 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:10:59.506703 28811 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:59.521230 28762 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:59.527356 28080 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:59.530751 28811 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:10:59.591936 28843 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:59.603904 28843 raft_consensus.cc:2955] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.614253 28762 raft_consensus.cc:2955] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.636842 27982 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.660069 28843 raft_consensus.cc:2955] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.662257 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet e53fd092d3dd48c492e04080969cb666 with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50454
I20260709 13:10:59.667568 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e53fd092d3dd48c492e04080969cb666 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:59.672170 28153 raft_consensus.cc:2955] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.689857 28866 tablet_replica.cc:333] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:10:59.690905 28866 raft_consensus.cc:2243] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:59.691790 28866 raft_consensus.cc:2272] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:59.695413 28866 ts_tablet_manager.cc:1916] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:59.699720 28086 raft_consensus.cc:2955] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.713290 28762 raft_consensus.cc:2955] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.723978 28866 ts_tablet_manager.cc:1929] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:59.724470 28866 log.cc:1199] T e53fd092d3dd48c492e04080969cb666 P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/e53fd092d3dd48c492e04080969cb666
I20260709 13:10:59.726387 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet e53fd092d3dd48c492e04080969cb666 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:10:59.725893 27982 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.732714 28086 raft_consensus.cc:2955] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.752677 28230 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:10:59.757829 28762 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:59.759497 28865 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:59.759087 28086 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:10:59.765098 28843 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:59.766942 28842 raft_consensus.cc:2955] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.773838 28080 raft_consensus.cc:2955] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.777168 28762 raft_consensus.cc:2955] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.783191 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 923fa739e01e4f05a05099628c53a6c3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:10:59.802495 27982 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:10:59.814831 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet 923fa739e01e4f05a05099628c53a6c3 with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50454
I20260709 13:10:59.815800 28866 tablet_replica.cc:333] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:10:59.816578 28866 raft_consensus.cc:2243] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:59.817219 28866 raft_consensus.cc:2272] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:59.819752 28866 ts_tablet_manager.cc:1916] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:59.831241 28866 ts_tablet_manager.cc:1929] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:10:59.831561 28866 log.cc:1199] T 923fa739e01e4f05a05099628c53a6c3 P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/923fa739e01e4f05a05099628c53a6c3
I20260709 13:10:59.832973 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet 923fa739e01e4f05a05099628c53a6c3 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:00.900882 28080 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:00.908962 28235 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:00.909200 28154 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:00.910466 28847 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:00.911432 28698 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:00.921900 28847 raft_consensus.cc:2955] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:11:00.925374 28049 consensus_peers.cc:597] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1115d5b02e774b14935e3f48488f82ea. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.924626 28154 raft_consensus.cc:2955] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:00.927634 28235 raft_consensus.cc:2955] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:00.938616 28235 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:00.945487 27981 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:11:00.964011 28198 consensus_peers.cc:597] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fab37ead976b4378b1298941cec677a8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:00.972973 28154 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:00.974442 28877 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:00.980926 28080 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:00.985203 28842 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:01.001260 28842 raft_consensus.cc:2955] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.013384 28086 raft_consensus.cc:2955] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.011499 28157 raft_consensus.cc:2955] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.037343 28086 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:01.064668 27982 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.078806 28234 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:01.080392 28847 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:01.083835 28157 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:01.084242 28049 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60a7d81169af4acc91d8f3b342ac4885. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:01.087522 28847 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:01.112290 28878 raft_consensus.cc:2955] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.114440 28234 raft_consensus.cc:2955] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.114440 28157 raft_consensus.cc:2955] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.130537 28086 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:01.134719 27982 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:11:01.141217 28049 consensus_peers.cc:597] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd81a684ef6b4e0f9cc48bc3bcc280e8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:01.141927 28230 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:01.143340 28847 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:01.147871 28153 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:01.150137 28867 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:01.156884 28878 raft_consensus.cc:2955] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.159204 28230 raft_consensus.cc:2955] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.186326 28153 raft_consensus.cc:2955] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.190315 27983 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:01.397969 28892 ts_tablet_manager.cc:933] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:01.400549 28892 tablet_copy_client.cc:323] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:11:01.402442 28244 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet fab37ead976b4378b1298941cec677a8 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:11:01.402976 28244 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet fab37ead976b4378b1298941cec677a8 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-fab37ead976b4378b1298941cec677a8
I20260709 13:11:01.410149 28244 tablet_copy_source_session.cc:215] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:01.413621 28892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab37ead976b4378b1298941cec677a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:01.435266 28892 tablet_copy_client.cc:806] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:01.435894 28892 tablet_copy_client.cc:670] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:01.440619 28892 tablet_copy_client.cc:538] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:01.443075 28895 ts_tablet_manager.cc:933] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:01.447300 28895 tablet_copy_client.cc:323] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:01.453624 28892 tablet_bootstrap.cc:492] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:01.494015 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet 1115d5b02e774b14935e3f48488f82ea from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:01.494685 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet 1115d5b02e774b14935e3f48488f82ea from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-1115d5b02e774b14935e3f48488f82ea
I20260709 13:11:01.505388 28096 tablet_copy_source_session.cc:215] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:01.510609 28895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1115d5b02e774b14935e3f48488f82ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:01.535296 28895 tablet_copy_client.cc:806] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:01.536002 28895 tablet_copy_client.cc:670] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:01.536453 28892 tablet_bootstrap.cc:492] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:01.537580 28892 tablet_bootstrap.cc:492] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:01.538267 28892 ts_tablet_manager.cc:1403] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.085s	user 0.016s	sys 0.027s
I20260709 13:11:01.541872 28895 tablet_copy_client.cc:538] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:01.542479 28892 raft_consensus.cc:359] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:01.543426 28892 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:01.544154 28892 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:01.552937 28895 tablet_bootstrap.cc:492] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:01.558722 28892 ts_tablet_manager.cc:1434] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.011s
I20260709 13:11:01.579252 28244 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-fab37ead976b4378b1298941cec677a8 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:11:01.579758 28244 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-fab37ead976b4378b1298941cec677a8 on tablet fab37ead976b4378b1298941cec677a8 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:01.599576 28892 ts_tablet_manager.cc:933] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:01.603303 28895 tablet_bootstrap.cc:492] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:01.604555 28895 tablet_bootstrap.cc:492] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:01.605337 28895 ts_tablet_manager.cc:1403] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.053s	user 0.025s	sys 0.008s
I20260709 13:11:01.611491 28892 tablet_copy_client.cc:323] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:01.614598 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet cd81a684ef6b4e0f9cc48bc3bcc280e8 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:01.615150 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet cd81a684ef6b4e0f9cc48bc3bcc280e8 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-cd81a684ef6b4e0f9cc48bc3bcc280e8
I20260709 13:11:01.623216 28096 tablet_copy_source_session.cc:215] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:01.626983 28895 raft_consensus.cc:359] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:01.629155 28892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd81a684ef6b4e0f9cc48bc3bcc280e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:01.629657 28895 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:01.637952 28895 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:01.642437 28895 ts_tablet_manager.cc:1434] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.001s
I20260709 13:11:01.646656 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-1115d5b02e774b14935e3f48488f82ea received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:01.647145 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-1115d5b02e774b14935e3f48488f82ea on tablet 1115d5b02e774b14935e3f48488f82ea with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:01.652917 28892 tablet_copy_client.cc:806] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:01.653913 28892 tablet_copy_client.cc:670] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:01.658449 28892 tablet_copy_client.cc:538] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:01.666318 28895 ts_tablet_manager.cc:933] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:01.670470 28895 tablet_copy_client.cc:323] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:01.672247 28892 tablet_bootstrap.cc:492] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:01.672598 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet 60a7d81169af4acc91d8f3b342ac4885 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:01.673239 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet 60a7d81169af4acc91d8f3b342ac4885 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-60a7d81169af4acc91d8f3b342ac4885
I20260709 13:11:01.686724 28096 tablet_copy_source_session.cc:215] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:01.690815 28895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60a7d81169af4acc91d8f3b342ac4885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:01.720260 28892 tablet_bootstrap.cc:492] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:01.721297 28892 tablet_bootstrap.cc:492] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:01.722007 28892 ts_tablet_manager.cc:1403] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.050s	user 0.019s	sys 0.012s
I20260709 13:11:01.725744 28892 raft_consensus.cc:359] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:01.726650 28892 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:01.726900 28895 tablet_copy_client.cc:806] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:01.727494 28895 tablet_copy_client.cc:670] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:01.786243 28895 tablet_copy_client.cc:538] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:01.727468 28892 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:01.828156 28895 tablet_bootstrap.cc:492] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:01.839540 28762 raft_consensus.cc:1217] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:01.865119 28892 ts_tablet_manager.cc:1434] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.140s	user 0.006s	sys 0.005s
I20260709 13:11:01.867595 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-cd81a684ef6b4e0f9cc48bc3bcc280e8 received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:01.868541 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-cd81a684ef6b4e0f9cc48bc3bcc280e8 on tablet cd81a684ef6b4e0f9cc48bc3bcc280e8 with peer 70996712c3c942468a6006602de6477f
W20260709 13:11:01.872377 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 62746.1us
I20260709 13:11:01.974562 28895 tablet_bootstrap.cc:492] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:01.975838 28895 tablet_bootstrap.cc:492] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:01.976922 28895 ts_tablet_manager.cc:1403] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.149s	user 0.026s	sys 0.007s
I20260709 13:11:01.981421 28895 raft_consensus.cc:359] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:01.982869 28895 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:01.983961 28895 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:01.987391 28895 ts_tablet_manager.cc:1434] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:11:01.991575 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-60a7d81169af4acc91d8f3b342ac4885 received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:01.995126 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-60a7d81169af4acc91d8f3b342ac4885 on tablet 60a7d81169af4acc91d8f3b342ac4885 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:02.067130 28762 raft_consensus.cc:1217] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:02.091622 28762 raft_consensus.cc:1217] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:02.189692 28762 raft_consensus.cc:1217] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:02.303490 28910 raft_consensus.cc:1064] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:02.306607 28910 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:02.317178 28762 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.318169 28154 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.319274 28084 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.318917 28910 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:02.320586 28908 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:02.321671 28846 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:02.347074 28846 raft_consensus.cc:2955] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.350692 28153 raft_consensus.cc:2955] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.350675 28084 raft_consensus.cc:2955] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.361481 28762 raft_consensus.cc:2955] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.376832 27982 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.419035 28234 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:02.426700 28153 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:02.426894 28762 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:02.428128 28846 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:02.430310 28894 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:02.440928 28894 raft_consensus.cc:2955] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.443836 28157 raft_consensus.cc:2955] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.453537 28762 raft_consensus.cc:2955] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.455377 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fab37ead976b4378b1298941cec677a8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:02.465430 27984 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:02.504040 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet fab37ead976b4378b1298941cec677a8 with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:02.509403 28920 tablet_replica.cc:333] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:11:02.510744 28920 raft_consensus.cc:2243] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:02.511518 28920 raft_consensus.cc:2272] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:02.515506 28920 ts_tablet_manager.cc:1916] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:02.548702 28920 ts_tablet_manager.cc:1929] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:02.549379 28920 log.cc:1199] T fab37ead976b4378b1298941cec677a8 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/fab37ead976b4378b1298941cec677a8
I20260709 13:11:02.551767 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet fab37ead976b4378b1298941cec677a8 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:02.661731 28847 raft_consensus.cc:1064] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:02.664359 28847 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:02.669674 28867 raft_consensus.cc:1064] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:02.673523 28867 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:02.680541 28762 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.683029 28235 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.685103 28230 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.683126 28763 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.687405 28154 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.690928 28867 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:02.691618 28153 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.693133 28907 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:02.695258 28878 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:02.699414 28847 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:02.701402 28847 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:02.718842 28867 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:02.726758 28907 raft_consensus.cc:1064] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:02.730669 28907 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:02.916980 28928 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.204s	user 0.001s	sys 0.001s
W20260709 13:11:02.922031 28927 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 13:11:02.928185 28926 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.132s	user 0.000s	sys 0.002s
I20260709 13:11:02.944057 28763 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.949087 28235 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.951015 28934 raft_consensus.cc:2955] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.951404 28923 raft_consensus.cc:2955] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.954952 28878 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:02.956974 28878 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:02.968525 28154 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:02.980324 28235 raft_consensus.cc:2955] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.979833 28234 raft_consensus.cc:2955] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.997054 28867 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:03.005479 28157 raft_consensus.cc:2955] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:02.997917 28764 raft_consensus.cc:2955] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.003300 28154 raft_consensus.cc:2955] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.042806 27984 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:03.080600 28935 raft_consensus.cc:2955] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.090996 28160 raft_consensus.cc:2955] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.140753 28232 raft_consensus.cc:2955] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.140800 27981 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.161917 28763 raft_consensus.cc:2955] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.177800 28762 raft_consensus.cc:2955] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.223498 27983 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.243057 28081 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:03.245929 28082 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:03.252849 28762 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:03.255872 28232 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:03.255652 28081 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:03.258265 28878 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:03.260270 28935 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:03.261611 28157 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:03.264053 28157 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:03.264209 28764 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:03.265424 28934 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:03.266105 28878 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:03.268317 28935 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:03.269536 28907 raft_consensus.cc:2955] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.275509 28941 raft_consensus.cc:2955] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.286387 28867 raft_consensus.cc:2955] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.289186 28763 raft_consensus.cc:2955] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.299484 28764 raft_consensus.cc:2955] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.301455 28154 raft_consensus.cc:2955] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.306955 28155 raft_consensus.cc:2955] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.274207 28762 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:03.309026 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cd81a684ef6b4e0f9cc48bc3bcc280e8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:03.313271 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1115d5b02e774b14935e3f48488f82ea with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:03.331180 27984 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:03.330104 28934 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:11:03.345697 27982 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.348556 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 60a7d81169af4acc91d8f3b342ac4885 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:03.353046 28232 raft_consensus.cc:2955] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.364140 28216 tablet_service.cc:1558] Processing DeleteTablet for tablet cd81a684ef6b4e0f9cc48bc3bcc280e8 with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
I20260709 13:11:03.366982 28764 raft_consensus.cc:2955] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.371377 27983 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:03.386026 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet 60a7d81169af4acc91d8f3b342ac4885 with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50454
I20260709 13:11:03.397697 28953 tablet_replica.cc:333] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:11:03.399019 28953 raft_consensus.cc:2243] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.565584 28953 raft_consensus.cc:2272] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.571120 28953 ts_tablet_manager.cc:1916] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:03.605582 28952 tablet_replica.cc:333] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:11:03.628439 28952 raft_consensus.cc:2243] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.629563 28952 raft_consensus.cc:2272] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.664471 28216 tablet_service.cc:1558] Processing DeleteTablet for tablet 1115d5b02e774b14935e3f48488f82ea with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
I20260709 13:11:03.671446 28952 ts_tablet_manager.cc:1916] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:03.682677 28953 ts_tablet_manager.cc:1929] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:03.683372 28953 log.cc:1199] T 60a7d81169af4acc91d8f3b342ac4885 P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/60a7d81169af4acc91d8f3b342ac4885
I20260709 13:11:03.685189 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet 60a7d81169af4acc91d8f3b342ac4885 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:03.700870 28952 ts_tablet_manager.cc:1929] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:03.701486 28952 log.cc:1199] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/cd81a684ef6b4e0f9cc48bc3bcc280e8
I20260709 13:11:03.703384 28952 tablet_replica.cc:333] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:11:03.704510 28952 raft_consensus.cc:2243] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.705307 28952 raft_consensus.cc:2272] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.709064 28952 ts_tablet_manager.cc:1916] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:03.712580 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet cd81a684ef6b4e0f9cc48bc3bcc280e8 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:03.725476 28952 ts_tablet_manager.cc:1929] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:03.725975 28952 log.cc:1199] T 1115d5b02e774b14935e3f48488f82ea P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/1115d5b02e774b14935e3f48488f82ea
I20260709 13:11:03.728587 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet 1115d5b02e774b14935e3f48488f82ea (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:04.669934 28157 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:04.677220 28082 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:04.678176 28232 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:04.678833 28959 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:04.680205 28856 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:04.681901 28124 consensus_peers.cc:597] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 552d81eee50848758aae23da5ed8927b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.693362 28856 raft_consensus.cc:2955] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.698199 28082 raft_consensus.cc:2955] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.698202 28232 raft_consensus.cc:2955] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.713177 28086 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:04.722980 27982 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:04.758090 28232 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:04.760120 28049 consensus_peers.cc:597] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e09b3a74da19442187d6c447c95018a9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.760963 28957 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:04.769524 28155 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:04.771520 28847 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:04.812435 28965 raft_consensus.cc:2955] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.823278 28235 raft_consensus.cc:2955] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.838614 28235 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:04.840943 28154 raft_consensus.cc:2955] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.850538 28157 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:04.854851 28198 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f16469a2e44408daf3969b51ae89aa6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.856910 28942 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:04.862957 27984 catalog_manager.cc:5697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:04.865490 28086 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:04.866906 28908 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:04.872805 28942 raft_consensus.cc:2955] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.874904 28154 raft_consensus.cc:2955] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.885244 28081 raft_consensus.cc:2955] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.896085 27984 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 2 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.896027 28157 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:04.933636 28082 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:04.934199 28235 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:04.934302 28124 consensus_peers.cc:597] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d56e9de7a9c54453bc5fc3aac0fa589c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.937288 28959 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:04.938185 28962 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:04.951817 28962 raft_consensus.cc:2955] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.953624 28081 raft_consensus.cc:2955] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.954965 28235 raft_consensus.cc:2955] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:04.970902 27984 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:05.157407 28979 ts_tablet_manager.cc:933] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:05.161854 28979 tablet_copy_client.cc:323] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:05.163755 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet e09b3a74da19442187d6c447c95018a9 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:05.164345 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet e09b3a74da19442187d6c447c95018a9 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-e09b3a74da19442187d6c447c95018a9
I20260709 13:11:05.172304 28096 tablet_copy_source_session.cc:215] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:05.175997 28979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e09b3a74da19442187d6c447c95018a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.200006 28979 tablet_copy_client.cc:806] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:05.200665 28979 tablet_copy_client.cc:670] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:05.205273 28979 tablet_copy_client.cc:538] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:05.213263 28979 tablet_bootstrap.cc:492] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:05.233194 28979 tablet_bootstrap.cc:492] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:05.234226 28979 tablet_bootstrap.cc:492] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:05.234995 28979 ts_tablet_manager.cc:1403] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.009s
I20260709 13:11:05.238212 28979 raft_consensus.cc:359] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:05.239200 28979 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:05.239995 28979 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:05.245574 28979 ts_tablet_manager.cc:1434] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:11:05.247457 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-e09b3a74da19442187d6c447c95018a9 received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:05.247946 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-e09b3a74da19442187d6c447c95018a9 on tablet e09b3a74da19442187d6c447c95018a9 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:05.273260 28979 ts_tablet_manager.cc:933] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:05.280853 28979 tablet_copy_client.cc:323] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:11:05.283171 28171 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet 552d81eee50848758aae23da5ed8927b from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:11:05.283794 28171 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet 552d81eee50848758aae23da5ed8927b from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-552d81eee50848758aae23da5ed8927b
I20260709 13:11:05.292707 28171 tablet_copy_source_session.cc:215] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:05.297299 28979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552d81eee50848758aae23da5ed8927b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.323964 28979 tablet_copy_client.cc:806] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:05.324664 28979 tablet_copy_client.cc:670] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:05.334554 28979 tablet_copy_client.cc:538] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:05.361207 28984 ts_tablet_manager.cc:933] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:05.363523 28979 tablet_bootstrap.cc:492] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:05.363742 28984 tablet_copy_client.cc:323] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:11:05.366158 28244 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet 9f16469a2e44408daf3969b51ae89aa6 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:11:05.366765 28244 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet 9f16469a2e44408daf3969b51ae89aa6 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-9f16469a2e44408daf3969b51ae89aa6
I20260709 13:11:05.375181 28244 tablet_copy_source_session.cc:215] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:05.379972 28984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f16469a2e44408daf3969b51ae89aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.385530 28979 tablet_bootstrap.cc:492] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:05.386430 28979 tablet_bootstrap.cc:492] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:05.387089 28979 ts_tablet_manager.cc:1403] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.004s
I20260709 13:11:05.389827 28979 raft_consensus.cc:359] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:05.390620 28979 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:05.391811 28979 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:05.397872 28984 tablet_copy_client.cc:806] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:05.398514 28984 tablet_copy_client.cc:670] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:05.402588 28984 tablet_copy_client.cc:538] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:05.402590 28979 ts_tablet_manager.cc:1434] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:11:05.406206 28171 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-552d81eee50848758aae23da5ed8927b received from {username='slave'} at 127.0.0.1:43184
I20260709 13:11:05.406740 28171 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-552d81eee50848758aae23da5ed8927b on tablet 552d81eee50848758aae23da5ed8927b with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:05.412719 28984 tablet_bootstrap.cc:492] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:05.435556 28984 tablet_bootstrap.cc:492] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:05.436527 28984 tablet_bootstrap.cc:492] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:05.437238 28984 ts_tablet_manager.cc:1403] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.006s
I20260709 13:11:05.440243 28984 raft_consensus.cc:359] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:05.441172 28984 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:05.441797 28984 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:05.444125 28984 ts_tablet_manager.cc:1434] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:11:05.446126 28244 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-9f16469a2e44408daf3969b51ae89aa6 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:11:05.446559 28244 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-9f16469a2e44408daf3969b51ae89aa6 on tablet 9f16469a2e44408daf3969b51ae89aa6 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:05.552022 28984 ts_tablet_manager.cc:933] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:05.553763 28984 tablet_copy_client.cc:323] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:11:05.561046 28171 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet d56e9de7a9c54453bc5fc3aac0fa589c from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:11:05.561538 28171 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet d56e9de7a9c54453bc5fc3aac0fa589c from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-d56e9de7a9c54453bc5fc3aac0fa589c
I20260709 13:11:05.568022 28171 tablet_copy_source_session.cc:215] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:05.573596 28984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56e9de7a9c54453bc5fc3aac0fa589c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.589047 28984 tablet_copy_client.cc:806] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:05.589622 28984 tablet_copy_client.cc:670] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:05.593195 28984 tablet_copy_client.cc:538] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:05.601603 28984 tablet_bootstrap.cc:492] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:05.623095 28984 tablet_bootstrap.cc:492] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:05.624048 28984 tablet_bootstrap.cc:492] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:05.624763 28984 ts_tablet_manager.cc:1403] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.008s
I20260709 13:11:05.627856 28984 raft_consensus.cc:359] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:05.628722 28984 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:05.629401 28984 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:05.631845 28984 ts_tablet_manager.cc:1434] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:11:05.633968 28171 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-d56e9de7a9c54453bc5fc3aac0fa589c received from {username='slave'} at 127.0.0.1:43184
I20260709 13:11:05.634461 28171 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-d56e9de7a9c54453bc5fc3aac0fa589c on tablet d56e9de7a9c54453bc5fc3aac0fa589c with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:05.655854 28764 raft_consensus.cc:1217] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:05.673837 28764 raft_consensus.cc:1217] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:05.872524 28764 raft_consensus.cc:1217] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:11:05.995582 28764 raft_consensus.cc:1217] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:06.186897 28962 raft_consensus.cc:1064] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:06.189601 28962 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:06.197219 28764 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:06.199618 28235 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:06.199747 28081 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:06.201144 28856 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.207651 28959 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.213900 28962 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.222138 28856 raft_consensus.cc:2955] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.240262 28081 raft_consensus.cc:2955] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.239499 28764 raft_consensus.cc:2955] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.246491 28235 raft_consensus.cc:2955] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.255491 27983 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.287007 28847 raft_consensus.cc:1064] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:06.294067 28847 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:06.311847 28764 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:06.318203 28154 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:06.322268 28231 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:06.330242 28847 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.332957 28965 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.332330 28154 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:06.337651 28847 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.351612 28764 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:06.354584 28235 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:06.356946 28856 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:06.357988 28962 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:06.367908 28998 raft_consensus.cc:2955] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.385047 28762 raft_consensus.cc:2955] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.389640 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 552d81eee50848758aae23da5ed8927b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:06.402451 28988 raft_consensus.cc:2955] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.416396 28991 raft_consensus.cc:1064] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:06.415627 28763 raft_consensus.cc:2955] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.417448 28231 raft_consensus.cc:2955] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.428341 27981 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:06.419343 28991 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:06.421965 27984 catalog_manager.cc:5697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:06.443217 28235 raft_consensus.cc:2955] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.450399 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet 552d81eee50848758aae23da5ed8927b with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:06.460330 28155 raft_consensus.cc:2955] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.462117 29008 tablet_replica.cc:333] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:11:06.463728 29008 raft_consensus.cc:2243] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.464442 29008 raft_consensus.cc:2272] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.471884 29008 ts_tablet_manager.cc:1916] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:06.508551 29008 ts_tablet_manager.cc:1929] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:06.509030 29008 log.cc:1199] T 552d81eee50848758aae23da5ed8927b P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/552d81eee50848758aae23da5ed8927b
I20260709 13:11:06.510586 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet 552d81eee50848758aae23da5ed8927b (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:06.511852 28763 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:06.512036 28155 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:06.512918 28081 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:06.514600 28082 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:06.514555 28991 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:06.517520 28991 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.521359 28992 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.531865 28762 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:06.533879 28988 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:06.535971 28991 raft_consensus.cc:2955] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.541926 28155 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:06.543155 28081 raft_consensus.cc:2955] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.547439 28762 raft_consensus.cc:2955] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.551192 28988 raft_consensus.cc:2955] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.562624 28763 raft_consensus.cc:2955] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.564597 28155 raft_consensus.cc:2955] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.568142 28965 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:06.567629 27983 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:06.580549 28154 raft_consensus.cc:2955] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.610767 28231 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:06.615295 28998 raft_consensus.cc:1064] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:06.619704 27982 catalog_manager.cc:5697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.625488 28762 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:11:06.626922 28975 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:06.620600 28998 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:06.635901 28936 raft_consensus.cc:2955] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.640422 28216 tablet_service.cc:1558] Processing DeleteTablet for tablet e09b3a74da19442187d6c447c95018a9 with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
I20260709 13:11:06.644698 28082 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:11:06.647981 28763 raft_consensus.cc:2955] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.649039 28991 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:06.650646 29015 tablet_replica.cc:333] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:11:06.682567 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9f16469a2e44408daf3969b51ae89aa6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:06.687363 28082 raft_consensus.cc:2955] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.694870 29015 raft_consensus.cc:2243] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.695741 29015 raft_consensus.cc:2272] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.715058 29015 ts_tablet_manager.cc:1916] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:06.738027 28082 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:06.730957 28763 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:06.733572 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e09b3a74da19442187d6c447c95018a9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:06.737084 28235 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:06.741036 28997 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.719682 27983 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 2 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:06.746069 28962 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.748085 28962 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:06.758961 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet 9f16469a2e44408daf3969b51ae89aa6 with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50454
I20260709 13:11:06.770334 29015 ts_tablet_manager.cc:1929] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:06.770948 29015 log.cc:1199] T e09b3a74da19442187d6c447c95018a9 P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/e09b3a74da19442187d6c447c95018a9
I20260709 13:11:06.776401 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet e09b3a74da19442187d6c447c95018a9 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:06.796335 28856 raft_consensus.cc:2955] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.799458 28763 raft_consensus.cc:2955] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.811738 29023 tablet_replica.cc:333] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:11:06.811664 28235 raft_consensus.cc:2955] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.813019 29023 raft_consensus.cc:2243] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.817411 29023 raft_consensus.cc:2272] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.820637 29023 ts_tablet_manager.cc:1916] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:06.825196 28086 raft_consensus.cc:2955] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.828153 27983 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.855310 29023 ts_tablet_manager.cc:1929] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:11:06.855904 29023 log.cc:1199] T 9f16469a2e44408daf3969b51ae89aa6 P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/9f16469a2e44408daf3969b51ae89aa6
I20260709 13:11:06.858146 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet 9f16469a2e44408daf3969b51ae89aa6 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:06.880926 28160 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:06.885669 28763 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:06.887387 28231 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:06.887225 28962 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:06.888821 28856 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:06.894966 28962 raft_consensus.cc:2955] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.897022 28231 raft_consensus.cc:2955] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.903069 28763 raft_consensus.cc:2955] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.905367 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d56e9de7a9c54453bc5fc3aac0fa589c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:06.915225 27984 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:06.925115 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet d56e9de7a9c54453bc5fc3aac0fa589c with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:06.926061 29008 tablet_replica.cc:333] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:11:06.927063 29008 raft_consensus.cc:2243] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:06.927799 29008 raft_consensus.cc:2272] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:06.931156 29008 ts_tablet_manager.cc:1916] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:06.946720 29008 ts_tablet_manager.cc:1929] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:06.947113 29008 log.cc:1199] T d56e9de7a9c54453bc5fc3aac0fa589c P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/d56e9de7a9c54453bc5fc3aac0fa589c
I20260709 13:11:06.948832 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet d56e9de7a9c54453bc5fc3aac0fa589c (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:08.003999 28082 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:11:08.013959 28049 consensus_peers.cc:597] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6810464df4a245f49df908fc877020d7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:08.015314 28154 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:08.013998 28235 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:08.017024 29026 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:08.017889 29025 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:08.027170 28988 raft_consensus.cc:2955] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.028704 28235 raft_consensus.cc:2955] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.028903 28154 raft_consensus.cc:2955] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.047533 27982 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.049598 28232 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.092635 28160 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:08.097153 28082 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:08.098645 28198 consensus_peers.cc:597] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d0ccc53d55d4d73baf657119bf89f1c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:08.099330 29031 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:08.101333 29035 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:08.111328 28992 raft_consensus.cc:2955] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.112951 28082 raft_consensus.cc:2955] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.114789 28157 raft_consensus.cc:2955] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.132001 27982 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.135010 28082 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.143730 28232 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:08.146675 28157 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:08.147064 28049 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8d07cbc383b4089b326bc02e8b33622. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:08.148900 29019 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:08.149979 28988 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:08.160506 28988 raft_consensus.cc:2955] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.161895 28232 raft_consensus.cc:2955] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.162643 28154 raft_consensus.cc:2955] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.177798 28082 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.181428 27984 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.191241 28232 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:08.193817 28988 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:08.194895 28157 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:08.197942 28049 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aa289960fe9a4d2c8e61deb822de8412. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:08.199021 29019 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:08.204283 29026 raft_consensus.cc:2955] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.211244 28232 raft_consensus.cc:2955] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.218714 28157 raft_consensus.cc:2955] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:08.222394 27981 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.565022 29045 ts_tablet_manager.cc:933] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:08.567765 29045 tablet_copy_client.cc:323] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:08.570266 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet d8d07cbc383b4089b326bc02e8b33622 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:08.570890 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet d8d07cbc383b4089b326bc02e8b33622 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-d8d07cbc383b4089b326bc02e8b33622
I20260709 13:11:08.579324 28096 tablet_copy_source_session.cc:215] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:08.592010 29047 ts_tablet_manager.cc:933] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:08.592574 29045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8d07cbc383b4089b326bc02e8b33622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:08.604209 29047 tablet_copy_client.cc:323] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:11:08.609892 28244 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet 2d0ccc53d55d4d73baf657119bf89f1c from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:11:08.610534 28244 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet 2d0ccc53d55d4d73baf657119bf89f1c from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-2d0ccc53d55d4d73baf657119bf89f1c
I20260709 13:11:08.617549 29049 ts_tablet_manager.cc:933] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:08.619835 28244 tablet_copy_source_session.cc:215] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:08.623068 29049 tablet_copy_client.cc:323] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:08.626130 29047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d0ccc53d55d4d73baf657119bf89f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:08.626451 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet 6810464df4a245f49df908fc877020d7 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:08.627066 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet 6810464df4a245f49df908fc877020d7 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-6810464df4a245f49df908fc877020d7
I20260709 13:11:08.630084 29045 tablet_copy_client.cc:806] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:08.630816 29045 tablet_copy_client.cc:670] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:08.640821 28096 tablet_copy_source_session.cc:215] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:08.637145 29045 tablet_copy_client.cc:538] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:08.644538 29049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6810464df4a245f49df908fc877020d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:08.659394 29045 tablet_bootstrap.cc:492] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:08.660115 29047 tablet_copy_client.cc:806] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:08.660847 29047 tablet_copy_client.cc:670] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:08.672587 29047 tablet_copy_client.cc:538] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:08.682667 29047 tablet_bootstrap.cc:492] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:08.691633 29049 tablet_copy_client.cc:806] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:08.692651 29049 tablet_copy_client.cc:670] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:08.699703 29049 tablet_copy_client.cc:538] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:08.720150 29049 tablet_bootstrap.cc:492] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:08.720615 29045 tablet_bootstrap.cc:492] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:08.721735 29045 tablet_bootstrap.cc:492] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:08.722512 29045 ts_tablet_manager.cc:1403] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.063s	user 0.026s	sys 0.027s
I20260709 13:11:08.731089 29045 raft_consensus.cc:359] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.733613 29045 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:08.734319 29045 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.735219 29047 tablet_bootstrap.cc:492] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:08.736227 29047 tablet_bootstrap.cc:492] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:08.742972 29045 ts_tablet_manager.cc:1434] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.007s
I20260709 13:11:08.744937 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-d8d07cbc383b4089b326bc02e8b33622 received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:08.745285 29047 ts_tablet_manager.cc:1403] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.063s	user 0.021s	sys 0.004s
I20260709 13:11:08.745430 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-d8d07cbc383b4089b326bc02e8b33622 on tablet d8d07cbc383b4089b326bc02e8b33622 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:08.748582 29047 raft_consensus.cc:359] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.749657 29047 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:08.750382 29047 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.753384 29047 ts_tablet_manager.cc:1434] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:11:08.755528 28244 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-2d0ccc53d55d4d73baf657119bf89f1c received from {username='slave'} at 127.0.0.1:44070
I20260709 13:11:08.756009 28244 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-2d0ccc53d55d4d73baf657119bf89f1c on tablet 2d0ccc53d55d4d73baf657119bf89f1c with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:08.758370 29049 tablet_bootstrap.cc:492] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:08.758816 29047 ts_tablet_manager.cc:933] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:08.759680 29049 tablet_bootstrap.cc:492] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:08.760428 29049 ts_tablet_manager.cc:1403] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.041s	user 0.024s	sys 0.001s
I20260709 13:11:08.765856 29049 raft_consensus.cc:359] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.766793 29049 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:08.766966 29047 tablet_copy_client.cc:323] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:08.767552 29049 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.768908 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet aa289960fe9a4d2c8e61deb822de8412 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:08.769510 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet aa289960fe9a4d2c8e61deb822de8412 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-aa289960fe9a4d2c8e61deb822de8412
I20260709 13:11:08.771157 29049 ts_tablet_manager.cc:1434] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:11:08.773842 28095 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-6810464df4a245f49df908fc877020d7 received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:08.774279 28095 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-6810464df4a245f49df908fc877020d7 on tablet 6810464df4a245f49df908fc877020d7 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:08.783923 28096 tablet_copy_source_session.cc:215] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:08.790287 29047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa289960fe9a4d2c8e61deb822de8412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:08.814095 29047 tablet_copy_client.cc:806] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:08.814743 29047 tablet_copy_client.cc:670] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:08.818933 29047 tablet_copy_client.cc:538] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:08.830396 29047 tablet_bootstrap.cc:492] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:08.854336 29047 tablet_bootstrap.cc:492] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:08.856271 29047 tablet_bootstrap.cc:492] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:08.857079 29047 ts_tablet_manager.cc:1403] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.027s	user 0.021s	sys 0.005s
I20260709 13:11:08.861754 29047 raft_consensus.cc:359] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.862602 29047 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:08.863219 29047 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:08.866200 29047 ts_tablet_manager.cc:1434] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:11:08.868397 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-aa289960fe9a4d2c8e61deb822de8412 received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:08.868917 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-aa289960fe9a4d2c8e61deb822de8412 on tablet aa289960fe9a4d2c8e61deb822de8412 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:08.971894 28763 raft_consensus.cc:1217] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:09.034832 28763 raft_consensus.cc:1217] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:09.070581 28762 raft_consensus.cc:1217] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:09.197829 28762 raft_consensus.cc:1217] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:09.422183 28992 raft_consensus.cc:1064] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:09.424549 28992 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:09.433686 28763 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.434024 28154 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.435523 28991 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.438344 28086 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.440095 28991 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.441936 29031 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.463389 29031 raft_consensus.cc:2955] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.465616 28086 raft_consensus.cc:2955] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.478724 28762 raft_consensus.cc:2955] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.485987 29026 raft_consensus.cc:1064] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:09.478900 28154 raft_consensus.cc:2955] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.488710 29026 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:09.496417 27984 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:09.504892 28763 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.507086 28157 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.508574 29058 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.512107 29026 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.523432 28232 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.525997 29057 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:09.548286 29057 raft_consensus.cc:2955] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.560554 28763 raft_consensus.cc:2955] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.562886 28235 raft_consensus.cc:2955] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.585808 28154 raft_consensus.cc:2955] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.593945 28235 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:09.596930 29026 raft_consensus.cc:1064] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:09.600790 28763 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:09.601341 29026 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:09.603664 28991 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:09.598767 27983 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:09.610397 28154 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:09.631516 28991 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:09.637037 29031 raft_consensus.cc:2955] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.646371 28082 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:09.648941 28763 raft_consensus.cc:2955] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.654597 28154 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.655126 28762 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:09.656523 28235 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.665616 28764 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.676103 29026 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.677767 28988 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.679591 28235 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:09.679807 29058 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:09.680401 28988 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.696368 28157 raft_consensus.cc:2955] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.700752 29026 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:09.707279 29057 raft_consensus.cc:2955] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.711467 28988 raft_consensus.cc:2955] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.714147 28232 raft_consensus.cc:2955] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.717876 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6810464df4a245f49df908fc877020d7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:09.727893 28764 raft_consensus.cc:2955] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.729139 27983 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.728421 27984 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:09.737687 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2d0ccc53d55d4d73baf657119bf89f1c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:09.740221 28762 raft_consensus.cc:2955] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.748129 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet 6810464df4a245f49df908fc877020d7 with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50454
I20260709 13:11:09.759475 28235 raft_consensus.cc:2955] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.762789 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d0ccc53d55d4d73baf657119bf89f1c with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:09.772517 28160 raft_consensus.cc:2955] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.772927 27982 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:09.775841 29058 raft_consensus.cc:1064] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:09.789238 28082 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:09.792351 29078 tablet_replica.cc:333] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:11:09.795271 29078 raft_consensus.cc:2243] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:09.796058 29078 raft_consensus.cc:2272] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:09.798128 28764 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:09.798830 28160 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:09.809257 29079 tablet_replica.cc:333] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:11:09.810499 29079 raft_consensus.cc:2243] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:09.810724 29078 ts_tablet_manager.cc:1916] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:09.812314 28988 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:09.817410 28988 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:09.819980 29064 raft_consensus.cc:2955] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.823334 29058 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:09.827088 28764 raft_consensus.cc:2955] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.828349 28157 raft_consensus.cc:2955] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.831916 29078 ts_tablet_manager.cc:1929] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:09.833681 29078 log.cc:1199] T 6810464df4a245f49df908fc877020d7 P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/6810464df4a245f49df908fc877020d7
I20260709 13:11:09.835322 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet 6810464df4a245f49df908fc877020d7 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:09.832844 29079 raft_consensus.cc:2272] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:09.843799 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d8d07cbc383b4089b326bc02e8b33622 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:09.843783 29079 ts_tablet_manager.cc:1916] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:09.845633 27984 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.860580 28216 tablet_service.cc:1558] Processing DeleteTablet for tablet d8d07cbc383b4089b326bc02e8b33622 with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
I20260709 13:11:09.871168 29079 ts_tablet_manager.cc:1929] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:09.871790 29079 log.cc:1199] T 2d0ccc53d55d4d73baf657119bf89f1c P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/2d0ccc53d55d4d73baf657119bf89f1c
I20260709 13:11:09.873879 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet 2d0ccc53d55d4d73baf657119bf89f1c (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:09.876364 28764 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.876609 29081 tablet_replica.cc:333] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:11:09.876801 28157 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.877089 28235 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:09.878098 29058 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.879066 29026 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:09.877718 29081 raft_consensus.cc:2243] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:09.882680 28988 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:09.883359 29081 raft_consensus.cc:2272] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:09.889192 29081 ts_tablet_manager.cc:1916] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:09.897148 29064 raft_consensus.cc:2955] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.901647 28764 raft_consensus.cc:2955] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.915977 27984 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.922647 28235 raft_consensus.cc:2955] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.931139 28157 raft_consensus.cc:2955] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.946273 29081 ts_tablet_manager.cc:1929] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:09.946408 28082 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:09.947204 29081 log.cc:1199] T d8d07cbc383b4089b326bc02e8b33622 P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/d8d07cbc383b4089b326bc02e8b33622
I20260709 13:11:09.952025 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet d8d07cbc383b4089b326bc02e8b33622 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:09.957219 28160 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:09.959075 29064 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:11:09.958810 28764 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:09.971560 29064 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:09.973040 29058 raft_consensus.cc:2955] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.979933 28160 raft_consensus.cc:2955] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.984407 28762 raft_consensus.cc:2955] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:09.987932 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet aa289960fe9a4d2c8e61deb822de8412 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:10.001449 27983 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:10.013114 28216 tablet_service.cc:1558] Processing DeleteTablet for tablet aa289960fe9a4d2c8e61deb822de8412 with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
I20260709 13:11:10.014148 29081 tablet_replica.cc:333] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:11:10.015259 29081 raft_consensus.cc:2243] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:10.015964 29081 raft_consensus.cc:2272] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:10.020385 29081 ts_tablet_manager.cc:1916] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:10.037037 29081 ts_tablet_manager.cc:1929] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:10.037671 29081 log.cc:1199] T aa289960fe9a4d2c8e61deb822de8412 P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/aa289960fe9a4d2c8e61deb822de8412
I20260709 13:11:10.039085 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet aa289960fe9a4d2c8e61deb822de8412 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:11.102032 28235 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.109087 28160 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:11.109959 28198 consensus_peers.cc:597] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c20222bc64224781bd97e8b75f2ddfa1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:11.110561 28081 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:11.110623 29086 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:11.112054 29059 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:11.120707 28992 raft_consensus.cc:2955] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.122911 28160 raft_consensus.cc:2955] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.123538 28081 raft_consensus.cc:2955] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.140170 27984 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.153028 28235 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.167467 28157 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:11.169832 29086 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:11.170375 28081 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:11.172816 28198 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd7077e3c7d94184913764cd90b92757. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:11.173823 29086 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:11.186545 29086 raft_consensus.cc:2955] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.194221 28086 raft_consensus.cc:2955] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.197794 28154 raft_consensus.cc:2955] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.205420 28235 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.213058 27983 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:11.220620 28154 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:11.221436 28198 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d952e9d07d9e4c5ba47d0f8a9601dc73. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:11.223058 29086 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:11.223623 28086 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:11.230446 29086 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:11.233522 28992 raft_consensus.cc:2955] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.235147 28154 raft_consensus.cc:2955] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.251775 28235 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.253167 27982 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.254347 28086 raft_consensus.cc:2955] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.263679 28081 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:11.265470 28157 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:11.265770 28198 consensus_peers.cc:597] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9145e2ecf82c4a03b0034029f10bab74. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:11.269701 29099 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:11.277297 29099 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:11.316169 28086 raft_consensus.cc:2955] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.303843 29086 raft_consensus.cc:2955] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.354209 27984 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.354853 28157 raft_consensus.cc:2955] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:11.495743 29107 ts_tablet_manager.cc:933] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:11.497985 29107 tablet_copy_client.cc:323] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:11:11.499799 28244 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet c20222bc64224781bd97e8b75f2ddfa1 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:11:11.500341 28244 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet c20222bc64224781bd97e8b75f2ddfa1 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-c20222bc64224781bd97e8b75f2ddfa1
I20260709 13:11:11.509584 28244 tablet_copy_source_session.cc:215] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:11.513092 29107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20222bc64224781bd97e8b75f2ddfa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:11.527740 29107 tablet_copy_client.cc:806] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:11.528375 29107 tablet_copy_client.cc:670] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:11.532163 29107 tablet_copy_client.cc:538] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:11.540683 29107 tablet_bootstrap.cc:492] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:11.564630 29107 tablet_bootstrap.cc:492] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:11.565610 29107 tablet_bootstrap.cc:492] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:11.566354 29107 ts_tablet_manager.cc:1403] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.011s
I20260709 13:11:11.569512 29107 raft_consensus.cc:359] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.570452 29107 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:11.571165 29107 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.577438 29107 ts_tablet_manager.cc:1434] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:11:11.580014 28244 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-c20222bc64224781bd97e8b75f2ddfa1 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:11:11.580433 28244 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-c20222bc64224781bd97e8b75f2ddfa1 on tablet c20222bc64224781bd97e8b75f2ddfa1 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:11.712965 29107 ts_tablet_manager.cc:933] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:11.716092 29107 tablet_copy_client.cc:323] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:11:11.717842 28244 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet dd7077e3c7d94184913764cd90b92757 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:11:11.718390 28244 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet dd7077e3c7d94184913764cd90b92757 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-dd7077e3c7d94184913764cd90b92757
I20260709 13:11:11.722365 29113 ts_tablet_manager.cc:933] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:11.724443 29113 tablet_copy_client.cc:323] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:11:11.726320 28245 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet d952e9d07d9e4c5ba47d0f8a9601dc73 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:11:11.726346 28244 tablet_copy_source_session.cc:215] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:11.726950 28245 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet d952e9d07d9e4c5ba47d0f8a9601dc73 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-d952e9d07d9e4c5ba47d0f8a9601dc73
I20260709 13:11:11.730695 29107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd7077e3c7d94184913764cd90b92757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:11.734464 28245 tablet_copy_source_session.cc:215] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:11.738278 29113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d952e9d07d9e4c5ba47d0f8a9601dc73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:11.750885 29107 tablet_copy_client.cc:806] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:11.751503 29107 tablet_copy_client.cc:670] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:11.756322 29107 tablet_copy_client.cc:538] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:11.758212 29113 tablet_copy_client.cc:806] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:11.758894 29113 tablet_copy_client.cc:670] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:11.762784 29113 tablet_copy_client.cc:538] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:11.765790 29107 tablet_bootstrap.cc:492] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:11.772549 29113 tablet_bootstrap.cc:492] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:11.795991 29107 tablet_bootstrap.cc:492] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:11.797017 29107 tablet_bootstrap.cc:492] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:11.797721 29107 ts_tablet_manager.cc:1403] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.032s	user 0.024s	sys 0.005s
I20260709 13:11:11.800477 29113 tablet_bootstrap.cc:492] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:11.800812 29107 raft_consensus.cc:359] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.801453 29113 tablet_bootstrap.cc:492] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:11.801719 29107 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:11.802255 29113 ts_tablet_manager.cc:1403] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.012s
I20260709 13:11:11.802408 29107 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.805651 29107 ts_tablet_manager.cc:1434] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:11:11.809864 28245 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-dd7077e3c7d94184913764cd90b92757 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:11:11.810318 28245 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-dd7077e3c7d94184913764cd90b92757 on tablet dd7077e3c7d94184913764cd90b92757 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:11.813377 29113 raft_consensus.cc:359] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.817346 29113 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:11.818068 29113 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.820559 29113 ts_tablet_manager.cc:1434] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.004s
I20260709 13:11:11.823465 28245 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-d952e9d07d9e4c5ba47d0f8a9601dc73 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:11:11.823863 28245 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-d952e9d07d9e4c5ba47d0f8a9601dc73 on tablet d952e9d07d9e4c5ba47d0f8a9601dc73 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:11.827630 29107 ts_tablet_manager.cc:933] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:11.832000 29107 tablet_copy_client.cc:323] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:11:11.833709 28245 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet 9145e2ecf82c4a03b0034029f10bab74 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:11:11.834244 28245 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet 9145e2ecf82c4a03b0034029f10bab74 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-9145e2ecf82c4a03b0034029f10bab74
I20260709 13:11:11.842337 28245 tablet_copy_source_session.cc:215] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:11.845458 29107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9145e2ecf82c4a03b0034029f10bab74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:11.865315 29107 tablet_copy_client.cc:806] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:11.865949 29107 tablet_copy_client.cc:670] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:11.870229 29107 tablet_copy_client.cc:538] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:11.879433 29107 tablet_bootstrap.cc:492] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:11.899022 29107 tablet_bootstrap.cc:492] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:11.939821 29107 tablet_bootstrap.cc:492] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:11.942251 29107 ts_tablet_manager.cc:1403] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.005s
I20260709 13:11:11.948596 29107 raft_consensus.cc:359] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:11.961882 28764 raft_consensus.cc:1217] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:11.952998 29107 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:11.980687 29107 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:12.001194 29107 ts_tablet_manager.cc:1434] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.006s
I20260709 13:11:12.006546 28245 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-9145e2ecf82c4a03b0034029f10bab74 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:11:12.007339 28245 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-9145e2ecf82c4a03b0034029f10bab74 on tablet 9145e2ecf82c4a03b0034029f10bab74 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:12.185097 28764 raft_consensus.cc:1217] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:12.241755 28764 raft_consensus.cc:1217] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:12.430863 28764 raft_consensus.cc:1217] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:12.563105 28992 raft_consensus.cc:1064] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:12.565943 28992 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:12.580014 28764 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:12.581936 28082 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:12.582623 28160 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:12.583089 29099 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:12.586946 29086 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:12.589118 29086 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:12.636644 29121 raft_consensus.cc:2955] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.645503 28160 raft_consensus.cc:2955] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.652748 28086 raft_consensus.cc:2955] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.664209 29099 raft_consensus.cc:1064] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:12.668051 29099 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:12.671116 27984 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.688266 28764 raft_consensus.cc:2955] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.707835 28761 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:12.709316 28157 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:12.713585 29086 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:12.716506 29059 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:12.720549 28082 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:12.724180 29086 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:12.744639 29059 raft_consensus.cc:2955] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.746799 28157 raft_consensus.cc:2955] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.747517 28086 raft_consensus.cc:2955] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.758699 28764 raft_consensus.cc:2955] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.770565 27981 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:12.794363 29121 raft_consensus.cc:1064] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:12.797516 29121 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:12.799919 28232 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:12.800063 28235 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:12.810405 28157 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:12.812604 29086 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:12.812971 28762 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:12.817543 28761 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:12.818640 28992 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:12.822646 29136 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:12.816870 28160 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:12.828238 28764 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:12.828027 29136 raft_consensus.cc:2955] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.835821 29059 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:12.833283 28992 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:11:12.816975 28082 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:12.829661 28761 raft_consensus.cc:2955] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.857952 29086 raft_consensus.cc:2955] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.861460 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c20222bc64224781bd97e8b75f2ddfa1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:12.861970 29136 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:12.869478 28081 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:12.871804 28160 raft_consensus.cc:2955] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.871351 28762 raft_consensus.cc:2955] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.888649 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dd7077e3c7d94184913764cd90b92757 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:12.888931 29059 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:12.906853 27981 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:12.908759 27981 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:12.911578 28992 raft_consensus.cc:1064] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:12.914322 28992 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:12.919241 28082 raft_consensus.cc:2955] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:11:12.923723 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 78868.1us
I20260709 13:11:12.937047 29059 raft_consensus.cc:2955] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.941143 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet dd7077e3c7d94184913764cd90b92757 with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50454
I20260709 13:11:12.954579 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet c20222bc64224781bd97e8b75f2ddfa1 with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:12.956501 28160 raft_consensus.cc:2955] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.977865 29144 tablet_replica.cc:333] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:11:12.977772 28082 raft_consensus.cc:2955] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:12.980559 29145 tablet_replica.cc:333] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:11:12.990650 28762 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:12.992436 29136 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:12.995748 29145 raft_consensus.cc:2243] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:12.996630 29145 raft_consensus.cc:2272] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:13.001152 29145 ts_tablet_manager.cc:1916] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:13.003561 28761 raft_consensus.cc:2955] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.014947 29144 raft_consensus.cc:2243] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:13.016074 29144 raft_consensus.cc:2272] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:12.998445 27981 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:13.020427 29144 ts_tablet_manager.cc:1916] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:13.020447 28157 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:13.033452 28086 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:13.039347 29099 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:13.064465 29099 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:13.084517 29145 ts_tablet_manager.cc:1929] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:13.085322 29145 log.cc:1199] T c20222bc64224781bd97e8b75f2ddfa1 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/c20222bc64224781bd97e8b75f2ddfa1
I20260709 13:11:13.088924 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet c20222bc64224781bd97e8b75f2ddfa1 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:13.091670 28235 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:13.094424 29121 raft_consensus.cc:2955] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.102715 29144 ts_tablet_manager.cc:1929] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:13.103368 29144 log.cc:1199] T dd7077e3c7d94184913764cd90b92757 P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/dd7077e3c7d94184913764cd90b92757
I20260709 13:11:13.105366 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet dd7077e3c7d94184913764cd90b92757 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:13.116652 28157 raft_consensus.cc:2955] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.118216 28761 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:13.119959 29136 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:13.126225 28081 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:13.128036 29146 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:13.137720 29099 raft_consensus.cc:2955] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.138893 28761 raft_consensus.cc:2955] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.140811 27981 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.142023 28762 raft_consensus.cc:2955] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.154768 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d952e9d07d9e4c5ba47d0f8a9601dc73 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:13.162665 28235 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:13.162215 28082 raft_consensus.cc:2955] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.163582 27981 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.169176 28157 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:13.171017 29146 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:13.171435 28081 raft_consensus.cc:2955] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.183399 28762 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:13.185693 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet d952e9d07d9e4c5ba47d0f8a9601dc73 with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50454
I20260709 13:11:13.186672 29144 tablet_replica.cc:333] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:11:13.187577 29144 raft_consensus.cc:2243] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:13.188186 29144 raft_consensus.cc:2272] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:13.192754 29144 ts_tablet_manager.cc:1916] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:13.196259 29136 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:13.198909 29146 raft_consensus.cc:2955] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.202222 28157 raft_consensus.cc:2955] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.207254 28762 raft_consensus.cc:2955] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.226639 27984 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:13.234305 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9145e2ecf82c4a03b0034029f10bab74 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:13.246510 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet 9145e2ecf82c4a03b0034029f10bab74 with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:13.247610 29145 tablet_replica.cc:333] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:11:13.261617 29144 ts_tablet_manager.cc:1929] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:13.262236 29144 log.cc:1199] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/d952e9d07d9e4c5ba47d0f8a9601dc73
I20260709 13:11:13.265007 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet d952e9d07d9e4c5ba47d0f8a9601dc73 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:13.290556 29145 raft_consensus.cc:2243] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:13.291709 29145 raft_consensus.cc:2272] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:13.318470 29145 ts_tablet_manager.cc:1916] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:13.355824 29145 ts_tablet_manager.cc:1929] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:13.356369 29145 log.cc:1199] T 9145e2ecf82c4a03b0034029f10bab74 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/9145e2ecf82c4a03b0034029f10bab74
I20260709 13:11:13.358408 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet 9145e2ecf82c4a03b0034029f10bab74 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
W20260709 13:11:13.504801 27969 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 146404us
I20260709 13:11:14.321061 28081 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:14.327648 28232 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:14.328363 28049 consensus_peers.cc:597] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f703ac7a390a45efae315d629c1dad89. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:14.329308 29120 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:14.333967 28154 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:14.336582 29120 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:14.348446 29120 raft_consensus.cc:2955] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:14.350431 28154 raft_consensus.cc:2955] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:14.350535 28232 raft_consensus.cc:2955] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:14.368142 28081 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:14.371311 27984 catalog_manager.cc:5697] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:11:14.380390 28049 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b9a6ec9a3a1422b8723b26150db560f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:14.382511 28232 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.384204 29152 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.382511 28154 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:14.386649 29152 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.400197 29152 raft_consensus.cc:2955] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:14.402160 28157 raft_consensus.cc:2955] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:14.411427 28232 raft_consensus.cc:2955] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:14.430852 28154 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:14.431807 27981 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:11:14.440673 28124 consensus_peers.cc:597] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0720cab120de4047b423cb89ed85154a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:14.446445 28232 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:14.448597 29123 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.450003 28081 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:14.451438 29123 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.456601 29165 raft_consensus.cc:2955] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:14.459935 28232 raft_consensus.cc:2955] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:14.477613 28154 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:14.477530 27981 catalog_manager.cc:5697] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.479426 28086 raft_consensus.cc:2955] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:11:14.498848 28124 consensus_peers.cc:597] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0a4a73cb5754c40af548cd4519da5b4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:14.499310 28081 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:14.501240 28232 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:14.501133 29123 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:14.502835 29168 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:14.503950 28786 debug-util.cc:398] Leaking SignalData structure 0x7b08002f6060 after lost signal to thread 26962
W20260709 13:11:14.505025 28786 debug-util.cc:398] Leaking SignalData structure 0x7b08002f6740 after lost signal to thread 28018
W20260709 13:11:14.506208 28786 debug-util.cc:398] Leaking SignalData structure 0x7b080032e560 after lost signal to thread 28110
W20260709 13:11:14.507206 28786 debug-util.cc:398] Leaking SignalData structure 0x7b08003059c0 after lost signal to thread 28185
I20260709 13:11:14.514474 29168 raft_consensus.cc:2955] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:11:14.517174 28786 debug-util.cc:398] Leaking SignalData structure 0x7b08001f6d60 after lost signal to thread 28259
W20260709 13:11:14.520197 28786 debug-util.cc:398] Leaking SignalData structure 0x7b08002cb960 after lost signal to thread 28789
I20260709 13:11:14.523011 28086 raft_consensus.cc:2955] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:14.529476 28232 raft_consensus.cc:2955] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:14.533915 27984 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.793249 29172 ts_tablet_manager.cc:933] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:14.802251 29172 tablet_copy_client.cc:323] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:14.804685 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet 6b9a6ec9a3a1422b8723b26150db560f from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:14.805426 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet 6b9a6ec9a3a1422b8723b26150db560f from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-6b9a6ec9a3a1422b8723b26150db560f
I20260709 13:11:14.816406 28096 tablet_copy_source_session.cc:215] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:14.822062 29172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b9a6ec9a3a1422b8723b26150db560f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:14.846427 29172 tablet_copy_client.cc:806] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:14.847195 29172 tablet_copy_client.cc:670] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:14.852341 29172 tablet_copy_client.cc:538] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:14.887774 29172 tablet_bootstrap.cc:492] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:14.909785 29174 ts_tablet_manager.cc:933] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:14.927634 29172 tablet_bootstrap.cc:492] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:14.937716 29172 tablet_bootstrap.cc:492] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:14.939095 29172 ts_tablet_manager.cc:1403] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.052s	user 0.023s	sys 0.006s
I20260709 13:11:14.945089 29172 raft_consensus.cc:359] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:14.950654 29172 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:14.951751 29172 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:14.963025 29172 ts_tablet_manager.cc:1434] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 13:11:14.965797 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-6b9a6ec9a3a1422b8723b26150db560f received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:14.966425 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-6b9a6ec9a3a1422b8723b26150db560f on tablet 6b9a6ec9a3a1422b8723b26150db560f with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:15.002900 29172 ts_tablet_manager.cc:933] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:15.035060 29172 tablet_copy_client.cc:323] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:11:15.038080 28171 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet c0a4a73cb5754c40af548cd4519da5b4 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:11:15.038895 28171 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet c0a4a73cb5754c40af548cd4519da5b4 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-c0a4a73cb5754c40af548cd4519da5b4
I20260709 13:11:15.076854 28171 tablet_copy_source_session.cc:215] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:11:15.077726 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 75346.3us
I20260709 13:11:15.080933 29180 ts_tablet_manager.cc:933] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:15.094779 29180 tablet_copy_client.cc:323] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:11:15.096942 28170 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet 0720cab120de4047b423cb89ed85154a from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:11:15.097775 28170 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet 0720cab120de4047b423cb89ed85154a from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-0720cab120de4047b423cb89ed85154a
I20260709 13:11:15.113883 29174 tablet_copy_client.cc:323] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:15.115737 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet f703ac7a390a45efae315d629c1dad89 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:15.116345 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet f703ac7a390a45efae315d629c1dad89 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-f703ac7a390a45efae315d629c1dad89
I20260709 13:11:15.122210 28170 tablet_copy_source_session.cc:215] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:15.123888 29172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0a4a73cb5754c40af548cd4519da5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:15.127266 29180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0720cab120de4047b423cb89ed85154a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:15.129953 28096 tablet_copy_source_session.cc:215] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:15.134240 29174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f703ac7a390a45efae315d629c1dad89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:15.185037 29180 tablet_copy_client.cc:806] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:15.185745 29180 tablet_copy_client.cc:670] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:15.191084 29180 tablet_copy_client.cc:538] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:15.194307 29174 tablet_copy_client.cc:806] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:15.195012 29174 tablet_copy_client.cc:670] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:15.202006 29174 tablet_copy_client.cc:538] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:15.212564 29174 tablet_bootstrap.cc:492] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:15.222692 29172 tablet_copy_client.cc:806] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:15.223451 29172 tablet_copy_client.cc:670] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:15.230829 29172 tablet_copy_client.cc:538] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:15.246755 29172 tablet_bootstrap.cc:492] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:15.247308 29180 tablet_bootstrap.cc:492] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:15.251621 29174 tablet_bootstrap.cc:492] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:15.252625 29174 tablet_bootstrap.cc:492] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:15.256086 29174 ts_tablet_manager.cc:1403] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.014s
I20260709 13:11:15.259267 29174 raft_consensus.cc:359] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:15.266640 29174 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:15.267357 29174 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:15.272622 29174 ts_tablet_manager.cc:1434] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.005s
I20260709 13:11:15.276499 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-f703ac7a390a45efae315d629c1dad89 received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:15.277071 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-f703ac7a390a45efae315d629c1dad89 on tablet f703ac7a390a45efae315d629c1dad89 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:15.282148 29172 tablet_bootstrap.cc:492] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:15.283241 29172 tablet_bootstrap.cc:492] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:15.283979 29172 ts_tablet_manager.cc:1403] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.011s
I20260709 13:11:15.286707 28761 raft_consensus.cc:1217] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:15.289359 29172 raft_consensus.cc:359] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:15.291457 29172 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:15.292109 29172 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:15.305298 29172 ts_tablet_manager.cc:1434] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:11:15.307158 28170 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-c0a4a73cb5754c40af548cd4519da5b4 received from {username='slave'} at 127.0.0.1:43184
I20260709 13:11:15.307662 28170 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-c0a4a73cb5754c40af548cd4519da5b4 on tablet c0a4a73cb5754c40af548cd4519da5b4 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:15.331002 29180 tablet_bootstrap.cc:492] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:15.333205 29180 tablet_bootstrap.cc:492] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:15.334112 29180 ts_tablet_manager.cc:1403] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.087s	user 0.023s	sys 0.019s
I20260709 13:11:15.337891 29180 raft_consensus.cc:359] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:15.338980 29180 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:15.339928 29180 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:15.342929 29180 ts_tablet_manager.cc:1434] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:11:15.344986 28170 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-0720cab120de4047b423cb89ed85154a received from {username='slave'} at 127.0.0.1:43184
I20260709 13:11:15.345423 28170 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-0720cab120de4047b423cb89ed85154a on tablet 0720cab120de4047b423cb89ed85154a with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:15.375679 28761 raft_consensus.cc:1217] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:11:15.510923 28761 raft_consensus.cc:1217] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:15.682579 28761 raft_consensus.cc:1217] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:15.713354 29152 raft_consensus.cc:1064] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:15.716642 29152 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:15.761803 28160 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:15.762183 28761 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:15.766327 28232 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:15.770323 29064 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:15.773308 29064 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:15.805312 29120 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:15.841601 29188 raft_consensus.cc:2955] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:15.843525 28761 raft_consensus.cc:2955] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:11:15.853652 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 84987.5us
I20260709 13:11:15.889132 28235 raft_consensus.cc:2955] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:15.898465 29064 raft_consensus.cc:1064] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:15.901240 29064 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:15.921785 27984 catalog_manager.cc:5697] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:15.954873 28762 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:15.956992 29120 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:15.964169 28153 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:15.991146 29064 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:11:15.971809 28235 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:16.006852 28160 raft_consensus.cc:2955] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.027070 29168 raft_consensus.cc:1064] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:16.033402 29168 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:16.097122 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 114113us
I20260709 13:11:16.102335 29123 raft_consensus.cc:1064] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:16.113137 29120 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:16.111155 29123 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:16.128376 29188 raft_consensus.cc:2955] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.133241 28762 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:16.135658 28764 raft_consensus.cc:2955] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.144305 29202 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:16.151788 28081 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:16.160627 29168 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:16.173346 28764 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:16.176596 28084 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:16.175560 29123 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:16.179674 28160 raft_consensus.cc:2955] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.185271 29123 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:16.200918 28230 raft_consensus.cc:2955] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.230330 29123 raft_consensus.cc:2955] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.232496 28086 raft_consensus.cc:2955] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.253643 28764 raft_consensus.cc:2955] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.263784 28232 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:16.266534 28232 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:16.267555 29168 raft_consensus.cc:2955] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.276341 29151 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:16.280577 29202 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:16.284005 28761 raft_consensus.cc:2955] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.309706 28086 raft_consensus.cc:2955] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.328667 27983 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.330608 27983 catalog_manager.cc:5697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.338675 28230 raft_consensus.cc:2955] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.342443 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:16.362637 28232 raft_consensus.cc:2955] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.364269 27984 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:16.380719 28160 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:16.411329 28761 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:16.413357 29202 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:16.426550 29168 raft_consensus.cc:2955] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.438184 28764 raft_consensus.cc:2955] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.486931 28081 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:16.488638 29202 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:16.496398 28086 raft_consensus.cc:2955] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.518433 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0720cab120de4047b423cb89ed85154a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:16.533437 28081 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:16.534384 27983 catalog_manager.cc:5697] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:16.540644 28764 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:16.540845 28154 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:16.542783 29188 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:16.545249 29188 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:16.564101 28086 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:16.564450 29189 raft_consensus.cc:2955] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.573803 28232 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:11:16.575762 29064 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:16.577085 28764 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:11:16.581137 29198 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:16.580901 28764 raft_consensus.cc:2955] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.585654 29198 raft_consensus.cc:2955] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.589022 28230 raft_consensus.cc:2955] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.593129 28160 raft_consensus.cc:2955] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.606086 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6b9a6ec9a3a1422b8723b26150db560f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:16.645200 28761 raft_consensus.cc:2955] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.654129 27984 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 3 to 4, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.668951 27982 catalog_manager.cc:5697] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.671352 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f703ac7a390a45efae315d629c1dad89 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:16.676673 28216 tablet_service.cc:1558] Processing DeleteTablet for tablet 0720cab120de4047b423cb89ed85154a with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
W20260709 13:11:16.675981 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54608.2us
I20260709 13:11:16.692660 29219 tablet_replica.cc:333] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:11:16.693696 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b9a6ec9a3a1422b8723b26150db560f with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50454
I20260709 13:11:16.693886 29219 raft_consensus.cc:2243] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:16.702800 29219 raft_consensus.cc:2272] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:16.712810 28216 tablet_service.cc:1558] Processing DeleteTablet for tablet f703ac7a390a45efae315d629c1dad89 with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
I20260709 13:11:16.713501 28230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:16.714197 28230 raft_consensus.cc:606] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:16.715271 29220 tablet_replica.cc:333] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:11:16.716377 29220 raft_consensus.cc:2243] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:16.717669 29219 ts_tablet_manager.cc:1916] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:16.728905 29220 raft_consensus.cc:2272] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:16.733059 29220 ts_tablet_manager.cc:1916] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:16.751559 29219 ts_tablet_manager.cc:1929] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:16.763744 28086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:16.767537 29219 log.cc:1199] T 0720cab120de4047b423cb89ed85154a P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/0720cab120de4047b423cb89ed85154a
I20260709 13:11:16.767622 28086 raft_consensus.cc:606] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:16.769764 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet 0720cab120de4047b423cb89ed85154a (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:16.770929 29219 tablet_replica.cc:333] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:11:16.771530 28081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:16.772135 28081 raft_consensus.cc:606] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:16.772176 29219 raft_consensus.cc:2243] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:16.774983 28081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:16.775125 29219 raft_consensus.cc:2272] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:16.775507 28081 raft_consensus.cc:606] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:16.778455 28081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:16.779062 28081 raft_consensus.cc:606] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:16.782305 29219 ts_tablet_manager.cc:1916] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:16.782763 28081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:16.783392 28081 raft_consensus.cc:606] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:16.787053 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:11:16.787885 29220 ts_tablet_manager.cc:1929] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:11:16.788384 29220 log.cc:1199] T 6b9a6ec9a3a1422b8723b26150db560f P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/6b9a6ec9a3a1422b8723b26150db560f
I20260709 13:11:16.788823 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:16.790330 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet 6b9a6ec9a3a1422b8723b26150db560f (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:16.800191 29219 ts_tablet_manager.cc:1929] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:16.800760 29219 log.cc:1199] T f703ac7a390a45efae315d629c1dad89 P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/f703ac7a390a45efae315d629c1dad89
I20260709 13:11:16.803140 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet f703ac7a390a45efae315d629c1dad89 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:16.807782 29182 raft_consensus.cc:993] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:16.808557 29195 raft_consensus.cc:1081] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:16.810504 28081 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:16.811233 28081 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:16.811589 28081 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.820464 28081 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:16.822590 28081 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:16.823645 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:16.824337 28230 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:16.824636 28230 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:16.824443 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:16.825556 28160 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.825476 28230 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:16.826315 28230 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.839772 28230 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:16.841238 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:16.842496 29120 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:16.842969 29120 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:16.843921 29120 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:16.837049 28160 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:16.863978 27981 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:16.912937 29120 raft_consensus.cc:993] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:16.913407 29120 raft_consensus.cc:1081] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:16.922571 29188 raft_consensus.cc:993] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:16.923132 29064 raft_consensus.cc:1081] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:16.924510 28160 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:50460
I20260709 13:11:16.925148 28160 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:16.925457 28160 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.928668 28761 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:56420
I20260709 13:11:16.929352 28761 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:16.929720 28761 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.942016 28154 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:16.947010 28160 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:16.949044 28160 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:16.952445 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:16.953142 28081 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:16.953585 28081 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:16.954401 28081 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:16.955775 28081 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.960322 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:16.961989 28230 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.963882 28081 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:16.965788 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:16.967051 29151 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:16.967576 29151 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:16.968524 29151 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:16.979813 28235 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:16.981432 28761 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:16.981721 29168 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:11:16.982483 28764 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:16.986716 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:16.988438 28761 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:16.988547 28086 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:16.989087 28086 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:16.991425 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:16.992092 28154 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.990116 28086 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:16.993343 28086 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:16.999713 29168 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:17.003672 29199 raft_consensus.cc:2955] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:17.007577 28235 raft_consensus.cc:2955] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:16.982739 28230 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:17.015290 28154 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:17.026927 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:17.028061 29218 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:17.028599 29218 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:17.029585 29218 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:17.044652 28764 raft_consensus.cc:2955] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:17.059849 27981 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:17.072867 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet c0a4a73cb5754c40af548cd4519da5b4 with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:17.081871 29217 raft_consensus.cc:993] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:17.082553 29198 raft_consensus.cc:1081] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:17.086082 28762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:56420
I20260709 13:11:17.086690 28762 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:17.087076 28762 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:17.099287 28762 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:17.106320 29226 tablet_replica.cc:333] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:11:17.107686 29226 raft_consensus.cc:2243] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:17.108558 29226 raft_consensus.cc:2272] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:17.111531 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:17.112214 28081 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:17.112560 28081 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:17.113363 28081 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:17.113955 28762 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:17.114506 28081 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:17.115677 29226 ts_tablet_manager.cc:1916] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:17.117096 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:17.117728 28232 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:17.108155 28086 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:17.124428 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c0a4a73cb5754c40af548cd4519da5b4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:17.129400 29064 raft_consensus.cc:993] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:17.129910 29064 raft_consensus.cc:1081] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:17.135515 28232 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:17.141587 28157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:50460
I20260709 13:11:17.143565 28157 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:17.143914 28157 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:17.144233 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:17.144766 28081 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:17.145190 29218 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:17.145709 29218 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:17.141587 27981 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:17.146651 29218 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:17.150890 28157 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:17.153839 28157 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:17.161080 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:17.161834 28081 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:17.162168 28081 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:17.163009 28081 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:17.164093 28081 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:17.168534 27983 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:17.176224 27982 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:17.182149 29226 ts_tablet_manager.cc:1929] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:17.182775 29226 log.cc:1199] T c0a4a73cb5754c40af548cd4519da5b4 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/c0a4a73cb5754c40af548cd4519da5b4
I20260709 13:11:17.184496 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet c0a4a73cb5754c40af548cd4519da5b4 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:17.171558 28081 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:17.173487 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:17.185516 28232 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:17.196979 28232 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:17.199416 28125 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:17.207281 29151 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:17.208006 29151 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:17.208992 29151 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:17.235572 27982 catalog_manager.cc:5697] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:17.292712 29120 raft_consensus.cc:993] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:17.293280 29217 raft_consensus.cc:1081] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:17.295001 28762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:56420
I20260709 13:11:17.295823 28762 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:17.296123 28762 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:17.302197 28762 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:17.304044 28762 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:17.305243 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:17.305644 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:17.305912 28081 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:17.306281 28157 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:17.306350 28081 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:17.307276 28081 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:17.308602 28081 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:17.312920 28157 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:17.314450 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:17.315348 29218 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:17.315874 29218 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:17.316879 29218 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:17.317636 28157 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:17.318135 28081 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:17.319334 29217 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:17.346094 27982 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:17.373502 28235 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:17.375953 29188 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:17.457443 28157 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:17.458794 29218 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:17.494202 28235 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:17.496557 29151 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:17.515089 28081 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:17.518520 29218 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:17.577665 28232 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:17.579274 29218 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:17.583225 28081 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:17.586117 29199 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:17.632375 28081 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:17.639685 29245 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:17.683408 28080 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:17.689980 29199 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:17.701584 28157 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:17.710062 29245 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:17.737404 28081 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:17.743448 29177 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:17.788339 28232 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:17.790055 29199 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:17.820312 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:18.200009 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:18.255131 28232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:18.255703 28232 raft_consensus.cc:606] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:18.258221 28232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:18.258785 28232 raft_consensus.cc:606] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:18.297676 28157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:18.302551 28157 raft_consensus.cc:606] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:18.296677 28232 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:18.306615 28235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:18.307125 28235 raft_consensus.cc:606] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:18.309437 28081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:18.310000 28081 raft_consensus.cc:606] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:18.314381 28157 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:18.315661 28081 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:18.316145 28198 consensus_peers.cc:597] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33b72fb1e20942d9a3907c50252930bb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.317883 29238 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:18.318629 28235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fab37ead976b4378b1298941cec677a8"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:18.319968 29182 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:18.324083 28235 raft_consensus.cc:606] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:18.328994 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:11:18.330799 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:18.347451 29195 raft_consensus.cc:2955] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.354812 28081 raft_consensus.cc:2955] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.357609 28157 raft_consensus.cc:2955] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.379797 28161 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:18.384262 27983 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.402271 28235 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:11:18.403146 28124 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 847d15cbd80f44b18e547291eed25617. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.404793 28081 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:18.406190 29151 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:18.407534 29255 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:18.461001 29258 raft_consensus.cc:2955] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.466567 28235 raft_consensus.cc:2955] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.472721 28081 raft_consensus.cc:2955] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.481508 29255 raft_consensus.cc:993] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:18.482034 29255 raft_consensus.cc:1081] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:18.485939 28083 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:41006
I20260709 13:11:18.486577 28083 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:18.486950 28083 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.494234 28231 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:18.495061 27983 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:18.497908 28083 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:18.506764 29238 raft_consensus.cc:993] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:18.507411 29238 raft_consensus.cc:1081] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:18.509258 28762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fab37ead976b4378b1298941cec677a8"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:35896
I20260709 13:11:18.509856 28762 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:18.510234 28762 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.518620 28083 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:18.523136 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:18.523803 28157 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:18.524088 28157 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:11:18.527002 28198 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a732ff5937ea47eaaef6755cac5ad60e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.528134 28161 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:18.527776 28762 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.531870 28762 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:18.534520 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:18.535159 28161 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.535153 28083 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:18.538703 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:18.539381 28235 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.542477 28157 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:18.546613 28157 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.538713 29238 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:11:18.557313 29195 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:18.547401 28235 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:18.561718 28052 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:18.563030 29189 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:18.549523 28161 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:18.570093 29189 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:18.571158 29189 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:18.533883 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:18.573282 28231 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:18.573470 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:18.574584 29246 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:18.575100 29246 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:18.573700 28231 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:18.576016 29246 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.576960 28231 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.578230 28231 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.605445 28231 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:18.607167 29188 raft_consensus.cc:993] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:18.612584 29188 raft_consensus.cc:1081] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:18.615298 28157 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:18.617316 27983 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:18.618494 28762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:56420
I20260709 13:11:18.619098 28762 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:18.619472 28762 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.623044 29242 raft_consensus.cc:2955] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.631677 28157 raft_consensus.cc:2955] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.645905 28082 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:18.649849 28762 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.649709 27983 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:18.656774 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:18.657477 28081 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:18.657864 28081 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:18.662832 29195 raft_consensus.cc:993] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:18.668076 28083 raft_consensus.cc:2955] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.669976 29242 raft_consensus.cc:1081] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:18.673548 28763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:35896
I20260709 13:11:18.674013 29242 raft_consensus.cc:993] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:18.674201 28763 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:18.674599 29182 raft_consensus.cc:1081] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:18.674682 28763 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.658628 28081 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.679750 28081 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.681715 27984 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:18.688647 28763 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.691650 28762 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:18.694063 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:18.694445 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:18.698065 28161 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.698073 28153 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.701905 28081 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:18.703627 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:18.705204 28081 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:18.708124 28081 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:18.706216 28763 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:18.707231 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:18.709403 28231 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:18.709731 28231 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:18.708523 28081 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.706611 29246 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:18.711572 29246 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:18.710572 28231 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.712610 29246 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.713200 28231 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.717306 28153 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:18.719135 28235 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:18.722048 28231 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:18.722229 28161 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:18.725514 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:18.723992 28081 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:18.729988 29245 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:18.730558 29245 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:18.731587 29245 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.725299 29188 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:11:18.740787 28081 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:18.745951 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:18.746560 28161 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:18.749349 28049 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2929b1e07bb74c67863e4c504d8d09a6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:18.755465 29182 raft_consensus.cc:993] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:18.756276 29242 raft_consensus.cc:1081] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:18.757843 28763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:35896
I20260709 13:11:18.758365 28763 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:18.741667 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:18.758713 28763 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.759531 28231 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:18.760381 28231 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:18.763572 28231 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:18.766137 28231 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.777156 28153 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:18.777568 28231 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:18.781452 28763 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.779227 29189 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:11:18.781525 27983 catalog_manager.cc:5697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:18.784097 29120 raft_consensus.cc:2955] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.785461 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:18.786336 29217 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:18.786803 29217 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:18.787636 29217 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:18.789553 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:18.790112 28153 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.790805 28232 raft_consensus.cc:2955] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.801586 28153 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:18.802524 28161 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:18.803460 28763 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:18.804347 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:18.805088 28231 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:18.805405 28231 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:18.806295 28231 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.807585 28231 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:18.826452 28728 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:18.827536 29246 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:18.828118 29246 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:18.829422 29246 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:18.820032 27983 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:18.843371 27981 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 3, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.858143 28231 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:18.899206 28157 raft_consensus.cc:2955] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:18.915643 29284 ts_tablet_manager.cc:933] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:18.917570 29286 ts_tablet_manager.cc:933] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:18.918562 28182 debug-util.cc:398] Leaking SignalData structure 0x7b080009af60 after lost signal to thread 26962
W20260709 13:11:18.919654 28182 debug-util.cc:398] Leaking SignalData structure 0x7b0800149ce0 after lost signal to thread 28018
W20260709 13:11:18.920902 28182 debug-util.cc:398] Leaking SignalData structure 0x7b08002590c0 after lost signal to thread 28110
W20260709 13:11:18.921994 28182 debug-util.cc:398] Leaking SignalData structure 0x7b080031b440 after lost signal to thread 28185
W20260709 13:11:18.923105 28182 debug-util.cc:398] Leaking SignalData structure 0x7b080004e760 after lost signal to thread 28259
W20260709 13:11:18.924237 28182 debug-util.cc:398] Leaking SignalData structure 0x7b080013e440 after lost signal to thread 28789
I20260709 13:11:18.936106 27984 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:18.950999 27983 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:18.962450 28157 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:18.965615 29246 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:18.997746 29288 ts_tablet_manager.cc:933] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:19.000626 28231 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:19.003964 29246 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:11:19.052156 28232 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:19.060913 29217 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:19.113322 29284 tablet_copy_client.cc:323] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:11:19.159390 29286 tablet_copy_client.cc:323] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:11:19.142372 29288 tablet_copy_client.cc:323] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:11:19.194095 28157 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:19.197577 29217 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:19.217324 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 78120.3us
I20260709 13:11:19.222151 29303 ts_tablet_manager.cc:933] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:19.244509 28245 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet 33b72fb1e20942d9a3907c50252930bb from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:11:19.245200 28245 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet 33b72fb1e20942d9a3907c50252930bb from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-33b72fb1e20942d9a3907c50252930bb
I20260709 13:11:19.282311 29303 tablet_copy_client.cc:323] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:19.289644 28244 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet a732ff5937ea47eaaef6755cac5ad60e from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:11:19.290359 28244 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet a732ff5937ea47eaaef6755cac5ad60e from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-a732ff5937ea47eaaef6755cac5ad60e
W20260709 13:11:19.295276 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 154613us
I20260709 13:11:19.298507 28170 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet 847d15cbd80f44b18e547291eed25617 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:11:19.299216 28170 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet 847d15cbd80f44b18e547291eed25617 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-847d15cbd80f44b18e547291eed25617
I20260709 13:11:19.306293 28160 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:19.289849 28245 tablet_copy_source_session.cc:215] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:19.317664 28081 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:19.318408 28244 tablet_copy_source_session.cc:215] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:19.318758 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet 2929b1e07bb74c67863e4c504d8d09a6 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:19.324860 28157 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:19.328047 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet 2929b1e07bb74c67863e4c504d8d09a6 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-2929b1e07bb74c67863e4c504d8d09a6
I20260709 13:11:19.326164 28170 tablet_copy_source_session.cc:215] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:19.328702 29246 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:19.331832 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:19.332253 29299 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:19.333781 29284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847d15cbd80f44b18e547291eed25617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.340776 29299 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:19.342398 28096 tablet_copy_source_session.cc:215] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:19.352960 29288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a732ff5937ea47eaaef6755cac5ad60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.352878 28232 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:19.354050 29286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b72fb1e20942d9a3907c50252930bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.354893 28153 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:19.362496 29303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2929b1e07bb74c67863e4c504d8d09a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:19.364003 29299 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:19.388382 28232 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:19.390668 29177 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:11:19.403952 29189 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:11:19.405795 28235 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:19.407493 29177 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:11:19.436519 28154 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:19.450824 29217 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:19.467214 29288 tablet_copy_client.cc:806] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:19.633520 29284 tablet_copy_client.cc:806] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:19.639715 29284 tablet_copy_client.cc:670] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:19.642338 29286 tablet_copy_client.cc:806] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:19.643177 29286 tablet_copy_client.cc:670] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:19.616811 29288 tablet_copy_client.cc:670] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
W20260709 13:11:19.736049 29314 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.106s	user 0.003s	sys 0.000s
I20260709 13:11:19.741844 29286 tablet_copy_client.cc:538] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:19.762516 29288 tablet_copy_client.cc:538] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:11:19.790371 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 186310us
W20260709 13:11:19.796635 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 131889us
I20260709 13:11:19.873862 29288 tablet_bootstrap.cc:492] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:19.875481 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:19.877255 29309 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:11:19.876744 29286 tablet_bootstrap.cc:492] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Bootstrap starting.
W20260709 13:11:19.904654 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46844.3us
W20260709 13:11:19.993542 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 85716.9us
I20260709 13:11:20.016068 29303 tablet_copy_client.cc:806] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:20.016876 29303 tablet_copy_client.cc:670] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
W20260709 13:11:20.048053 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 580881us
W20260709 13:11:20.051832 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 89655.4us
W20260709 13:11:20.060566 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 100466us
I20260709 13:11:20.206017 29303 tablet_copy_client.cc:538] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:11:20.216568 29334 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:11:20.218216 29303 tablet_bootstrap.cc:492] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:20.224720 29288 tablet_bootstrap.cc:492] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:20.225783 29288 tablet_bootstrap.cc:492] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:20.229485 29288 ts_tablet_manager.cc:1403] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.356s	user 0.033s	sys 0.010s
I20260709 13:11:20.236152 29288 raft_consensus.cc:359] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.237076 29288 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:20.237751 29288 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.250193 29288 ts_tablet_manager.cc:1434] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:11:20.252451 28245 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-a732ff5937ea47eaaef6755cac5ad60e received from {username='slave'} at 127.0.0.1:44070
I20260709 13:11:20.252933 28245 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-a732ff5937ea47eaaef6755cac5ad60e on tablet a732ff5937ea47eaaef6755cac5ad60e with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:20.266033 29286 tablet_bootstrap.cc:492] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:20.267206 29286 tablet_bootstrap.cc:492] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:20.269134 29286 ts_tablet_manager.cc:1403] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.416s	user 0.019s	sys 0.008s
I20260709 13:11:20.384925 29286 raft_consensus.cc:359] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.387020 29286 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:20.387902 29286 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.391283 29284 tablet_copy_client.cc:538] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:20.400421 29303 tablet_bootstrap.cc:492] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:20.426901 28156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0720cab120de4047b423cb89ed85154a"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:20.427613 28156 raft_consensus.cc:606] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:20.428040 29303 tablet_bootstrap.cc:492] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:20.428869 29303 ts_tablet_manager.cc:1403] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.211s	user 0.019s	sys 0.024s
I20260709 13:11:20.443560 28235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:20.444209 28235 raft_consensus.cc:606] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:20.445114 29303 raft_consensus.cc:359] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.456447 28079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:20.457152 28079 raft_consensus.cc:606] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:20.467429 29286 ts_tablet_manager.cc:1434] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.195s	user 0.008s	sys 0.000s
I20260709 13:11:20.445971 29303 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:20.477417 29303 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.482213 29303 ts_tablet_manager.cc:1434] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 13:11:20.482445 28762 raft_consensus.cc:1217] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:20.484234 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-2929b1e07bb74c67863e4c504d8d09a6 received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:20.484383 28245 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-33b72fb1e20942d9a3907c50252930bb received from {username='slave'} at 127.0.0.1:44070
I20260709 13:11:20.484671 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-2929b1e07bb74c67863e4c504d8d09a6 on tablet 2929b1e07bb74c67863e4c504d8d09a6 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:20.484923 28245 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-33b72fb1e20942d9a3907c50252930bb on tablet 33b72fb1e20942d9a3907c50252930bb with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:20.499390 28233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:20.500108 28233 raft_consensus.cc:606] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:20.502558 28762 raft_consensus.cc:1217] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:20.507462 28233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:20.508113 28233 raft_consensus.cc:606] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:20.517838 28079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:20.518500 28079 raft_consensus.cc:606] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:20.527470 29284 tablet_bootstrap.cc:492] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:20.527992 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:11:20.544209 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:20.558581 29239 raft_consensus.cc:993] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:20.559172 29239 raft_consensus.cc:1081] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:20.569681 28079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:20.570119 29302 raft_consensus.cc:993] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:20.570637 29302 raft_consensus.cc:1081] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:20.570344 28079 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:20.571256 28079 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.576354 28077 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:20.577102 28077 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:20.577469 28077 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.587356 28079 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:20.592412 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:20.593086 28235 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:20.593552 28235 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:20.594401 28235 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:20.595598 28235 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.592435 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:20.601878 28156 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.610169 28079 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:20.621951 29189 raft_consensus.cc:993] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:20.622519 29189 raft_consensus.cc:1081] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:20.621932 28077 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.625384 28762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:56420
I20260709 13:11:20.639523 28762 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:20.639943 28762 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:20.651329 28762 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:20.629076 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:20.659368 28233 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:20.659719 28233 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:20.660631 28233 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.629902 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:20.662818 28157 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.669770 28233 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.678483 28157 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:20.686127 28050 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:20.687963 29331 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:20.688611 29331 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:20.689677 29331 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.691437 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:20.692047 28234 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:20.692615 28762 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:20.657087 28235 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:20.627388 28077 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:20.682957 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:20.697060 28079 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:20.697453 28079 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:20.698330 28079 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:20.698890 28234 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:20.699921 28079 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:20.700414 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:20.657938 28156 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:20.702432 29325 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:20.704489 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:20.706985 28233 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:20.702951 29325 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 3 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:20.706949 29315 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:20.711031 29315 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:20.711968 29315 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:20.715627 29325 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:20.734182 29284 tablet_bootstrap.cc:492] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:20.735141 29284 tablet_bootstrap.cc:492] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:20.735891 29284 ts_tablet_manager.cc:1403] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.209s	user 0.024s	sys 0.012s
I20260709 13:11:20.754818 28079 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:20.763373 27981 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 2 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 3 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:20.769120 27984 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:20.775542 29284 raft_consensus.cc:359] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.777693 29284 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:20.786679 28762 raft_consensus.cc:1217] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:11:20.779167 29284 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:20.794808 29258 raft_consensus.cc:993] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:20.795322 29168 raft_consensus.cc:1081] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:20.797150 28762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0720cab120de4047b423cb89ed85154a"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:35892
I20260709 13:11:20.797745 28762 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:20.798130 28762 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.813517 28762 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:20.816288 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:20.816990 28078 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.817874 29284 ts_tablet_manager.cc:1434] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.005s
I20260709 13:11:20.818708 29242 raft_consensus.cc:993] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:20.819209 29242 raft_consensus.cc:1081] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:20.821688 28077 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:20.821713 28170 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-847d15cbd80f44b18e547291eed25617 received from {username='slave'} at 127.0.0.1:43184
I20260709 13:11:20.822260 28077 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:20.822813 28170 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-847d15cbd80f44b18e547291eed25617 on tablet 847d15cbd80f44b18e547291eed25617 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:20.823141 28077 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:20.829815 28078 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:20.832077 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:20.835649 28156 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:20.836084 28156 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:20.837303 28156 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:20.840788 28762 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:20.841723 28077 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:20.842250 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:20.842837 28156 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:20.843253 29325 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:20.844921 29325 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:20.845803 29325 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:20.846835 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:20.847398 28762 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:20.850737 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:20.853331 28233 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:20.855052 28233 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:20.856447 29315 raft_consensus.cc:993] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:20.860984 29331 raft_consensus.cc:1081] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:20.863256 28161 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:50460
I20260709 13:11:20.863875 28161 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:20.860635 28233 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:20.851106 28077 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:20.864327 28161 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:20.866335 28233 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:20.873848 28762 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:20.877106 28049 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:20.878021 29331 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:20.878175 28233 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:20.878563 29331 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:20.879268 28161 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:20.883430 29195 raft_consensus.cc:1064] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:20.886396 28161 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:20.879459 29331 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:20.888476 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:20.889087 28077 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:20.888916 29195 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:20.889607 28077 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:20.890456 28077 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:20.891707 28077 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:20.889078 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:20.898469 28156 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:20.899863 28077 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:20.903192 28233 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:20.904623 28761 raft_consensus.cc:1217] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:20.923584 28761 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:20.923867 28161 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:20.929349 29323 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:11:20.930652 29195 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:20.931411 28125 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:20.932931 28233 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:20.933770 28078 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:20.938045 29333 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:20.938694 29333 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:20.939591 29333 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:20.959713 29239 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:21.009840 27983 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.036552 27984 catalog_manager.cc:5697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.039858 27984 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 3, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 3 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:21.058923 29195 raft_consensus.cc:2955] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:21.062479 28079 raft_consensus.cc:2955] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:21.100914 28233 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:21.100656 28761 raft_consensus.cc:2955] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:21.112247 29317 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:21.117290 27983 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:21.146689 27981 catalog_manager.cc:5697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:21.193845 28077 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:21.199456 28156 raft_consensus.cc:2955] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:21.205176 29299 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:11:21.212582 28157 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:21.216136 29315 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:21.233264 28233 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:21.243439 29299 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:21.243933 28235 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:21.251655 28761 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:21.253379 29242 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:21.262463 29323 raft_consensus.cc:2955] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:21.300714 28761 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Advancing to term 2
I20260709 13:11:21.312646 28762 raft_consensus.cc:2955] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:21.319730 28078 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:21.322471 29299 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:21.324306 28234 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:21.325563 28154 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:21.325974 29189 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:21.328339 28161 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:21.345741 27982 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:21.349045 29315 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:21.350562 28235 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:21.355792 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 33b72fb1e20942d9a3907c50252930bb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:21.358963 29317 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:21.367573 29195 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:21.371856 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet 33b72fb1e20942d9a3907c50252930bb with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:21.392609 29315 raft_consensus.cc:1064] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:11:21.400848 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.422145 28762 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:21.424017 29317 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:21.442133 29333 raft_consensus.cc:1064] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:21.445600 29333 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:21.402268 29315 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:21.460361 28161 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:21.463109 29299 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:11:21.507855 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 140483us
I20260709 13:11:21.518476 28153 raft_consensus.cc:2955] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:21.312577 28761 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:21.531915 29368 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:21.536842 28077 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:21.538714 29168 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:21.546665 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:21.578814 29331 raft_consensus.cc:1064] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:21.584100 29331 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:21.607985 28761 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:11:21.609504 29317 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:21.614138 28234 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:11:21.621037 29317 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:21.647974 28761 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:11:21.651280 29320 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:21.656159 28154 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:11:21.659349 29189 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:21.649998 28234 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:11:21.652383 29366 tablet_replica.cc:333] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:11:21.690830 28234 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:11:21.744122 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:21.770180 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 89961.2us
I20260709 13:11:21.729763 29189 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
W20260709 13:11:21.855613 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64576.5us
I20260709 13:11:21.860291 28761 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:21.844301 29381 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:11:21.884505 29368 raft_consensus.cc:2955] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:21.947888 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:21.901582 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 93490.8us
I20260709 13:11:22.018558 29366 raft_consensus.cc:2243] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:22.019325 29366 raft_consensus.cc:2272] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:22.081415 29168 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:22.084043 28156 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 13:11:22.086127 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 62563us
W20260709 13:11:22.096410 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 119294us
I20260709 13:11:22.100891 29366 ts_tablet_manager.cc:1916] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:11:22.107259 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 93529.7us
I20260709 13:11:22.109464 28229 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:11:22.160121 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 78924.4us
W20260709 13:11:22.190562 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45619.9us
W20260709 13:11:22.235994 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46687.7us
I20260709 13:11:22.227308 28157 raft_consensus.cc:2955] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:11:22.321035 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 70253.8us
I20260709 13:11:22.222143 29390 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 13:11:22.283344 29398 raft_consensus.cc:2955] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:22.326287 29320 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:11:22.236634 28229 raft_consensus.cc:2955] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:22.355058 29366 ts_tablet_manager.cc:1929] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:22.355581 29366 log.cc:1199] T 33b72fb1e20942d9a3907c50252930bb P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/33b72fb1e20942d9a3907c50252930bb
W20260709 13:11:22.428119 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 232404us
W20260709 13:11:22.428119 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 115384us
I20260709 13:11:22.432765 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet 33b72fb1e20942d9a3907c50252930bb (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:22.454627 28079 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:22.503886 28760 raft_consensus.cc:2955] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:22.515268 28763 raft_consensus.cc:2955] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:11:22.584188 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 51531.1us
W20260709 13:11:22.600854 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46712.8us
W20260709 13:11:22.614995 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 89619.7us
I20260709 13:11:22.555614 28231 raft_consensus.cc:2955] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:22.634562 28153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:22.637282 28153 raft_consensus.cc:606] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:22.679701 29394 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:22.680393 29394 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:22.684453 29394 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:22.714419 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:22.718851 28086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:22.719518 28086 raft_consensus.cc:606] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:22.699747 29414 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:11:22.727923 28229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:22.728646 28229 raft_consensus.cc:606] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:22.726895 27981 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:22.734040 28082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:22.734706 28082 raft_consensus.cc:606] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:22.738746 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:11:22.741298 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:22.793355 29383 raft_consensus.cc:993] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:22.794622 29383 raft_consensus.cc:1081] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:22.794767 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:22.816429 29381 raft_consensus.cc:2955] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:22.854658 28157 raft_consensus.cc:2955] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:22.866894 28082 raft_consensus.cc:2955] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:22.874591 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:22.877116 29315 raft_consensus.cc:993] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:22.879554 29315 raft_consensus.cc:1081] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:22.878981 28229 raft_consensus.cc:2955] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:22.880509 28231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39698
I20260709 13:11:22.881325 28231 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:22.881776 28231 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.895246 28760 raft_consensus.cc:2955] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:22.899694 29389 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:22.908327 28155 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:50460
I20260709 13:11:22.909010 28155 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:22.909389 28155 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.924446 28155 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:22.926666 28155 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:22.939131 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:22.952986 29315 raft_consensus.cc:993] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:22.941563 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:22.953529 29315 raft_consensus.cc:1081] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:22.953552 28233 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.952908 28085 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:22.955883 28234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39712
I20260709 13:11:22.954245 28231 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:22.954699 28085 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:22.959654 28085 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:22.962188 28085 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.956774 28234 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:22.970381 28234 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:22.992794 28085 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:22.994674 28125 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:22.995646 29414 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:22.996183 29414 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:22.997205 29414 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:23.001786 28233 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:23.005199 28231 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:23.006590 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:23.007342 28155 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:23.007675 28155 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:23.013013 28155 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:23.020401 27984 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index 2 to 4, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:23.027381 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:23.027948 28155 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:23.028143 28085 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:23.013808 28234 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:23.038818 28234 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:23.039899 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:23.039928 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:23.042297 28082 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:23.041069 28153 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:23.043795 28155 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:23.045224 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:23.047571 29242 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:23.048303 29242 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:23.042934 28082 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:23.049432 28082 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:23.049341 29242 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:23.050927 28082 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:23.055050 27981 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:23.069133 28083 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:23.080174 28082 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:23.081327 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:23.088181 28153 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:23.102177 28763 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:23.104213 29315 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:23.105657 29397 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:23.106253 29397 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:23.107543 29397 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:23.112860 28234 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:23.119693 29389 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:23.125272 28085 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:23.126251 29384 raft_consensus.cc:2955] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:23.130481 28231 raft_consensus.cc:2955] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:23.135339 28763 raft_consensus.cc:2955] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:23.144094 28086 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:23.156805 27982 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 4 to 5, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:23.158744 28234 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:23.160663 29400 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:23.155560 27981 catalog_manager.cc:5697] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:23.172510 29387 raft_consensus.cc:993] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:23.174237 29387 raft_consensus.cc:1081] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:23.177610 28082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:23.178975 28082 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:23.179553 28082 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:23.190646 29389 raft_consensus.cc:2955] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:23.197788 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet 2929b1e07bb74c67863e4c504d8d09a6 with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50454
I20260709 13:11:23.204389 28231 raft_consensus.cc:2955] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:23.205382 28763 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:23.212774 29398 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:23.208178 28082 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:23.445582 29428 tablet_replica.cc:333] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:11:23.447243 29428 raft_consensus.cc:2243] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:23.460675 29428 raft_consensus.cc:2272] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:24.179647 29428 ts_tablet_manager.cc:1916] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:11:24.247511 28231 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.041s	user 0.004s	sys 0.021s
I20260709 13:11:24.255164 28085 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:24.258260 29428 ts_tablet_manager.cc:1929] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:11:24.261047 29428 log.cc:1199] T 2929b1e07bb74c67863e4c504d8d09a6 P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/2929b1e07bb74c67863e4c504d8d09a6
I20260709 13:11:24.262547 29428 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50454 (request call id 72) took 1065 ms. Trace:
I20260709 13:11:24.263089 29428 rpcz_store.cc:276] 0709 13:11:23.197237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.197462 (+   225us) service_pool.cc:224] Handling call
0709 13:11:23.444913 (+247451us) ts_tablet_manager.cc:1151] Deleting tablet 2929b1e07bb74c67863e4c504d8d09a6
0709 13:11:23.445019 (+   106us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:11:24.238704 (+793685us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:24.258107 (+ 19403us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:24.262448 (+  4341us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":246129,"thread_start_us":368860,"threads_started":1}
I20260709 13:11:24.266891 29382 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:24.267519 29382 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.271166 29382 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:24.276394 28728 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.010s	sys 0.006s
W20260709 13:11:24.287026 28728 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.012s	sys 0.006s
W20260709 13:11:24.289080 29389 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.097s	user 0.010s	sys 0.017s
I20260709 13:11:24.289806 29389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:40978 (request call id 77) took 1228 ms. Trace:
I20260709 13:11:24.280350 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1215) took 1077 ms. Trace:
W20260709 13:11:24.283074 28730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.013s	sys 0.026s
I20260709 13:11:24.290195 29389 rpcz_store.cc:276] 0709 13:11:23.061421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.061569 (+   148us) service_pool.cc:224] Handling call
0709 13:11:23.150203 (+ 88634us) log.cc:844] Serialized 501 byte log entry
0709 13:11:24.289712 (+1139509us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8,"raft.queue_time_us":11102,"raft.run_cpu_time_us":1207,"raft.run_wall_time_us":4508}
W20260709 13:11:24.291761 28730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.014s	sys 0.027s
I20260709 13:11:24.285171 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.292168 29437 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:24.285766 29381 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:24.292999 28231 rpcz_store.cc:276] 0709 13:11:23.202935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.203743 (+   808us) service_pool.cc:224] Handling call
0709 13:11:23.204056 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.204265 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:24.248075 (+1043810us) log.cc:844] Serialized 14 byte log entry
0709 13:11:24.279970 (+ 31895us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.280027 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:24.280069 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.280154 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.280271 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6627,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":31211,"threads_started":1,"wal-append.queue_time_us":30423}
I20260709 13:11:24.310214 29382 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:24.311064 29382 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.315412 29382 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:24.317814 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:24.325045 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 73448.7us
I20260709 13:11:24.328565 29381 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:24.329582 29381 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.335003 29381 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:24.339375 29436 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:24.347520 29394 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:24.348176 29394 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.350808 29394 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:24.350863 29195 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:24.351528 29195 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.366603 29195 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.355829 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.360606 29395 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:24.370530 29395 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.360914 28082 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.360621 29437 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.373888 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40992 (request call id 1449) took 1197 ms. Trace:
I20260709 13:11:24.374285 28082 rpcz_store.cc:276] 0709 13:11:23.176593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.177490 (+   897us) service_pool.cc:224] Handling call
0709 13:11:24.373787 (+1196297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:24.375241 29395 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.375504 29437 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:24.340035 29436 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.380785 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:24.381438 28763 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:24.381754 29436 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.377007 29242 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:24.382474 29242 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.384702 29242 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.378130 29410 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:24.389259 29410 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.378342 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2929b1e07bb74c67863e4c504d8d09a6 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:24.391947 29410 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.393133 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.400893 28763 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:24.402531 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:24.403489 29384 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:24.404018 29384 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:24.404657 29445 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:24.405030 29384 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.405354 29445 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.409210 28157 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:24.409921 29383 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:24.410564 29383 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.413632 29436 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:24.414256 29436 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.414909 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.417114 28764 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:24.420009 29444 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:24.420411 29445 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.428678 29444 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.421721 29438 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:24.421904 29323 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:24.432986 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.433079 29323 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.423264 29242 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:24.435020 29242 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.435566 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.436828 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.437186 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.437767 29323 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.426815 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.427294 29387 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:24.439631 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.441571 29242 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.392391 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet 2929b1e07bb74c67863e4c504d8d09a6 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:24.442564 29438 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:24.443403 27969 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 51028.6us
I20260709 13:11:24.444059 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a732ff5937ea47eaaef6755cac5ad60e with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:24.447281 29445 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:24.449141 29396 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:24.449818 29396 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.452301 29396 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.448592 29445 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:24.453287 29445 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:24.454239 29445 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.455910 29445 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:24.428193 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.461602 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.424183 29383 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.457712 28160 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:24.459296 28256 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7200 after lost signal to thread 26962
I20260709 13:11:24.460510 28229 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:24.462764 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.482926 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.427955 29436 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.466526 27981 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58554 (request call id 237) took 1342 ms. Trace:
I20260709 13:11:24.465651 28082 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:24.489506 29444 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.489946 28158 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:24.493745 28081 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:24.496956 29438 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:24.483351 29383 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:11:24.497464 28256 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6da0 after lost signal to thread 28018
I20260709 13:11:24.497532 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.497871 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.498315 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.499274 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.499377 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.500540 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.501142 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.504245 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.506726 27982 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 4 to 5, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:24.526768 28153 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:24.530637 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:24.532904 28256 debug-util.cc:398] Leaking SignalData structure 0x7b08002e2120 after lost signal to thread 28110
I20260709 13:11:24.533388 28081 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:24.534138 28256 debug-util.cc:398] Leaking SignalData structure 0x7b080036d120 after lost signal to thread 28185
W20260709 13:11:24.534567 28256 debug-util.cc:398] Leaking SignalData structure 0x7b080036d040 after lost signal to thread 28186
I20260709 13:11:24.534291 27982 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 2 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:24.535714 28256 debug-util.cc:398] Leaking SignalData structure 0x7b08001c5ca0 after lost signal to thread 28259
W20260709 13:11:24.537452 28256 debug-util.cc:398] Leaking SignalData structure 0x7b0800370860 after lost signal to thread 28789
I20260709 13:11:24.537678 29323 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:24.538130 28256 debug-util.cc:398] Leaking SignalData structure 0x7b0800370780 after lost signal to thread 29317
W20260709 13:11:24.538494 28256 debug-util.cc:398] Leaking SignalData structure 0x7b08003707a0 after lost signal to thread 29368
I20260709 13:11:24.538334 29323 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:24.538940 28256 debug-util.cc:398] Leaking SignalData structure 0x7b08003708e0 after lost signal to thread 29391
W20260709 13:11:24.539324 28256 debug-util.cc:398] Leaking SignalData structure 0x7b0800370a80 after lost signal to thread 29399
I20260709 13:11:24.540652 29323 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.536842 27982 catalog_manager.cc:5697] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 2 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:24.568207 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:24.571565 29195 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:24.573557 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.565069 29396 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:24.575939 29437 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:24.576306 29396 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.572029 29195 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:24.577601 28086 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:24.577750 29195 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:24.578459 29396 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.559461 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.587184 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:24.581637 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:24.590175 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 72957.6us
I20260709 13:11:24.564983 29242 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:24.527733 29383 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.576481 29437 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.599489 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.602191 28155 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:24.601845 29387 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:24.603204 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.594405 29396 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:24.571620 29436 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:11:24.646142 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.650707 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.653306 29388 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:24.654676 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.655465 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.656193 29443 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.599328 29242 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.659942 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:24.653950 29388 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:24.661119 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 62898.9us
I20260709 13:11:24.512195 27981 rpcz_store.cc:276] 0709 13:11:23.123967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.134059 (+ 10092us) service_pool.cc:224] Handling call
0709 13:11:24.466351 (+1332292us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:23.193796 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:23.194424 (+   628us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:23.194463 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:11:23.194483 (+    20us) tablet.cc:662] Decoding operations
0709 13:11:23.194877 (+   394us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:23.194936 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:23.194968 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:11:23.195200 (+   232us) tablet.cc:701] Row locks acquired
0709 13:11:23.195218 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:11:23.195367 (+   149us) write_op.cc:270] Start()
0709 13:11:23.195483 (+   116us) write_op.cc:276] Timestamp: P: 1783602683195326 usec, L: 0
0709 13:11:23.195506 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:11:23.196174 (+   668us) log.cc:844] Serialized 541 byte log entry
0709 13:11:23.199074 (+  2900us) op_driver.cc:464] REPLICATION: finished
0709 13:11:23.199634 (+   560us) write_op.cc:301] APPLY: starting
0709 13:11:23.199751 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:23.199900 (+   149us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:23.200033 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:23.200346 (+   313us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:23.200564 (+   218us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:11:23.200640 (+    76us) write_op.cc:312] APPLY: finished
0709 13:11:23.200926 (+   286us) log.cc:844] Serialized 23 byte log entry
0709 13:11:23.201142 (+   216us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:23.201263 (+   121us) write_op.cc:454] Released schema lock
0709 13:11:23.201320 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:23.201732 (+   412us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:23.201776 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2495,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":326,"apply.run_cpu_time_us":2037,"apply.run_wall_time_us":2036,"num_ops":1,"prepare.queue_time_us":14683,"prepare.run_cpu_time_us":2678,"prepare.run_wall_time_us":2679,"replication_time_us":3433}]]}
I20260709 13:11:24.581565 28760 raft_consensus.cc:2955] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:24.663633 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.656809 29443 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.664551 29325 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:24.603824 29387 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.680560 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.665642 29435 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:24.669046 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.670424 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.670763 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.665683 28127 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:24.603247 29450 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:11:24.644554 29396 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.687055 29242 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.677067 29400 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.678763 28050 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:24.684099 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:24.687127 29396 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.691421 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:24.695621 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:24.694732 29398 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.700419 29414 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.688524 29383 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.692119 29395 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:24.713044 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.695205 29435 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:24.700295 29382 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.700906 29389 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.723754 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:24.725026 29323 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:24.725616 29452 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:11:24.687878 29400 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:24.727041 29437 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:24.728102 29400 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:24.728950 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.729472 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.709745 29396 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:24.731329 29195 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.731992 28234 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:24.733850 29195 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.734249 29443 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.735205 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:24.737788 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:24.737957 29445 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:24.689396 29325 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.737774 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.721457 29435 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:24.745462 29447 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:11:24.729472 29387 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.747937 29388 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:24.755724 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.756583 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.757591 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.757700 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:24.759445 29445 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.762499 29387 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.766700 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:24.764834 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:24.779093 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:24.780095 28762 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:24.780884 29195 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.783181 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.784881 29395 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:24.752563 29435 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.796494 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:24.798467 28730 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:24.801581 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:24.802223 28155 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:24.758881 29396 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.806360 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.807143 29396 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.796816 29325 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:24.803498 29456 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.810073 29435 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:24.802652 29437 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.819590 28234 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:24.828971 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:24.828982 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:24.821671 29381 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:11:24.842618 29451 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:11:24.843745 29388 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:24.844273 29388 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:24.839922 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:24.845355 28156 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:24.836406 29383 raft_consensus.cc:2955] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:24.845520 29388 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.840015 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:24.850466 28155 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:24.853483 28233 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:24.857646 28086 raft_consensus.cc:2955] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:24.862525 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:11:24.860852 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 162727us
W20260709 13:11:24.863979 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 162917us
I20260709 13:11:24.865211 29388 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:24.865769 29387 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:24.869294 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.867630 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:24.870430 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.867125 28199 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:24.874384 29195 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.875473 29437 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:24.875840 29459 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:24.872041 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.887523 29437 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:24.887746 28760 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:24.887950 29437 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:24.891948 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:24.872980 29387 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:24.889065 29437 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.873836 29450 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.896760 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:24.897819 29195 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.902020 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:24.905100 29195 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.900964 29436 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.909351 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:24.923066 29437 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:24.911738 29394 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 13:11:24.965658 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 63745.4us
I20260709 13:11:24.966663 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:24.970543 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:24.971261 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.971626 29242 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.914196 29387 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:24.974097 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:24.978791 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:24.988116 28234 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:24.982275 29323 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:24.982107 29396 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:24.999670 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 847d15cbd80f44b18e547291eed25617 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:25.008370 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:25.017709 28156 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
W20260709 13:11:25.023651 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 151405us
I20260709 13:11:25.028987 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:11:25.034281 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61355.7us
I20260709 13:11:25.020515 29396 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:25.050339 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:25.055176 29446 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:25.041090 29382 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:25.128702 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 75348us
W20260709 13:11:25.119252 29464 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:11:25.119261 29315 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:25.127103 29410 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:25.132830 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:25.153195 29468 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.171s
I20260709 13:11:25.192037 29410 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:25.107443 28760 raft_consensus.cc:2955] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:25.175325 29315 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:25.277073 29446 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:25.380002 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:25.330590 29446 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:25.458449 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 117295us
I20260709 13:11:25.401026 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:25.469769 28160 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:25.472230 28154 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:25.471148 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:25.474927 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 422693us
I20260709 13:11:25.478580 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:25.481343 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.481732 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:25.481838 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:25.490921 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.493022 29315 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:25.496351 29476 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:11:25.499899 29474 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.208s	user 0.004s	sys 0.000s
I20260709 13:11:25.516614 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:25.527343 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:25.539067 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:25.537566 28728 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:25.543431 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:25.551777 28728 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:25.555279 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:25.553301 29325 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:25.555991 29475 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:25.560778 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 160888us
I20260709 13:11:25.578248 29441 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:25.561484 29325 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:25.580961 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:25.585330 29488 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:25.595350 29410 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:25.568660 29388 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.699s	user 0.013s	sys 0.000s
W20260709 13:11:25.605015 29473 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.283s	user 0.007s	sys 0.000s
I20260709 13:11:25.609345 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet a732ff5937ea47eaaef6755cac5ad60e with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50454
I20260709 13:11:25.616977 27982 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 225) took 1123 ms. Trace:
I20260709 13:11:25.665496 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:25.689845 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:25.699079 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:25.700385 29242 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:25.674136 27982 rpcz_store.cc:276] 0709 13:11:24.493072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.493279 (+   207us) service_pool.cc:224] Handling call
0709 13:11:25.616903 (+1123624us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:24.782139 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:24.806146 (+ 24007us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:24.806187 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:11:24.806212 (+    25us) tablet.cc:662] Decoding operations
0709 13:11:24.806799 (+   587us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:24.806882 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:24.806923 (+    41us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:11:24.807270 (+   347us) tablet.cc:701] Row locks acquired
0709 13:11:24.807292 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:11:24.807441 (+   149us) write_op.cc:270] Start()
0709 13:11:24.807550 (+   109us) write_op.cc:276] Timestamp: P: 1783602684807395 usec, L: 0
0709 13:11:24.807573 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:11:24.809438 (+  1865us) log.cc:844] Serialized 1312 byte log entry
0709 13:11:24.897591 (+ 88153us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce672e28 00007f59d20ad581 00007f59d20acd58 00007f59d20b19f1 00007f59d20b19a9 00007f59d20b1939 00007f59d20b1901 00007f59d20b0b2d 000000000038f354 000000000038f199
0709 13:11:24.958038 (+ 60447us) op_driver.cc:464] REPLICATION: finished
0709 13:11:25.359952 (+401914us) write_op.cc:301] APPLY: starting
0709 13:11:25.360053 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:25.360205 (+   152us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:25.360230 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:25.481650 (+121420us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:25.498167 (+ 16517us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:11:25.516824 (+ 18657us) write_op.cc:312] APPLY: finished
0709 13:11:25.519862 (+  3038us) log.cc:844] Serialized 35 byte log entry
0709 13:11:25.520033 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:25.520121 (+    88us) write_op.cc:454] Released schema lock
0709 13:11:25.520163 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:25.520522 (+   359us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:25.520550 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":385435,"apply.run_cpu_time_us":3369,"apply.run_wall_time_us":160557,"num_ops":3,"prepare.queue_time_us":168144,"prepare.run_cpu_time_us":8477,"prepare.run_wall_time_us":140866,"raft.queue_time_us":133118,"raft.run_cpu_time_us":74255,"raft.run_wall_time_us":332613,"replication_time_us":150274,"spinlock_wait_cycles":2801408,"thread_start_us":538954,"threads_started":4,"wal-append.queue_time_us":13268}]]}
I20260709 13:11:25.722846 28764 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:25.731197 29242 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:25.737895 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:25.739787 29414 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:25.727839 27983 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:25.756750 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 202394us
I20260709 13:11:25.775403 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:25.787061 29492 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:11:25.794064 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:25.800628 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:25.801905 28156 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:25.806414 29394 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:25.822578 29434 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:25.882287 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 317197us
I20260709 13:11:25.892156 29501 tablet_replica.cc:333] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:11:25.916877 29501 raft_consensus.cc:2243] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:25.921276 29501 raft_consensus.cc:2272] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:25.941880 29456 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:25.945555 29501 ts_tablet_manager.cc:1916] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:25.942505 29456 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:25.994715 29456 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:25.983362 29465 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:11:26.019613 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:26.035250 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:26.038872 28199 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:26.039227 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:26.057109 29465 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:26.067318 27984 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.070603 27983 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 236) took 1202 ms. Trace:
I20260709 13:11:26.079523 28157 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:26.082103 29465 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:26.071099 27983 rpcz_store.cc:276] 0709 13:11:24.867902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.868044 (+   142us) service_pool.cc:224] Handling call
0709 13:11:26.070537 (+1202493us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:25.876849 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:25.877594 (+   745us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:25.877648 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:11:25.877676 (+    28us) tablet.cc:662] Decoding operations
0709 13:11:25.878105 (+   429us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:25.878201 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:25.878251 (+    50us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:11:25.878503 (+   252us) tablet.cc:701] Row locks acquired
0709 13:11:25.878532 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:11:25.878719 (+   187us) write_op.cc:270] Start()
0709 13:11:25.878871 (+   152us) write_op.cc:276] Timestamp: P: 1783602685878666 usec, L: 0
0709 13:11:25.878908 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:11:25.904995 (+ 26087us) log.cc:844] Serialized 536 byte log entry
0709 13:11:26.026579 (+121584us) op_driver.cc:464] REPLICATION: finished
0709 13:11:26.054989 (+ 28410us) write_op.cc:301] APPLY: starting
0709 13:11:26.055081 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:26.055194 (+   113us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:26.055329 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:26.055576 (+   247us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:26.055757 (+   181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:11:26.055813 (+    56us) write_op.cc:312] APPLY: finished
0709 13:11:26.056021 (+   208us) log.cc:844] Serialized 23 byte log entry
0709 13:11:26.059455 (+  3434us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:26.059548 (+    93us) write_op.cc:454] Released schema lock
0709 13:11:26.059587 (+    39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":23453,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":28139,"apply.run_cpu_time_us":2222,"apply.run_wall_time_us":26209,"num_ops":1,"prepare.queue_time_us":98224,"prepare.run_cpu_time_us":3689,"prepare.run_wall_time_us":29805,"replication_time_us":145262,"thread_start_us":114946,"threads_started":2}]]}
I20260709 13:11:26.106258 29501 ts_tablet_manager.cc:1929] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:11:26.106891 29501 log.cc:1199] T a732ff5937ea47eaaef6755cac5ad60e P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/a732ff5937ea47eaaef6755cac5ad60e
I20260709 13:11:26.109879 28158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:26.110615 28158 raft_consensus.cc:606] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:26.115012 28232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:26.115650 28232 raft_consensus.cc:606] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:26.117430 28760 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:26.119320 29465 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:26.132133 28216 tablet_service.cc:1558] Processing DeleteTablet for tablet 847d15cbd80f44b18e547291eed25617 with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
I20260709 13:11:26.136035 28232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:26.136761 28232 raft_consensus.cc:606] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:26.157922 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:11:26.159754 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:26.108906 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet a732ff5937ea47eaaef6755cac5ad60e (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
W20260709 13:11:26.177742 27969 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 68843.1us
I20260709 13:11:26.179422 27982 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 2 to 3, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 3 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.182626 27982 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 2 to 3, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 3 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.214864 29509 tablet_replica.cc:333] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:11:26.216243 29509 raft_consensus.cc:2243] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:26.219902 29509 raft_consensus.cc:2272] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:26.223259 29509 ts_tablet_manager.cc:1916] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:26.254711 29509 ts_tablet_manager.cc:1929] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:26.255676 29509 log.cc:1199] T 847d15cbd80f44b18e547291eed25617 P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/847d15cbd80f44b18e547291eed25617
I20260709 13:11:26.258056 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet 847d15cbd80f44b18e547291eed25617 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:26.340171 29469 raft_consensus.cc:993] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:26.340660 29469 raft_consensus.cc:1081] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:26.345537 28078 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:26.346176 28078 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.346575 28078 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.389668 28078 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:26.404414 28078 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:26.418643 29475 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:26.419323 29475 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:26.426609 29475 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:26.432051 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:26.432847 28227 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:26.433171 28227 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:26.433991 28227 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:26.471303 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:26.472126 28158 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.483595 28158 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:26.486743 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:26.489432 29488 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:26.489934 29488 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:26.490830 29488 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:26.499311 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:26.506455 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:26.500533 28227 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.517885 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:26.541203 29475 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:26.580590 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 58243.9us
W20260709 13:11:26.588362 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 46039.1us
I20260709 13:11:26.617419 29465 raft_consensus.cc:993] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:26.617983 29465 raft_consensus.cc:1081] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:26.624396 28227 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:26.631560 28760 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:35896
W20260709 13:11:26.635071 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 88989.2us
I20260709 13:11:26.637472 28760 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.638811 28760 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.656311 29414 raft_consensus.cc:993] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:26.656874 29414 raft_consensus.cc:1081] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:26.676116 28760 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:26.682349 28227 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39698
I20260709 13:11:26.682528 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:26.683055 28227 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:26.683164 28232 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:26.683554 28227 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.683571 28232 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:26.684648 28232 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:26.685873 28232 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.687247 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:26.687970 28078 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.694425 28232 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:26.695669 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:26.700047 29515 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:26.700573 29515 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:26.701530 29515 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:26.703405 27983 catalog_manager.cc:5697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.692790 28760 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:26.718912 28078 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:26.708459 28227 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:26.730059 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:26.731765 28078 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.735831 27984 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:26.741348 28227 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:26.753122 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:26.754040 28158 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:26.754370 28158 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:26.755156 28158 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:26.760986 28078 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:26.763867 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:26.764683 29511 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:26.765200 29511 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:26.766094 29511 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:26.772233 28158 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:26.803622 27982 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 2 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:26.808373 28158 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:26.942718 28158 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:26.946631 29400 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.004060 28227 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:27.005755 29518 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.160990 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:27.178433 28085 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:27.180308 29517 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:11:27.206295 28232 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:27.207823 29515 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:27.219405 28158 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:27.223730 29469 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.253751 28085 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:27.255556 29515 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:11:27.304019 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44751.5us
I20260709 13:11:27.445343 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:27.507582 28232 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:11:27.540153 28198 consensus_peers.cc:597] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53c6cdf8c6704ee59d0a736273582a48. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:27.541757 28158 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:27.543455 28085 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:27.545037 29469 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:27.547281 29511 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:27.557897 28232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:27.558605 28232 raft_consensus.cc:606] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:27.562578 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:27.563524 28078 raft_consensus.cc:606] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:27.570375 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:27.570863 29469 raft_consensus.cc:993] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:27.570988 28078 raft_consensus.cc:606] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:27.571686 29469 raft_consensus.cc:1081] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:27.572582 28085 raft_consensus.cc:2955] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.573019 29446 raft_consensus.cc:2955] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.579173 28078 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:27.588418 28227 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:27.589186 28227 raft_consensus.cc:606] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:27.616847 28083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:27.617503 28083 raft_consensus.cc:606] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:27.618613 28158 raft_consensus.cc:2955] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.600389 28153 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:27.624715 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:27.626571 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:27.631487 28078 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:27.632103 28078 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:27.639483 28078 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:11:27.652208 28124 consensus_peers.cc:597] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6b1c27f68974f4a9878c7c155cb43d2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:27.655512 28078 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:27.655983 28227 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:27.657552 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:27.663887 28232 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:27.664276 28232 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:27.665292 28085 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:27.665131 28232 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:27.666637 28232 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:27.664067 27984 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index -1 to 2, term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to <none>, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 2 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.671087 29514 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:27.673874 29514 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:27.661309 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:27.693889 28158 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:27.716027 28158 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:27.717316 28232 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:27.722540 28052 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:27.723686 29488 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:27.724155 29488 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:27.725283 29488 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:27.753940 29514 raft_consensus.cc:2955] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.762363 28085 raft_consensus.cc:2955] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.778504 29543 raft_consensus.cc:993] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:27.779402 29543 raft_consensus.cc:1081] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:27.783916 28160 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:50460
I20260709 13:11:27.784607 28160 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:27.784991 28160 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:27.796865 28160 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:27.799072 28160 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:27.799811 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:27.803273 28085 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:27.803570 28085 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:27.804250 28085 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:27.805332 28085 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:27.805922 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:27.806659 28228 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:27.806936 28227 raft_consensus.cc:2955] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.816012 28228 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:27.826346 28085 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:27.834823 27983 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 1, leader changed from <none> to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 1 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.844432 28125 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:27.846966 29536 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:27.847700 29536 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:27.848137 28160 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:27.848577 29536 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:27.872541 28124 consensus_peers.cc:597] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7ee90e83c2f49509f54dc87ea4d50e6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:27.860397 27983 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.886085 27984 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:27.890607 28085 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:27.901197 29550 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:11:27.907652 28228 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:27.921517 29536 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:27.922178 29400 raft_consensus.cc:993] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:27.922662 29400 raft_consensus.cc:1081] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:27.934151 28158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:50460
I20260709 13:11:27.934900 28158 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:27.935281 28158 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:27.942669 28158 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:27.944793 28158 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:27.942381 27983 catalog_manager.cc:5697] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:27.946169 29538 raft_consensus.cc:2955] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.956873 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:27.957849 28227 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:27.956873 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:27.959265 28078 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:27.959540 28078 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:27.960311 28078 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:27.962105 28078 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:27.969159 28085 raft_consensus.cc:2955] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:27.971944 28078 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:27.973445 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:27.974332 29536 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:27.974852 29536 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:27.975699 29536 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:28.000793 28078 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:28.005327 28232 raft_consensus.cc:2955] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:28.018484 27983 catalog_manager.cc:5697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:28.024402 27984 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index -1 to 4, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:28.045852 28227 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:28.058727 28227 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:28.060375 29389 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:28.061354 28049 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f5f1be456de4894af76dffdad9ea6a3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:28.089871 29389 raft_consensus.cc:2955] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:28.093812 29488 raft_consensus.cc:993] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:28.094328 29488 raft_consensus.cc:1081] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:28.095594 28157 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:11:28.099596 28760 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:56420
I20260709 13:11:28.100183 28760 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:28.100486 28760 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:28.107288 28228 raft_consensus.cc:2955] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:28.108552 28760 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:28.111416 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:28.111994 28085 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:28.112305 28085 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:28.113051 28085 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:28.114085 28085 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:28.117902 29400 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:28.129309 29469 raft_consensus.cc:993] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:28.129992 29469 raft_consensus.cc:1081] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:28.129422 27983 catalog_manager.cc:5697] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:28.131474 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:28.132035 28078 raft_consensus.cc:606] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Received request to transfer leadership
I20260709 13:11:28.132313 28760 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:28.137277 28085 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:28.142385 28078 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:28.143047 28078 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:28.143440 28078 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:28.144467 28158 raft_consensus.cc:2955] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:28.146850 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:28.151578 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:28.151618 28227 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:28.152544 29441 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:28.153043 29441 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 3 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:28.153910 29441 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:28.148175 28077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:28.156057 28077 raft_consensus.cc:606] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Received request to transfer leadership
W20260709 13:11:28.157485 27969 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 53c6cdf8c6704ee59d0a736273582a48 already in progress [suppressed 4 similar messages]
I20260709 13:11:28.195689 28078 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:28.197779 28078 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:28.201465 28227 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:28.211331 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:28.211939 28153 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:28.223304 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:28.225649 27983 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 2 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 3 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:28.229254 28229 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:28.234427 28229 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:28.235323 28229 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:28.236675 28229 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:28.234573 29559 ts_tablet_manager.cc:933] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:28.240459 28049 consensus_peers.cc:597] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53c6cdf8c6704ee59d0a736273582a48. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:28.252252 28153 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:28.253906 28050 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:28.256853 29389 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:28.260681 29389 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:28.262168 29389 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:28.273878 29559 tablet_copy_client.cc:323] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:11:28.275430 28229 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:28.275952 28170 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet e6b1c27f68974f4a9878c7c155cb43d2 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:11:28.278995 28170 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet e6b1c27f68974f4a9878c7c155cb43d2 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-e6b1c27f68974f4a9878c7c155cb43d2
I20260709 13:11:28.289129 28170 tablet_copy_source_session.cc:215] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:28.298885 29559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6b1c27f68974f4a9878c7c155cb43d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:28.300020 28153 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:28.311041 27983 catalog_manager.cc:5697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:28.316026 29567 ts_tablet_manager.cc:933] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:28.320188 29400 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:28.334777 29559 tablet_copy_client.cc:806] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:28.335496 29559 tablet_copy_client.cc:670] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:28.344995 29559 tablet_copy_client.cc:538] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:11:28.372193 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 168838us
I20260709 13:11:28.378022 28077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:28.378664 28077 raft_consensus.cc:606] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Received request to transfer leadership
I20260709 13:11:28.381521 29559 tablet_bootstrap.cc:492] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:28.393057 29543 raft_consensus.cc:993] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:28.393594 29543 raft_consensus.cc:1081] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:28.395622 28227 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:28.397619 28158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:50460
I20260709 13:11:28.398398 29538 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:28.407181 28235 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:11:28.420153 29400 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:11:28.422598 28158 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:28.423110 28158 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:28.427911 28032 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:11:28.428407 28032 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 53c6cdf8c6704ee59d0a736273582a48 already in progress
W20260709 13:11:28.428663 28032 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 53c6cdf8c6704ee59d0a736273582a48 already in progress
I20260709 13:11:28.429669 28077 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:28.431048 29567 tablet_copy_client.cc:323] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:11:28.431387 28158 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:28.436393 29550 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:28.439074 28227 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:28.444685 29550 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:28.454531 28170 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet b7ee90e83c2f49509f54dc87ea4d50e6 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:11:28.455125 28170 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet b7ee90e83c2f49509f54dc87ea4d50e6 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-b7ee90e83c2f49509f54dc87ea4d50e6
I20260709 13:11:28.471716 28158 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:28.475495 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:28.487699 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:28.488265 28078 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:28.488531 28078 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:28.492854 28085 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:28.494462 29536 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:28.500321 28078 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:28.502736 28078 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:28.513505 28078 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:28.515065 28125 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:28.516515 29550 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:28.517046 29550 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:28.517915 29550 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:28.535987 28170 tablet_copy_source_session.cc:215] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:28.546867 29567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ee90e83c2f49509f54dc87ea4d50e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:28.567564 27984 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:28.614480 29567 tablet_copy_client.cc:806] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:28.615871 29567 tablet_copy_client.cc:670] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:28.627759 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:28.650545 28078 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:28.652129 29562 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:28.661597 29567 tablet_copy_client.cc:538] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:28.705004 28235 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:28.712342 29562 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:11:28.731429 29559 tablet_bootstrap.cc:492] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:28.732538 29559 tablet_bootstrap.cc:492] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:28.733444 29559 ts_tablet_manager.cc:1403] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.352s	user 0.035s	sys 0.025s
I20260709 13:11:28.734882 29567 tablet_bootstrap.cc:492] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:28.748981 29559 raft_consensus.cc:359] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:28.750062 29559 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:28.750847 29559 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:28.756606 29559 ts_tablet_manager.cc:1434] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 13:11:28.759925 28170 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-e6b1c27f68974f4a9878c7c155cb43d2 received from {username='slave'} at 127.0.0.1:43184
I20260709 13:11:28.760422 28170 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-e6b1c27f68974f4a9878c7c155cb43d2 on tablet e6b1c27f68974f4a9878c7c155cb43d2 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:28.764427 28229 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:28.765800 29579 ts_tablet_manager.cc:933] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:28.767503 29400 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:28.773370 28158 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:28.794606 29543 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:28.877096 29579 tablet_copy_client.cc:323] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:11:28.885689 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet 6f5f1be456de4894af76dffdad9ea6a3 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:11:28.886308 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet 6f5f1be456de4894af76dffdad9ea6a3 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-6f5f1be456de4894af76dffdad9ea6a3
I20260709 13:11:28.895190 28096 tablet_copy_source_session.cc:215] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:28.896028 29567 tablet_bootstrap.cc:492] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:28.896953 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:28.897138 29567 tablet_bootstrap.cc:492] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:28.904117 29567 ts_tablet_manager.cc:1403] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.170s	user 0.031s	sys 0.002s
I20260709 13:11:28.911537 29579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f5f1be456de4894af76dffdad9ea6a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:28.930872 29567 raft_consensus.cc:359] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:28.976923 29567 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:28.992076 29567 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:28.995680 29567 ts_tablet_manager.cc:1434] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 13:11:28.997915 28170 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-b7ee90e83c2f49509f54dc87ea4d50e6 received from {username='slave'} at 127.0.0.1:43184
I20260709 13:11:28.998421 28170 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-b7ee90e83c2f49509f54dc87ea4d50e6 on tablet b7ee90e83c2f49509f54dc87ea4d50e6 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:29.006062 29579 tablet_copy_client.cc:806] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:29.010596 29579 tablet_copy_client.cc:670] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:29.014950 29579 tablet_copy_client.cc:538] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:29.049185 29579 tablet_bootstrap.cc:492] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:29.058327 28078 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:29.066455 29536 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:29.119760 29579 tablet_bootstrap.cc:492] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:29.120930 29579 tablet_bootstrap.cc:492] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:29.121882 29579 ts_tablet_manager.cc:1403] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.073s	user 0.023s	sys 0.003s
W20260709 13:11:29.124445 28124 consensus_peers.cc:597] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53c6cdf8c6704ee59d0a736273582a48. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:29.127005 29579 raft_consensus.cc:359] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:29.133728 29579 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:29.134692 29579 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:29.138132 29579 ts_tablet_manager.cc:1434] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:11:29.145695 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-6f5f1be456de4894af76dffdad9ea6a3 received from {username='slave'} at 127.0.0.1:36768
I20260709 13:11:29.146175 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-6f5f1be456de4894af76dffdad9ea6a3 on tablet 6f5f1be456de4894af76dffdad9ea6a3 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:29.154052 28760 raft_consensus.cc:1217] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:11:29.219372 28227 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:29.231640 28760 raft_consensus.cc:1217] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:11:29.350723 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 63206.6us
I20260709 13:11:29.389657 28227 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:11:29.407073 29538 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:11:29.453361 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 56422.2us
I20260709 13:11:29.522722 28083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:29.523545 28083 raft_consensus.cc:606] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:29.530372 28153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "847d15cbd80f44b18e547291eed25617"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:29.534597 28153 raft_consensus.cc:606] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:29.538964 28227 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:29.539654 28227 raft_consensus.cc:606] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:29.543044 28153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:29.543618 28153 raft_consensus.cc:606] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:29.549981 28153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:29.550602 28153 raft_consensus.cc:606] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:29.554956 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:29.556757 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:29.583631 29579 ts_tablet_manager.cc:933] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:29.588270 29572 raft_consensus.cc:993] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:29.588995 29572 raft_consensus.cc:1081] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:29.591550 28235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39712
I20260709 13:11:29.592178 28235 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.592546 28235 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:29.600263 29579 tablet_copy_client.cc:323] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:11:29.601670 28235 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:29.606678 29514 raft_consensus.cc:1064] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:11:29.608460 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 196151us
W20260709 13:11:29.661742 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54525.4us
I20260709 13:11:29.665174 29514 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:29.654388 28170 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet 53c6cdf8c6704ee59d0a736273582a48 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:11:29.666844 28170 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet 53c6cdf8c6704ee59d0a736273582a48 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-53c6cdf8c6704ee59d0a736273582a48
I20260709 13:11:29.676461 28235 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:29.679273 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:29.680382 28160 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:29.691614 28160 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:29.698172 28083 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:29.699256 28759 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:29.700973 28760 raft_consensus.cc:1217] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:11:29.716975 29516 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:29.703187 28170 tablet_copy_source_session.cc:215] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:29.731635 29579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c6cdf8c6704ee59d0a736273582a48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:29.720064 29514 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:29.732627 28227 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:29.733559 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:29.734176 28085 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:29.734484 28085 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:29.735153 28085 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:29.743974 29514 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:29.754349 28085 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:29.758044 29516 raft_consensus.cc:993] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:29.758773 29516 raft_consensus.cc:1081] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:29.760980 28083 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:41006
I20260709 13:11:29.761574 28083 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.761922 28083 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.762763 28085 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:29.771788 28083 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:29.778412 29511 raft_consensus.cc:993] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:29.779801 29511 raft_consensus.cc:1081] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:29.788272 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:29.787340 29514 raft_consensus.cc:2955] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:29.788932 28158 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:29.789279 28158 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:11:29.789427 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45959.6us
I20260709 13:11:29.789973 28158 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:29.790997 28158 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.791092 28083 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:29.792969 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:29.793550 28235 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.791038 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:29.794797 29563 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:29.795269 29563 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:29.796036 29563 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:29.805126 28083 raft_consensus.cc:2955] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:29.824100 29593 raft_consensus.cc:993] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:29.824939 29593 raft_consensus.cc:1081] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:29.825634 29536 raft_consensus.cc:1064] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:29.822579 28235 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:29.829623 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:29.830201 28760 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "847d15cbd80f44b18e547291eed25617"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:35892
I20260709 13:11:29.830839 28760 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.829918 28227 raft_consensus.cc:2955] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:29.831153 28760 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.832044 29488 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:29.832976 29488 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:29.833976 29488 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:29.837445 28762 raft_consensus.cc:2955] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:29.839918 28760 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:29.842970 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:29.844560 28083 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.858692 28083 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:29.860929 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:29.861907 29591 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:29.862452 29591 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:29.871834 28077 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:29.872123 28158 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:29.872509 28077 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:29.872967 28077 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.863420 29591 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:29.828370 29536 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:29.897087 27981 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 4 to 5, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:29.910567 28079 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:29.912240 29514 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:29.900429 28077 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:29.913362 29514 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:29.916612 29514 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:29.916998 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:29.917649 28227 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:29.917981 28227 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:29.917899 28760 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:29.917774 29514 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:29.918862 28227 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:29.919693 29514 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.919961 28227 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.922223 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:29.924978 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:29.925578 28153 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:29.933583 28079 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:29.934922 28760 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:29.935313 29593 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:29.936440 29536 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:29.944478 28227 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:29.946760 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:29.947695 28153 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:29.950088 28227 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:29.924960 28077 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:29.952843 29595 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:29.957019 29595 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:29.967278 29550 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:11:29.964516 29595 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:29.977990 29579 tablet_copy_client.cc:806] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:29.996717 29579 tablet_copy_client.cc:670] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:30.013895 27982 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 2 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 3 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:30.024801 29538 raft_consensus.cc:993] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:30.025377 29538 raft_consensus.cc:1081] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:30.027117 29579 tablet_copy_client.cc:538] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:30.036054 28228 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39698
I20260709 13:11:30.036782 28228 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:30.037189 28228 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:30.047412 28228 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:30.054121 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:30.054827 28760 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:30.055856 28762 raft_consensus.cc:2955] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:30.055519 29550 raft_consensus.cc:2955] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:30.067137 29579 tablet_bootstrap.cc:492] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:30.078135 28760 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:30.082155 28198 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:30.086958 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:30.088250 28153 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:30.088547 28153 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:30.089411 28153 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:30.090633 28153 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:30.096184 29563 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:30.099732 29563 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:30.100693 29563 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:30.078676 28228 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:30.127082 28153 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:30.147665 28158 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
W20260709 13:11:30.174597 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46266.1us
I20260709 13:11:30.180512 28079 raft_consensus.cc:2955] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:30.187896 27984 catalog_manager.cc:5697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:30.197991 29488 raft_consensus.cc:1064] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:30.228886 28158 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:30.249965 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 67193.9us
I20260709 13:11:30.252202 28762 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:30.273155 29536 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:30.252663 29488 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:30.283407 28227 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:30.280319 28228 raft_consensus.cc:2955] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:30.287352 28229 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:30.291492 29536 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:30.293951 27982 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 2 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:30.300237 29615 raft_consensus.cc:2955] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:30.302024 28760 raft_consensus.cc:2955] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:30.306937 28077 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:30.264961 27984 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:30.317036 27984 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:30.320838 29543 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:30.331593 28158 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:30.337129 28077 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:30.341238 29594 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:30.343320 29617 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:30.352972 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b7ee90e83c2f49509f54dc87ea4d50e6 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:30.358834 29617 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:30.366041 28227 raft_consensus.cc:2955] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:30.378237 28153 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:30.384459 29594 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:30.391944 28153 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:30.398765 29572 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:11:30.426594 28160 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:30.533663 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 96953us
I20260709 13:11:30.534371 27984 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 5 to 6, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:30.527596 27983 catalog_manager.cc:5697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:30.558851 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:30.531270 29579 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:11:30.702664 28760 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:30.742398 29550 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:30.778837 28227 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:30.786208 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet b7ee90e83c2f49509f54dc87ea4d50e6 with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:30.823436 28762 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:30.799969 29543 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:11:30.839788 28158 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:30.864324 28764 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:11:30.872694 28158 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:30.875727 28153 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:30.967128 28084 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:30.988972 29640 tablet_replica.cc:333] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:11:30.993640 29640 raft_consensus.cc:2243] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.994927 29640 raft_consensus.cc:2272] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.012400 29589 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:11:31.016155 29579 tablet_bootstrap.cc:492] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:31.020598 29640 ts_tablet_manager.cc:1916] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:31.002343 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:31.025002 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 52595.7us
W20260709 13:11:31.028402 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55407.3us
I20260709 13:11:31.042007 29579 tablet_bootstrap.cc:492] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:31.042975 29579 ts_tablet_manager.cc:1403] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.976s	user 0.025s	sys 0.021s
W20260709 13:11:31.091389 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 60987.5us
W20260709 13:11:31.133486 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 263984us
W20260709 13:11:31.137362 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 178344us
I20260709 13:11:31.139995 29517 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:31.150487 29599 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:31.155349 29658 raft_consensus.cc:2955] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:31.157215 29640 ts_tablet_manager.cc:1929] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:11:31.126710 29635 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:11:31.161706 28759 raft_consensus.cc:2955] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:31.166898 28235 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:31.168373 29633 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:31.169461 29579 raft_consensus.cc:359] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:11:31.185304 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 174229us
I20260709 13:11:31.187472 29640 log.cc:1199] T b7ee90e83c2f49509f54dc87ea4d50e6 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/b7ee90e83c2f49509f54dc87ea4d50e6
I20260709 13:11:31.209273 28085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6810464df4a245f49df908fc877020d7"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:31.209959 28085 raft_consensus.cc:606] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Received request to transfer leadership to TS 70996712c3c942468a6006602de6477f
I20260709 13:11:31.197124 29579 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:31.211813 29579 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:31.236497 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet b7ee90e83c2f49509f54dc87ea4d50e6 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:31.246349 29579 ts_tablet_manager.cc:1434] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.152s	user 0.003s	sys 0.003s
I20260709 13:11:31.183497 29652 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.124s
W20260709 13:11:31.272904 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 90164.8us
I20260709 13:11:31.280434 29650 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
W20260709 13:11:31.304116 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 51266.8us
I20260709 13:11:31.312928 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:31.313584 28078 raft_consensus.cc:606] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:31.330852 28761 raft_consensus.cc:1217] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Deduplicated request from leader. Original: 2.3->[3.4-3.4]   Dedup: 3.4->[]
W20260709 13:11:31.355254 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 52213.8us
W20260709 13:11:31.417703 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 61266.9us
W20260709 13:11:31.418309 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 60333us
W20260709 13:11:31.430867 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 82225.9us
I20260709 13:11:31.461910 28086 raft_consensus.cc:2955] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:31.458997 29658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:50454 (request call id 76) took 1031 ms. Trace:
W20260709 13:11:31.503680 29663 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:11:31.524068 28764 raft_consensus.cc:2955] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:31.481307 29658 rpcz_store.cc:276] 0709 13:11:30.422393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.422710 (+   317us) service_pool.cc:224] Handling call
0709 13:11:30.437944 (+ 15234us) log.cc:844] Serialized 501 byte log entry
0709 13:11:30.580398 (+142454us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b54002024a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f7b26 00007f59ce5f149a 00007f59ce5f0714 00007f59ce5d0eaa 00007f59ce5d7d53 00007f59ce5d7cd9 00007f59ce5d7c69 00007f59ce5d7c31 00007f59ce5d6dfd 000000000038f354
0709 13:11:31.450962 (+870564us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1386,"raft.queue_time_us":3391,"raft.run_cpu_time_us":3479,"raft.run_wall_time_us":302595,"spinlock_wait_cycles":7279872}
I20260709 13:11:31.554011 28170 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-53c6cdf8c6704ee59d0a736273582a48 received from {username='slave'} at 127.0.0.1:43184
I20260709 13:11:31.554481 28170 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-53c6cdf8c6704ee59d0a736273582a48 on tablet 53c6cdf8c6704ee59d0a736273582a48 with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:31.714965 29488 raft_consensus.cc:2955] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:31.749559 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e6b1c27f68974f4a9878c7c155cb43d2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:31.824187 29550 raft_consensus.cc:1064] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:11:31.827828 29550 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:31.850127 28762 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:31.851939 29649 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:31.865839 29635 raft_consensus.cc:993] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:31.866380 29635 raft_consensus.cc:1081] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:31.891921 29657 raft_consensus.cc:993] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:31.892976 29657 raft_consensus.cc:1081] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Signalling peer 70996712c3c942468a6006602de6477f to start an election
I20260709 13:11:31.898133 28764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6810464df4a245f49df908fc877020d7"
dest_uuid: "70996712c3c942468a6006602de6477f"
 from {username='slave'} at 127.0.0.1:56420
I20260709 13:11:31.899019 28764 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:31.899581 28764 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:31.908264 29517 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:31.910142 29517 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:31.913991 29517 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:31.913846 28227 raft_consensus.cc:2955] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:31.928414 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:31.975549 28153 raft_consensus.cc:2955] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:31.989570 28157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:50460
I20260709 13:11:31.990254 28157 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:31.990623 28157 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:32.020143 27982 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:32.021894 27982 catalog_manager.cc:5697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 2 to 3, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:32.024190 28157 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:32.037315 28764 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:32.027593 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:32.027379 28078 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:32.038662 28157 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:32.053385 28764 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:32.053627 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:32.054234 28235 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:32.056007 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:32.056684 28078 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:32.057010 28078 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:32.057719 28078 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:32.059036 28078 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:32.070684 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:32.070971 28235 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:32.071372 28233 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:32.071689 29607 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:32.074956 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:32.076471 28232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:32.077090 28232 raft_consensus.cc:606] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:32.084132 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:32.085641 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:32.096654 28078 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:32.114195 28233 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:32.120177 28079 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:32.099309 29653 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:32.123540 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:11:32.113701 28228 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:32.120651 28079 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:32.126677 28079 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:32.127693 28079 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:32.130100 28227 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:32.130581 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:32.133090 29653 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:32.134104 29653 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:32.145758 28079 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:32.154815 28199 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:32.156468 29617 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:32.156638 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:32.157640 29615 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:32.158183 29615 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:32.159055 29615 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:32.165227 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51672.6us
I20260709 13:11:32.167022 29686 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:32.170115 29607 raft_consensus.cc:2955] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:32.174481 28078 raft_consensus.cc:2955] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:11:32.164146 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 67948.7us
I20260709 13:11:32.184230 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:32.229745 29661 raft_consensus.cc:993] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:32.230340 29661 raft_consensus.cc:1081] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:11:32.248073 28764 raft_consensus.cc:2955] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:11:32.261169 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49013.8us
I20260709 13:11:32.265640 28159 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:32.273885 28079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:40992
I20260709 13:11:32.274616 28079 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:32.275009 28079 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:32.286646 28078 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:32.296412 28764 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:32.302575 28159 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:11:32.307355 29635 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:32.314998 27983 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 2 to 5, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:32.322825 29572 raft_consensus.cc:2955] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:32.327687 28764 raft_consensus.cc:2955] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:32.317201 27983 catalog_manager.cc:5697] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:32.348775 27984 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:32.373944 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6f5f1be456de4894af76dffdad9ea6a3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:11:32.379776 28050 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.26.83.254:42913, user_credentials={real_user=slave}} blocked reactor thread for 77559.1us
I20260709 13:11:32.386996 28079 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:32.391534 28079 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:32.395936 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:32.396612 28233 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:32.396970 28233 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:32.397702 28233 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:32.398718 28233 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:32.417464 29635 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:32.419173 27984 catalog_manager.cc:5697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New cstate: current_term: 1 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:32.425922 28077 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:32.436517 28233 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:32.430974 28156 raft_consensus.cc:2955] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:32.427380 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:32.442381 28159 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:32.441505 28235 raft_consensus.cc:2955] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:32.442747 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:32.446663 29572 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:32.447409 29572 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:32.451552 29572 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:32.450275 28159 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:32.469480 28216 tablet_service.cc:1558] Processing DeleteTablet for tablet e6b1c27f68974f4a9878c7c155cb43d2 with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
I20260709 13:11:32.496654 28156 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:32.508325 28764 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:32.511380 29550 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:32.520826 27982 catalog_manager.cc:5697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:32.522342 28223 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f5f1be456de4894af76dffdad9ea6a3 with delete_type TABLET_DATA_TOMBSTONED (TS 11f10c12c03e47dcac88b050d4c1431b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39684
I20260709 13:11:32.524051 29696 tablet_replica.cc:333] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:11:32.530404 29631 raft_consensus.cc:2955] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:32.534821 29696 raft_consensus.cc:2243] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.546494 29696 raft_consensus.cc:2272] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.549183 28764 raft_consensus.cc:2955] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:32.550443 29696 ts_tablet_manager.cc:1916] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:32.578749 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53c6cdf8c6704ee59d0a736273582a48 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:32.584583 28235 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:32.590168 29607 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:32.647372 28233 raft_consensus.cc:2955] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:11:33.030498 28786 debug-util.cc:398] Leaking SignalData structure 0x7b0800247860 after lost signal to thread 26962
W20260709 13:11:33.031836 28786 debug-util.cc:398] Leaking SignalData structure 0x7b08002fe1e0 after lost signal to thread 28018
W20260709 13:11:33.033885 28786 debug-util.cc:398] Leaking SignalData structure 0x7b08002f6d80 after lost signal to thread 28110
W20260709 13:11:33.035352 28786 debug-util.cc:398] Leaking SignalData structure 0x7b08002f6520 after lost signal to thread 28185
W20260709 13:11:33.037493 28786 debug-util.cc:398] Leaking SignalData structure 0x7b080036f100 after lost signal to thread 28259
W20260709 13:11:33.039310 28786 debug-util.cc:398] Leaking SignalData structure 0x7b080034ff80 after lost signal to thread 28789
W20260709 13:11:33.040619 28786 debug-util.cc:398] Leaking SignalData structure 0x7b080034ffa0 after lost signal to thread 29623
W20260709 13:11:34.526460 28107 debug-util.cc:398] Leaking SignalData structure 0x7b08001dbf60 after lost signal to thread 26962
W20260709 13:11:34.601388 28051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.004s	sys 0.009s
W20260709 13:11:34.601730 28127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.017s	sys 0.003s
W20260709 13:11:34.602370 28127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.017s	sys 0.003s
W20260709 13:11:34.602398 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.008s	sys 0.004s
W20260709 13:11:34.603458 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.009s	sys 0.004s
I20260709 13:11:34.603542 29704 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.615389 29704 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:34.613787 28730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.894s	user 0.008s	sys 0.015s
W20260709 13:11:34.601852 28051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.004s	sys 0.009s
W20260709 13:11:34.618381 28730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.898s	user 0.009s	sys 0.016s
I20260709 13:11:34.630155 29702 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.653342 29702 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.667385 29704 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.673242 29704 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.673733 29705 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.674645 29704 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.674981 29705 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.673889 29710 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:34.677111 29710 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.685442 29704 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.693092 29710 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.687458 29711 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:34.698007 29711 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.701460 29702 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.711651 29711 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.729526 29705 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.748893 29715 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.751708 29704 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.749703 29715 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.736027 29710 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.763890 29717 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:34.765153 29707 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.764928 29717 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.752625 29704 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.753362 29705 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.754704 29711 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:34.766103 29707 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.769119 29706 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.790946 29712 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:34.797242 28728 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.050s	user 0.009s	sys 0.016s
I20260709 13:11:34.779377 29707 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.793337 29715 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.783141 29710 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.782452 29709 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.795625 29705 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.816033 29709 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.818115 29705 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.818598 29709 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.796352 29714 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.819273 29707 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.819536 29714 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.775671 29702 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.796792 29718 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:34.800350 29713 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.802641 29704 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.823598 29714 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:34.801306 28728 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.054s	user 0.010s	sys 0.016s
I20260709 13:11:34.800992 29711 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.797655 29706 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.810225 29719 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.823705 29702 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.819963 29707 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.791671 29708 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.822144 29713 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.797959 29712 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.806123 29722 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.815032 29710 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.828199 29718 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.828676 29719 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:34.857290 28233 consensus_meta.cc:287] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 2.206s	user 0.013s	sys 0.022s
I20260709 13:11:34.824319 29594 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.842432 29714 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.870939 29714 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.851684 29708 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.874639 29708 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.855393 29713 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.846132 29706 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.838431 29720 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.876574 29720 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.879307 29722 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.881892 29720 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.882341 29722 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.882493 29720 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.883347 29720 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.883898 29717 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.885931 29712 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.886507 29712 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.886696 29720 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.887281 29712 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.888957 29711 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.890180 29702 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.865731 29718 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.867621 29719 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:34.869047 29724 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.868582 29594 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.898716 29723 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.913061 29594 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.916072 29714 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.919555 28078 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:34.911202 29709 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.904162 29721 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.899840 29723 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:33.784818 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:34.927634 29723 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.921968 29707 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.923734 29724 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.923599 29721 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.922992 29709 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.955511 29709 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.941835 29712 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.942653 27982 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 266) took 2436 ms. Trace:
I20260709 13:11:34.956315 29707 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.957199 29707 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.960347 29707 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.956539 27982 rpcz_store.cc:276] 0709 13:11:32.506179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.509185 (+  3006us) service_pool.cc:224] Handling call
0709 13:11:32.545148 (+ 35963us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:32.545180 (+    32us) write_op.cc:454] Released schema lock
0709 13:11:34.942286 (+2397106us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:32.535180 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:32.535682 (+   502us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:32.535731 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:11:32.535749 (+    18us) tablet.cc:662] Decoding operations
0709 13:11:32.536069 (+   320us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:32.536128 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:32.536155 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:11:32.536335 (+   180us) tablet.cc:701] Row locks acquired
0709 13:11:32.536348 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:11:32.536488 (+   140us) write_op.cc:270] Start()
0709 13:11:32.536594 (+   106us) write_op.cc:276] Timestamp: P: 1783602692536442 usec, L: 0
0709 13:11:32.536615 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:11:32.537220 (+   605us) log.cc:844] Serialized 541 byte log entry
0709 13:11:32.539029 (+  1809us) op_driver.cc:464] REPLICATION: finished
0709 13:11:32.539497 (+   468us) write_op.cc:301] APPLY: starting
0709 13:11:32.539570 (+    73us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:32.539663 (+    93us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:32.539749 (+    86us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:32.539975 (+   226us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:32.540092 (+   117us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:11:32.540120 (+    28us) write_op.cc:312] APPLY: finished
0709 13:11:32.540282 (+   162us) log.cc:844] Serialized 23 byte log entry
0709 13:11:32.540461 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:32.540534 (+    73us) write_op.cc:454] Released schema lock
0709 13:11:32.540574 (+    40us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":4547,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":289,"apply.run_cpu_time_us":1333,"apply.run_wall_time_us":1333,"num_ops":1,"prepare.queue_time_us":9907,"prepare.run_cpu_time_us":2268,"prepare.run_wall_time_us":2268,"replication_time_us":2301}]]}
I20260709 13:11:34.966336 29724 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.910188 29711 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.920816 29696 ts_tablet_manager.cc:1929] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:34.973181 29696 log.cc:1199] T e6b1c27f68974f4a9878c7c155cb43d2 P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/e6b1c27f68974f4a9878c7c155cb43d2
I20260709 13:11:34.975059 29696 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39684 (request call id 79) took 2506 ms. Trace:
I20260709 13:11:34.946233 29727 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.946830 29721 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.975528 29696 rpcz_store.cc:276] 0709 13:11:32.469048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.469283 (+   235us) service_pool.cc:224] Handling call
0709 13:11:32.523731 (+ 54448us) ts_tablet_manager.cc:1151] Deleting tablet e6b1c27f68974f4a9878c7c155cb43d2
0709 13:11:32.523788 (+    57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:11:32.591794 (+ 68006us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:34.920470 (+2328676us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:34.974895 (+ 54425us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":53661,"thread_start_us":104678,"threads_started":1}
W20260709 13:11:34.948873 28729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.112s	user 0.015s	sys 0.008s
W20260709 13:11:34.983500 28729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.147s	user 0.015s	sys 0.008s
I20260709 13:11:34.983676 29696 tablet_replica.cc:333] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:11:34.984495 29711 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.984941 29721 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.986129 29721 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:34.987687 29711 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:34.989023 29712 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.988535 29720 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:11:34.952383 28727 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.925s	user 0.011s	sys 0.011s
I20260709 13:11:34.952462 29730 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.952944 29704 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:34.940872 29716 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.989557 29721 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:34.951617 29714 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.993757 29583 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.993896 29729 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.994760 29594 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:11:34.998306 28727 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.973s	user 0.012s	sys 0.011s
I20260709 13:11:34.998184 29719 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.995380 29716 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.000319 29719 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.012526 29714 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:34.989959 29712 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.000211 29730 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.019582 29594 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.013904 29722 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.014454 29713 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:34.999274 29704 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.016109 29717 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.016229 29706 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.000830 29729 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.041350 29669 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.041699 29732 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.042115 29706 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.042599 29732 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.043869 29713 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.024055 29707 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:34.998968 29583 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.048413 29653 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.049072 29653 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.049889 29722 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.051134 29653 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.052268 29722 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.025857 29720 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.058175 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1243) took 2450 ms. Trace:
I20260709 13:11:35.058259 29736 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.059340 29731 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.061779 29720 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.058631 28233 rpcz_store.cc:276] 0709 13:11:32.593876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.593994 (+   118us) service_pool.cc:224] Handling call
0709 13:11:32.597891 (+  3897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:32.600027 (+  2136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:32.600051 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:32.604365 (+  4314us) log.cc:844] Serialized 501 byte log entry
0709 13:11:32.647258 (+ 42893us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:34.857984 (+2210726us) log.cc:844] Serialized 14 byte log entry
0709 13:11:35.019240 (+161256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.031824 (+ 12584us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:35.037202 (+  5378us) raft_consensus.cc:1685] finished
0709 13:11:35.037214 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:35.044307 (+  7093us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8314,"raft.queue_time_us":57381,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":208,"thread_start_us":78569,"threads_started":1,"wal-append.queue_time_us":79038}
I20260709 13:11:35.062083 29717 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.062530 29720 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.063100 29738 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.065284 29713 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.063177 29720 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.077114 29594 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.024698 29727 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.056811 29583 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.064599 29717 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.059175 29736 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.091996 29712 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.096449 29714 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.096894 29711 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.097469 29729 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.099778 29696 raft_consensus.cc:2243] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:35.100857 28107 debug-util.cc:398] Leaking SignalData structure 0x7b0800177bc0 after lost signal to thread 28018
I20260709 13:11:35.082721 29707 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:35.102432 28107 debug-util.cc:398] Leaking SignalData structure 0x7b08000c3880 after lost signal to thread 28110
I20260709 13:11:35.097240 29719 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.099949 29704 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.060164 29653 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.042166 29669 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.107913 29732 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.108546 29706 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.087226 29721 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.116988 29721 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.119730 29721 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.109238 29734 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.110131 29713 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.115083 29716 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.121263 29713 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.121022 27983 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 5 to 6, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:11:35.122211 29707 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.123067 29728 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.123594 29713 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.110654 29722 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.110795 27984 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 254) took 2573 ms. Trace:
I20260709 13:11:35.111545 29727 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.097833 29711 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:35.103945 28107 debug-util.cc:398] Leaking SignalData structure 0x7b080035daa0 after lost signal to thread 28111
I20260709 13:11:35.124831 27984 rpcz_store.cc:276] 0709 13:11:32.536832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.540929 (+  4097us) service_pool.cc:224] Handling call
0709 13:11:35.110603 (+2569674us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3}
I20260709 13:11:35.066108 29738 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.141080 29738 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.123714 29728 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.077683 29731 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.142123 29727 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.135102 29704 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.131068 29722 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.156138 29631 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.157032 29740 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.157172 29696 raft_consensus.cc:2272] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:35.136170 29653 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.166155 29631 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:35.161051 28107 debug-util.cc:398] Leaking SignalData structure 0x7b0800355de0 after lost signal to thread 28185
I20260709 13:11:35.153964 29716 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.170127 29711 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.170357 29741 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.176884 29727 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.177924 29742 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.179490 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.180939 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.158829 29720 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.193382 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.194487 29653 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.151688 29721 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.194947 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.195746 28078 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:35.195459 29721 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.187886 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.197029 28156 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:35.188282 28077 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:35.188459 29731 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.188782 29716 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.189357 29704 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.189607 29583 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:35.120807 29734 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.164292 29705 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.169234 29706 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.202970 28159 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:35.203851 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.203238 29704 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.203840 29736 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.191298 29712 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.196815 29742 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.202967 29727 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.217957 28158 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
W20260709 13:11:35.216115 28107 debug-util.cc:398] Leaking SignalData structure 0x7b080032afa0 after lost signal to thread 28259
I20260709 13:11:35.190438 29741 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.223078 29723 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.225618 29729 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.226421 29730 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.227255 29738 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.190124 29740 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.231235 29707 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.240379 29706 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.241889 29631 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.238775 29713 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:11:35.238063 28107 debug-util.cc:398] Leaking SignalData structure 0x7b0800338c80 after lost signal to thread 28260
I20260709 13:11:35.218592 29728 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.254698 29703 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.238075 29721 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.255131 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.260880 29745 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.260959 29707 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.245630 29702 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.237592 29705 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.265926 29747 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.266013 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.245807 29717 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.267850 29653 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.262389 29713 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.268646 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.273756 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.246099 29712 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.251223 29723 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.268126 29748 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.259318 29669 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.268005 28762 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:35.278154 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.256829 29703 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.280205 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.282253 29716 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.261756 29745 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.285660 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.287887 28760 raft_consensus.cc:1813] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Handling vote request from an unknown peer 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:35.288156 28764 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:35.286319 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.261142 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:35.283638 28107 debug-util.cc:398] Leaking SignalData structure 0x7b08002a36e0 after lost signal to thread 28789
I20260709 13:11:35.290968 28757 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
W20260709 13:11:35.291188 28107 debug-util.cc:398] Leaking SignalData structure 0x7b08002afa20 after lost signal to thread 28790
W20260709 13:11:35.291698 28107 debug-util.cc:398] Leaking SignalData structure 0x7b0800367460 after lost signal to thread 29635
I20260709 13:11:35.291814 29734 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.258247 29729 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.279210 29731 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.297603 28760 raft_consensus.cc:2410] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:11:35.307889 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.307889 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.309151 28763 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:35.269317 29738 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.280256 29748 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.297909 29722 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.333065 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.333456 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.333943 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.337621 28077 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:35.337968 28085 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:35.330068 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.327091 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.287272 29717 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.334810 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.335665 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.335995 29696 ts_tablet_manager.cc:1916] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:35.336388 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.332687 29705 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.332747 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.331591 29706 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.329770 29723 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.344559 28086 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:35.294543 29741 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.353538 29743 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.324386 29653 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.289359 29702 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.360441 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.360355 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.359795 29742 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.266708 29747 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.386025 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.387639 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.390895 28762 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:35.401036 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.385381 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.404371 28084 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:35.354907 29731 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.408378 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.409564 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.410210 28158 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:35.413560 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.399847 29742 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.421209 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.409606 29702 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.412364 29712 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.354185 29743 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.414424 29746 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.427030 28761 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:35.415287 29752 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.432117 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.418664 29713 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.433297 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.438259 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.437734 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.445158 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.447609 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.447981 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.448441 28158 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:35.449168 29740 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.432075 29752 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.453368 29737 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.428304 29745 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.436816 28758 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:35.457814 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.459157 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.459376 28085 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:35.464057 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.455264 29729 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.465436 28160 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:35.465750 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.467665 28077 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:35.448072 29747 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.443746 29742 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.472921 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.450217 29719 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.474773 28086 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:35.475446 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.477074 28156 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
W20260709 13:11:35.479524 29750 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:11:35.480293 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.480753 29744 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.481446 29744 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.481686 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.482519 29748 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.483160 29748 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.479564 29729 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.483964 29744 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.483763 29748 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.484206 29729 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.486351 29748 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.486954 29748 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.487586 29748 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.489341 29729 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.445165 29753 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.493429 29753 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.494321 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.495473 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.496086 28764 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:35.481855 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.466795 29738 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.469343 29707 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.496040 29753 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.518184 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.518792 28158 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:35.521564 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.522305 28084 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:35.525229 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.525947 28156 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:35.530441 29743 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.476871 29755 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.532783 29731 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.544699 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.506467 29737 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.533195 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.535766 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.539965 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.557723 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.542150 29653 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.558430 28086 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
I20260709 13:11:35.557911 28761 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:35.561064 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.561812 28760 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:35.568655 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.570248 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.570344 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.526265 29738 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.573294 28077 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:35.580915 29729 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.581646 29729 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.582079 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.533169 29719 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.603029 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.617100 29719 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.549093 29755 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.570631 29669 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.471225 29746 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.617360 29756 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:35.622502 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.624644 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.568907 29717 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.583801 29703 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:35.640308 29737 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.633390 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.627363 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:35.634285 28159 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:35.634488 29758 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.636843 29748 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.637856 29742 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.623854 29738 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.638509 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.639148 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.568118 29754 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.633515 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.641137 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.642294 29746 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.634359 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.651355 28762 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:35.655861 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.657189 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.666245 28124 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.648082 29758 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.656356 29734 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:35.675334 29734 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.675853 29734 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:35.676119 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.670985 29746 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.658371 29716 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:35.671967 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.671708 29754 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.677122 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.676021 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.681386 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.656627 29756 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.691587 28161 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:35.690359 29716 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.693545 29716 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:35.693668 29756 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.681634 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.695616 29746 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.660955 29669 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.682423 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.682469 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.683511 29705 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:35.677791 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:35.668292 29631 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:35.666494 29721 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:11:35.700212 29669 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.701159 29729 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.701936 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.702383 29704 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:35.698088 28158 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:35.687016 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.680444 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.686095 29696 ts_tablet_manager.cc:1929] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:11:35.700059 29631 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:35.704697 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.701056 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.699627 29746 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.704406 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.706311 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.704994 28728 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.699689 29705 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.713164 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.710462 28158 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
I20260709 13:11:35.708053 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:35.711834 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.716089 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.714895 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.702744 29752 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:35.720158 29754 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.708626 29631 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:35.714002 29696 log.cc:1199] T 6f5f1be456de4894af76dffdad9ea6a3 P 11f10c12c03e47dcac88b050d4c1431b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/wals/6f5f1be456de4894af76dffdad9ea6a3
I20260709 13:11:35.722071 29696 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39684 (request call id 80) took 3200 ms. Trace:
I20260709 13:11:35.722445 29696 rpcz_store.cc:276] 0709 13:11:32.521419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.522047 (+   628us) service_pool.cc:224] Handling call
0709 13:11:34.983084 (+2461037us) ts_tablet_manager.cc:1151] Deleting tablet 6f5f1be456de4894af76dffdad9ea6a3
0709 13:11:34.983203 (+   119us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:11:35.544397 (+561194us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:35.685940 (+141543us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:35.721993 (+ 36053us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":152025,"tablet-delete.queue_time_us":2460091}
I20260709 13:11:35.723778 29712 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:35.724233 29712 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.724779 29712 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:35.708518 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.708913 29717 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:35.728890 29741 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:35.729085 29717 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.729451 29741 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.729782 29717 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:35.709069 29653 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:35.730386 29653 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.730827 29653 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:35.732272 28124 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.741696 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.732000 29721 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:35.749984 29721 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.736649 29723 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:35.747539 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.750885 29723 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.751372 29723 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:35.715507 29705 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:35.745433 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.746415 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.729941 29741 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:35.753163 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.753278 28050 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:35.745436 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet e6b1c27f68974f4a9878c7c155cb43d2 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:35.757229 29721 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:35.713502 29760 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:35.702811 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 52190.5us
W20260709 13:11:35.762099 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 59960.3us
I20260709 13:11:35.764556 28125 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:35.765756 28124 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.766541 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.765596 28125 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:35.768118 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:35.739612 29752 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.743300 29727 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:35.772704 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:35.766640 29711 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:35.758858 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:35.776199 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.754431 29746 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:35.769661 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.765110 29703 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:35.764891 29756 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.781090 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:35.792703 28049 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:35.776485 29702 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:35.797916 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:35.770990 29727 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.776052 29711 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.780237 29755 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.809960 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.810856 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:35.788419 29703 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.770067 29707 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:11:35.789831 29751 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:35.789599 28152 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
I20260709 13:11:35.825666 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:35.833185 29747 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:35.808511 29711 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:35.805018 29702 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.833547 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.834273 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.835052 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.836246 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.806488 29752 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.836963 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.833630 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:35.858472 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 92937.2us
I20260709 13:11:35.805235 29739 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:35.774164 29760 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.805840 29727 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:35.860817 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:35.819732 29731 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:35.861541 29747 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.900915 29747 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:35.901512 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.902460 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:35.827778 29703 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:35.802304 29746 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:35.858536 29732 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:35.870694 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 103413us
I20260709 13:11:35.858937 29751 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:11:35.879782 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 98752.9us
I20260709 13:11:35.865798 29702 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:35.860545 29762 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:35.873490 29766 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:35.922420 29762 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.922514 29766 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.922991 29762 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:35.923130 29766 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:35.898607 29763 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:35.896685 28125 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:35.869884 29756 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:35.911685 29743 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:35.921576 29751 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:35.897706 29731 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.949306 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.904919 29745 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:35.925068 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:35.926054 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:35.941534 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:35.945889 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.947220 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.949280 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.949700 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.950101 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.961542 28231 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:35.962445 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.963284 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.963660 28231 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:35.963840 29704 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:35.964170 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.964484 29704 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.965381 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.966020 28231 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:35.966809 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.966892 29758 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:35.960467 28229 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
I20260709 13:11:35.967908 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.968256 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:35.968770 28231 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
I20260709 13:11:35.917443 29732 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:35.974449 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49501.9us
I20260709 13:11:35.980365 28233 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:35.965281 29704 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:35.989636 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:35.941982 29763 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:35.958434 29731 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:36.004590 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 214625us
W20260709 13:11:36.001870 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 90965.8us
I20260709 13:11:35.988101 29732 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:35.988376 29718 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:36.021426 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:36.028326 28728 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:36.037199 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.040478 28235 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:36.018599 29763 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:36.044684 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.026947 29730 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:36.049181 29769 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:35.978351 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 180559us
I20260709 13:11:36.030092 29718 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.060981 28050 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:36.023725 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:36.018589 29631 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.061228 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:36.043133 29760 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:36.090775 29772 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.043370 29739 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.060366 29718 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:36.111990 28230 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:36.112092 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.115093 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.115862 28228 raft_consensus.cc:1813] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Handling vote request from an unknown peer 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:36.116686 28228 raft_consensus.cc:2410] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:11:36.092741 29706 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:36.122608 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 201962us
I20260709 13:11:36.123467 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:36.124658 28124 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:36.105760 29739 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.139192 29767 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.139672 29767 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.140123 29767 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.118527 29772 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.123558 29706 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.146371 27983 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 183) took 3553 ms. Trace:
I20260709 13:11:36.129153 29730 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.151254 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.145196 29772 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.136055 29739 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.150388 29706 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.167380 28233 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:36.135217 29712 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.153071 29730 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.174186 29756 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:36.175312 29768 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:11:36.177953 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.178797 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.178894 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.179870 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.181113 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.157300 29738 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:36.164805 29752 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:36.148224 27983 rpcz_store.cc:276] 0709 13:11:32.582422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.590425 (+2008003us) service_pool.cc:224] Handling call
0709 13:11:36.136141 (+1545716us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:35.210325 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:35.211132 (+   807us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:35.211183 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:11:35.211203 (+    20us) tablet.cc:662] Decoding operations
0709 13:11:35.211661 (+   458us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:35.211763 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:35.211797 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:11:35.212133 (+   336us) tablet.cc:701] Row locks acquired
0709 13:11:35.212186 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:11:35.212378 (+   192us) write_op.cc:270] Start()
0709 13:11:35.212545 (+   167us) write_op.cc:276] Timestamp: P: 1783602695212308 usec, L: 0
0709 13:11:35.212592 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:11:35.213499 (+   907us) log.cc:844] Serialized 536 byte log entry
0709 13:11:35.690413 (+476914us) op_driver.cc:464] REPLICATION: finished
0709 13:11:35.913032 (+222619us) write_op.cc:301] APPLY: starting
0709 13:11:35.919826 (+  6794us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:35.919967 (+   141us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:35.920151 (+   184us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:35.920457 (+   306us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:35.920706 (+   249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:11:35.920780 (+    74us) write_op.cc:312] APPLY: finished
0709 13:11:35.921066 (+   286us) log.cc:844] Serialized 23 byte log entry
0709 13:11:35.921268 (+   202us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:35.921370 (+   102us) write_op.cc:454] Released schema lock
0709 13:11:35.921431 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:35.949115 (+ 27684us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:35.949151 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":385775,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":176071,"apply.run_cpu_time_us":2287,"apply.run_wall_time_us":46793,"num_ops":1,"prepare.queue_time_us":57668,"prepare.run_cpu_time_us":19668,"prepare.run_wall_time_us":20600,"raft.queue_time_us":105860,"raft.run_cpu_time_us":23060,"raft.run_wall_time_us":158703,"replication_time_us":447653,"thread_start_us":252373,"threads_started":4,"wal-append.queue_time_us":53313}]]}
I20260709 13:11:36.208971 29765 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:36.211400 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.212123 28761 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:36.214671 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 186405us
W20260709 13:11:36.218503 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 155458us
I20260709 13:11:36.219444 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:36.188033 29730 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:36.224241 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 245763us
I20260709 13:11:36.167670 29717 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.231112 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.233238 28079 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:36.249571 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:36.251643 29732 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.252100 29702 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.253399 28085 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:36.253815 29732 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:36.255282 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.256007 28229 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:36.258756 29631 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:36.259034 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.259791 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.260360 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.219637 29746 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:36.268968 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.278811 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.224329 29765 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.227769 29712 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.282631 28079 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:36.282239 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.289386 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.289876 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.294593 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:36.255491 29774 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:36.261993 29721 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:36.286849 29723 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.009s
I20260709 13:11:36.282092 29732 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:36.335817 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.336333 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:36.285552 29765 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:36.289217 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.339104 29723 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.343873 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.345399 29763 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.343114 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:36.357496 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:36.358911 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.335319 28076 tablet_service.cc:1558] Processing DeleteTablet for tablet 53c6cdf8c6704ee59d0a736273582a48 with delete_type TABLET_DATA_TOMBSTONED (TS 0faaace2b36c4bcebf2fa6835235e92b not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:40978
I20260709 13:11:36.361723 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.366758 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.390908 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.395659 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.338126 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:36.399010 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55980.6us
I20260709 13:11:36.400628 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:36.402339 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:36.403487 29745 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:36.404898 29775 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:36.406680 28083 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:36.409130 29745 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:36.409430 29763 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.413758 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.414783 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.422930 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:36.425119 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 160323us
I20260709 13:11:36.425709 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:36.426465 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.429244 28730 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:11:36.424799 29653 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.689s	user 0.020s	sys 0.003s
I20260709 13:11:36.437799 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:36.396957 28228 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:36.434983 29723 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.469046 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.433736 29743 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:36.487872 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:36.473929 28730 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:36.477932 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.486624 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.463599 29775 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:36.498958 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.500432 29731 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.501026 29702 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.502616 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:36.480103 29779 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:36.506922 29775 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:36.479664 29721 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:36.445847 29653 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.480893 29743 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:36.508715 28125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.010s	sys 0.014s
W20260709 13:11:36.525211 28125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.010s	sys 0.014s
W20260709 13:11:36.525847 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 584336us
I20260709 13:11:36.527907 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:36.529301 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.530952 29783 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:36.510905 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.531428 29783 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:36.484238 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:36.504899 28052 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.515s (SENT)
I20260709 13:11:36.498471 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:36.509052 29747 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.002s
I20260709 13:11:36.540561 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:36.531910 29783 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:36.532967 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:36.509188 29734 consensus_meta.cc:287] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.833s	user 0.010s	sys 0.004s
I20260709 13:11:36.509964 29779 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:36.533757 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:36.537088 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.508807 28759 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:36.528776 29763 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:36.544874 29734 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.545408 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.547842 28763 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:36.545132 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.577787 28227 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:36.581491 29731 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.545806 29747 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.584008 29747 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.546372 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:36.585039 28756 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:36.554934 28125 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:36.525449 29717 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:36.535503 29743 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:36.595423 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:36.564229 29703 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.614s	user 0.007s	sys 0.007s
I20260709 13:11:36.565037 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.601109 28228 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:36.573864 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.556512 29779 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:36.580564 29704 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
W20260709 13:11:36.559057 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 54187.7us
I20260709 13:11:36.586288 28078 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:36.571853 29766 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.007s
I20260709 13:11:36.572795 28161 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c654c99561449cb823642a1b85879ac"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:36.598762 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.599020 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.558434 29769 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.599773 29739 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.597527 28125 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:36.622310 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.622771 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:11:36.609074 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 60869.5us
W20260709 13:11:36.573639 29716 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.879s	user 0.011s	sys 0.004s
I20260709 13:11:36.611075 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.612632 28161 raft_consensus.cc:606] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:11:36.615568 29741 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.844s	user 0.012s	sys 0.007s
I20260709 13:11:36.616605 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:36.587378 29730 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.624980 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:36.630061 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:36.609488 29721 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.609771 29776 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.631728 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:36.639626 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:36.641800 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:36.642289 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.621979 29769 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.643356 28086 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:36.627177 29763 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:36.646157 28156 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:11:36.652979 29718 consensus_meta.cc:287] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.005s
I20260709 13:11:36.653609 29718 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.655017 28082 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:36.612501 29703 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.656528 29718 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.658704 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.659862 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.660233 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:36.640014 29776 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.670130 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:36.630509 29631 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:36.636245 29721 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.672344 28154 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:36.619977 29766 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.618425 29704 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.647994 29763 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:36.623842 29716 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.623965 28226 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:36.648237 29769 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.633628 29741 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.619275 29767 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.661661 29782 tablet_replica.cc:333] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:11:36.658066 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.659272 29773 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:36.854604 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.674546 28080 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:36.750319 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.729873 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:36.888602 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 228429us
W20260709 13:11:36.667217 29727 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.755s	user 0.015s	sys 0.000s
I20260709 13:11:36.902254 28231 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:36.906728 29747 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:36.907366 29747 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.908545 29779 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.909204 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.910897 29779 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.911715 28756 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:36.911578 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:36.912967 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.913153 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.913497 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.913770 28234 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:36.914127 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:36.914611 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:36.914786 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.915220 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:36.915764 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.917608 29730 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:36.918917 29711 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.057s	user 0.009s	sys 0.008s
I20260709 13:11:36.919291 29703 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.919377 29711 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.921928 29739 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.673033 29631 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:36.923574 29631 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:36.923728 29766 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:36.773825 29705 consensus_meta.cc:287] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.022s	user 0.013s	sys 0.011s
W20260709 13:11:36.862038 29706 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.000s
W20260709 13:11:36.863161 29762 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.940s	user 0.005s	sys 0.007s
I20260709 13:11:36.675261 29721 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.928915 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.673311 29776 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:36.931766 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 421869us
I20260709 13:11:36.891778 29763 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:36.730756 29774 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:36.952163 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.847033 29653 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:36.930678 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 305734us
I20260709 13:11:36.937481 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:36.942416 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:36.943241 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:36.950333 28755 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:36.831137 29765 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.930181 29705 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.902112 29773 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:36.855533 29734 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:36.950502 28159 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:36.960283 28078 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 2: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:11:36.932025 29772 consensus_meta.cc:287] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.766s	user 0.008s	sys 0.003s
I20260709 13:11:36.969141 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.969915 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:36.970490 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:36.970577 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:36.985317 28160 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:36.971947 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:36.988336 28153 raft_consensus.cc:2393] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 2: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:36.982059 28233 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:36.931602 29706 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:36.934171 29762 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.979810 29773 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:36.925796 29727 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:36.984069 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:36.972926 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:37.000860 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 57429.5us
W20260709 13:11:36.999930 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44997.2us
I20260709 13:11:37.010501 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 196) took 1302 ms. Trace:
I20260709 13:11:37.011358 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.011873 28081 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.012710 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:11:37.013471 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.011152 28153 rpcz_store.cc:276] 0709 13:11:35.704706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.704827 (+   121us) service_pool.cc:224] Handling call
0709 13:11:37.006941 (+1302114us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400255654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:37.007082 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2930585472}
W20260709 13:11:37.010663 29751 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.020s	user 0.008s	sys 0.008s
I20260709 13:11:37.007191 29747 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:36.986083 29772 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.036582 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:37.037593 28730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.015s	sys 0.013s
W20260709 13:11:37.037962 28730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.015s	sys 0.013s
I20260709 13:11:37.021453 29711 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:37.038702 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 536137us
I20260709 13:11:37.024349 29704 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.040035 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.044348 29778 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.044871 29778 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:37.045275 29778 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.045555 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1522) took 1378 ms. Trace:
I20260709 13:11:37.047580 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.028283 29751 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.054728 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:37.057719 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:37.059053 29784 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.059655 29784 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:37.060067 29784 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.041837 29743 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.065364 28763 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:37.051394 29767 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:37.069670 28077 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:37.069972 29705 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:37.071329 29631 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.046032 28159 rpcz_store.cc:276] 0709 13:11:35.666754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.668419 (+  1665us) service_pool.cc:224] Handling call
0709 13:11:35.684022 (+ 15603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.985174 (+1301152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:37.045237 (+ 60063us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540010a754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:37.045454 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2861913856}
I20260709 13:11:37.072283 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:37.074640 29721 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.075363 29741 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.079097 29721 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.079432 29772 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.065908 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.080549 29772 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:37.081051 29772 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:37.081534 29772 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:37.083098 29631 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:37.083413 29795 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.132s	user 0.002s	sys 0.000s
I20260709 13:11:37.084872 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:37.085943 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:37.092983 29801 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:37.093595 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 543) took 1301 ms. Trace:
I20260709 13:11:37.085233 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:37.053514 29716 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:37.094081 28755 rpcz_store.cc:276] 0709 13:11:35.792114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.799123 (+  7009us) service_pool.cc:224] Handling call
0709 13:11:35.833026 (+ 33903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.976369 (+1143343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:37.093302 (+116933us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54006c02d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:37.093508 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2517342080}
I20260709 13:11:37.102438 29751 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.105731 29743 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.107792 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.109046 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.109591 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.113149 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:11:37.065729 29783 consensus_meta.cc:287] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.000s
I20260709 13:11:37.118184 28760 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.115161 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.116432 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:37.113220 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.123252 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:37.126368 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.127660 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:37.129051 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.122515 29783 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.107712 29762 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:37.136339 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:37.140216 28758 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:37.140834 29800 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:37.141944 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:37.107247 29763 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.150372 28759 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:37.151017 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:37.159363 29782 raft_consensus.cc:2243] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:37.171213 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:37.171732 27971 catalog_manager.cc:5028] TS 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): tablet 6f5f1be456de4894af76dffdad9ea6a3 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:37.161039 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.161360 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:37.173492 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.173559 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:37.173864 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:37.174479 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:37.174929 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:37.175134 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.175601 28078 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:37.172542 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:37.177141 29776 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.177920 29763 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.179804 29776 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.165948 29727 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:37.163591 29706 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.181881 29773 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.182958 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.183818 28161 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 2: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:37.183936 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.184437 29773 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:37.184883 28153 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:37.185113 28159 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:11:37.186844 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 175552us
I20260709 13:11:37.142777 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.169637 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:37.180202 29772 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.200582 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.138628 29765 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:37.164465 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.194998 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.195226 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.195827 29799 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.196416 28079 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.196754 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.206748 28083 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:37.197059 28158 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 2: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:37.196837 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.202642 28085 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:37.202512 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.193279 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.209299 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.204030 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:11:37.210389 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 65998.2us
I20260709 13:11:37.211163 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.232545 28081 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:37.232689 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:37.233963 28161 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 2: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:37.135519 29784 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.239939 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.247321 28155 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 2: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:37.241015 28755 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:37.241345 28154 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:37.259464 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 1667) took 1105 ms. Trace:
I20260709 13:11:37.241755 28082 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:37.242576 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.242854 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.245802 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:37.197408 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:37.247924 28762 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:37.261979 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.240353 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:37.261350 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:37.258236 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.264096 28079 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:37.265146 28154 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.265483 28161 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:37.266440 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.266463 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.268517 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:37.269327 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 70629.2us
I20260709 13:11:37.270051 28083 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:37.263018 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:37.271265 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.272068 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:37.272686 28080 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:37.259933 28077 rpcz_store.cc:276] 0709 13:11:36.147614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.147763 (+   149us) service_pool.cc:224] Handling call
0709 13:11:36.151332 (+  3569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:37.070135 (+918803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:37.252430 (+182295us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b5400296dd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:37.252673 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2111055104}
I20260709 13:11:37.248267 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:37.278599 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.281483 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.283720 28153 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.258138 29706 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:11:37.285526 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.284685 29739 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.296540 29806 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:37.295729 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.281667 29782 raft_consensus.cc:2272] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:37.276769 29805 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:11:37.298745 29803 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:11:37.282521 29732 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.299850 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.304057 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.305898 28153 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 2: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:37.300642 29747 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:37.274751 29796 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:11:37.313051 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.313934 28156 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:37.315567 29743 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.318985 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.319880 28160 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:37.321875 28152 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:37.323446 29784 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.323987 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.327589 29769 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.329038 28157 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 2: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:37.329222 28156 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.329463 29772 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.304484 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.330806 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.278863 29774 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:11:37.279683 29776 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:11:37.335017 28153 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:37.337833 28157 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:37.336194 28161 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.339483 28157 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:37.342590 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 101490us
I20260709 13:11:37.348285 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.348815 28160 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:37.349871 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:37.351785 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.360816 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:37.345501 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.424569 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.426189 29774 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:37.426659 29774 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:37.427723 29774 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:37.428864 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.424813 29794 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:11:37.429873 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73020.5us
W20260709 13:11:37.430130 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 125745us
I20260709 13:11:37.379571 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:37.436134 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 75513.4us
W20260709 13:11:37.484366 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 44160us
W20260709 13:11:37.488372 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 141086us
W20260709 13:11:37.487872 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57023us
I20260709 13:11:37.492187 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.492615 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.431686 29653 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:37.330672 29778 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.333737 29771 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:11:37.493453 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.441519 29741 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.442073 29713 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:37.499629 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:37.501555 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.485033 29772 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:11:37.497915 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 57022.2us
I20260709 13:11:37.485638 29794 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:37.431033 29762 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:11:37.511646 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.512941 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.503065 29767 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.516932 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:37.518124 29711 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:37.518504 29730 raft_consensus.cc:993] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:37.518676 29711 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:37.519059 29730 raft_consensus.cc:1081] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:11:37.519220 29711 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:37.519804 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:37.521167 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.502826 29713 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:37.516379 29745 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.514582 29739 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.530210 28127 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:37.536092 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:37.524537 29809 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:11:37.514878 29712 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:37.523586 29713 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:37.533174 29734 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.551369 29798 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:37.551934 29798 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:37.551985 29783 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.552410 29798 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:37.556777 29718 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:37.557402 29808 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:37.557511 29718 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:37.557871 29808 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.524992 29702 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.568971 28161 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.539690 29743 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.540287 29799 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.544876 29765 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:37.578101 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 63906.5us
I20260709 13:11:37.579931 28154 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:37.573210 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 79875.1us
I20260709 13:11:37.581480 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:37.582428 29702 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:37.582617 28229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:37.583010 29702 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:37.583518 29702 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:37.583381 29808 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.584687 29808 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:37.528028 29780 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.226s
I20260709 13:11:37.585171 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.583377 28229 raft_consensus.cc:606] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:37.586236 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.586787 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.585657 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.589881 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.595305 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.518162 29794 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.597685 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.597146 29767 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:37.599570 29804 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.315s	user 0.000s	sys 0.002s
I20260709 13:11:37.601374 29766 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:37.601715 29766 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:37.602592 29766 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.604470 29778 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.604377 28052 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.605041 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.605347 29767 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.528636 29785 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.154s
W20260709 13:11:37.607424 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 105942us
I20260709 13:11:37.619045 28758 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:37.620460 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.605865 29782 ts_tablet_manager.cc:1916] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:37.620899 28762 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.621562 28153 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:37.616371 29785 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:37.609218 29718 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.623687 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.622916 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.631361 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.605798 29769 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.635133 29747 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.635545 29734 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:37.624207 29803 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.643404 29803 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.644575 29803 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.645120 29767 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:37.645923 29803 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.638983 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.668463 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.638343 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:37.638486 29778 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.645975 29718 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:37.634151 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.672962 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.647980 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.649068 29765 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:37.675110 29765 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.650367 29746 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:37.675488 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.675774 29746 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.676996 29746 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.677444 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.678170 29746 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:37.650239 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.650599 29800 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:37.680500 29800 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.680758 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.681445 28229 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:37.681605 29800 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.682997 29800 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:37.646282 29751 raft_consensus.cc:3055] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.657189 28762 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.624591 29785 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:37.666208 28156 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.686782 29785 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.687480 28757 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.686772 28155 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:37.671730 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.671844 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.673033 29801 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.684621 29751 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:37.655313 29702 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.670107 29767 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.689932 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.697830 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.701373 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.689044 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.707543 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.707865 28125 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:37.709487 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.709800 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.710426 28234 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.710368 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.675547 29765 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:37.711784 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.714670 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.672199 29711 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.704448 29785 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:37.703995 29747 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.724603 29716 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.708413 29723 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.714170 29631 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:11:37.714665 29730 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:11:37.717528 28198 consensus_peers.cc:597] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 030bc6df8e7e4f2da3b18613d20d9bdc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:37.699129 29751 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.729506 29751 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.731230 29631 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.731729 29631 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:37.732668 29631 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.709386 29734 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:11:37.716570 29767 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.734952 29734 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.735531 29734 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:37.708577 28228 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:37.716208 29721 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.699523 29713 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.685015 29798 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.715620 29771 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.733274 29765 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.712281 29739 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:37.747807 28759 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:37.736963 28086 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
W20260709 13:11:37.748157 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 65624.2us
W20260709 13:11:37.747624 29762 consensus_queue.cc:1175] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "70996712c3c942468a6006602de6477f" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20260709 13:11:37.758502 29798 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.759764 28158 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:37.758780 29765 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.760982 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.761889 28159 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:37.764040 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:37.767457 29766 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:37.767838 29631 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:37.771159 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.771904 28229 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:37.772516 28234 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:37.736477 29734 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.781589 29734 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.781765 29732 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.782308 29732 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:37.785430 29819 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:11:37.782496 29812 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:37.787930 29812 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:37.788367 29812 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:37.789515 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 80069.5us
I20260709 13:11:37.791827 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.792423 29815 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.782613 29774 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:37.794183 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.793967 29784 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:37.796629 29820 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.798135 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.800568 28228 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:37.801062 29747 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.802402 28758 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:37.805061 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.808000 29806 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.808416 29806 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.805451 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.810873 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.811017 29809 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:37.825660 29809 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.826306 29713 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.826951 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.828838 29723 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:11:37.829687 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.830952 29794 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:37.809355 29806 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.834384 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:37.835150 28085 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:37.812166 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.836937 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.796681 29773 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:37.798604 29815 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.845988 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:37.846972 29704 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:37.847244 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:37.858489 28227 raft_consensus.cc:1813] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Handling vote request from an unknown peer 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:37.847808 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:37.859263 28226 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:37.859704 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.859928 29704 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:37.860054 29809 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.860460 29819 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.860947 28227 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:37.848003 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.848419 29780 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:37.796892 29732 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.846997 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.843123 29723 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.852809 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:37.789402 29775 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:11:37.797676 29702 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:37.857103 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.860944 29704 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.848640 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:37.863294 29796 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:37.879937 29801 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:37.880327 29801 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:37.869486 28161 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:37.798362 29803 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:11:37.863696 28759 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:37.874008 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.858023 29746 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:37.876427 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.887683 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.877355 29806 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.877310 29821 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:11:37.877931 29805 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.893786 29805 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.878353 29732 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.881125 29801 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.883623 29745 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:37.894829 29805 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.901528 29745 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:37.901996 29805 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.900276 29723 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:11:37.902525 29745 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.881747 29819 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:11:37.893199 29775 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:37.903050 29816 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:37.873052 29780 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.890450 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.902441 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.858680 29773 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.888329 29704 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:37.912456 29808 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:37.887363 29809 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.916164 29741 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.916237 29817 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:37.918689 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.920567 29775 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:37.897258 29784 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:37.921530 29819 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.921959 29816 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:37.889330 29818 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:11:37.924919 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:37.927461 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.929315 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:37.932089 29746 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.932540 29746 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.911116 29711 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:37.933019 29784 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:37.935348 29746 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.934832 29706 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:37.938026 29746 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.937767 29784 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.939018 29818 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:37.939448 29818 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.937384 29810 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:37.926301 29702 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:37.930302 29723 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:37.934094 29801 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:37.948426 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.936004 29739 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:37.930513 29819 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.950937 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:37.933840 29817 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.929248 29803 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:11:37.958535 29814 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:37.945672 29702 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:37.947299 29810 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:37.940492 29775 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:37.961462 29780 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:37.963523 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:37.964265 28161 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 2: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:37.952560 29739 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:37.942731 29821 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:11:37.975841 29739 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:37.977656 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.954788 29809 consensus_queue.cc:1059] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:11:37.963250 29702 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:37.965500 29814 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:37.983793 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:37.987669 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 275960us
I20260709 13:11:37.940404 29818 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.988384 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:11:37.990530 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.967223 29819 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:37.993201 29821 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:38.002542 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:38.005815 29762 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.001336 29734 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.010303 29822 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:38.013753 29739 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:37.961926 29810 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.022504 29814 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:38.001531 29821 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:11:38.024922 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.993793 29817 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.030560 29817 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:38.033221 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:38.038743 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:38.011078 29822 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.025985 29814 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:38.040007 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:38.042078 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:38.042526 28085 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:38.029210 29818 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:37.988549 29812 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.043622 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:38.033221 29819 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:38.029023 29773 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:38.040723 29810 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:38.052656 29734 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.062230 28083 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:38.042640 29821 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:38.067149 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.065528 28228 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:38.055832 29762 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:11:38.070674 29702 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.071894 28081 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:38.073480 29821 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:38.073762 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:38.079376 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.023409 29816 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.081830 29816 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:38.094882 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.097031 29812 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:38.098333 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.099113 29732 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:38.100050 29780 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:38.100495 29780 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:38.100901 29780 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:38.102164 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.104310 28228 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:38.109944 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.104501 29702 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:38.125990 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:38.145608 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 194709us
I20260709 13:11:38.146528 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:38.149896 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:38.113149 29773 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:11:38.155242 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:38.156874 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:38.158203 28083 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:38.083487 29814 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:38.153007 29823 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:11:38.164994 29809 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:38.175983 29809 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:38.176376 29809 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:38.162647 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:38.177839 28083 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:38.159487 28158 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:38.155036 29800 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.180205 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:38.181655 29762 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.183480 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:38.184908 28159 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:38.153218 29812 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:38.186187 29812 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:38.186574 29812 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:38.191871 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:38.193048 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:38.194715 28080 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
W20260709 13:11:38.196389 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.198135 28081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:38.198803 28081 raft_consensus.cc:606] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.199150 28081 raft_consensus.cc:614] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:38.201179 29794 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.201887 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:11:38.202262 29798 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.202363 29808 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:38.202847 29808 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:38.203552 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:11:38.206975 29826 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.001s
I20260709 13:11:38.203930 28228 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:38.212950 29814 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:38.214165 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.221625 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:38.222466 28763 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:38.223999 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.225320 29717 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:38.225720 29717 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:38.226131 29717 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:38.231570 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:38.232682 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.219620 29739 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:38.234925 29739 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:38.229367 29822 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:38.241190 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:38.244724 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 79416.3us
I20260709 13:11:38.203701 29808 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.247253 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:38.249527 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.236478 29739 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.253875 29631 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:38.254292 29631 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:11:38.247169 29824 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.218s	user 0.006s	sys 0.000s
I20260709 13:11:38.268600 28125 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:38.242941 29812 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.250965 29818 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.269358 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.249290 29806 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.257047 29631 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.270414 29711 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:38.276186 29711 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.277102 29711 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.279084 29782 ts_tablet_manager.cc:1929] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:11:38.273115 29814 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.281080 29631 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:38.270620 29808 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:38.283288 29812 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:38.294122 29825 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:38.294620 29825 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.279562 29782 log.cc:1199] T 53c6cdf8c6704ee59d0a736273582a48 P 0faaace2b36c4bcebf2fa6835235e92b: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/wals/53c6cdf8c6704ee59d0a736273582a48
I20260709 13:11:38.295499 29825 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.297596 28082 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:38.246071 29717 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.314325 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:38.314778 29782 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40978 (request call id 84) took 2163 ms. Trace:
I20260709 13:11:38.315378 29773 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:38.315924 29773 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.315163 29782 rpcz_store.cc:276] 0709 13:11:36.150938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.172031 (+ 21093us) service_pool.cc:224] Handling call
0709 13:11:36.624617 (+452586us) ts_tablet_manager.cc:1151] Deleting tablet 53c6cdf8c6704ee59d0a736273582a48
0709 13:11:36.630380 (+  5763us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:11:38.165556 (+1535176us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:38.278931 (+113375us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:38.314668 (+ 35737us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":247281,"tablet-delete.queue_time_us":214171,"thread_start_us":73176,"threads_started":1}
I20260709 13:11:38.318784 27969 catalog_manager.cc:5028] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): tablet 53c6cdf8c6704ee59d0a736273582a48 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:11:38.319217 29773 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.323655 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:38.306294 27984 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 3 to 4, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 4 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:38.331208 28228 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:38.336805 29806 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.320007 29780 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.338207 29772 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:38.338272 27984 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 3 to 4, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 4 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:38.339852 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:38.341624 29825 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:38.342036 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.340204 27984 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:38.346880 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.349226 28158 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:11:38.350517 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:38.348937 27983 catalog_manager.cc:5697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:38.347399 29806 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.357045 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.353638 29810 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:38.355196 29809 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.358639 29772 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:38.356128 29833 ts_tablet_manager.cc:933] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:38.362202 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:38.362671 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.363430 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:38.364995 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.350209 29812 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.366225 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.370458 28082 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:38.371845 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:38.369292 29819 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.364761 29810 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.366479 29814 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.374755 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:38.373764 29812 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.374845 29717 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:38.377193 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:38.377986 28161 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:38.374557 29808 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.375671 29772 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:38.375720 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:38.383476 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.386451 28763 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:38.381345 27982 catalog_manager.cc:5697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:38.386067 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.388378 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.389415 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.390599 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.385891 29814 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.398224 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:38.399093 28161 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 2: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:38.402590 29810 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.389575 29772 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:38.406765 29772 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.408877 29812 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.409915 29806 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.385273 29780 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:38.386202 29808 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:38.416908 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.417065 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.416982 29747 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:38.418834 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.420048 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.420639 28229 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:38.421584 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.422704 29809 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:38.422854 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.423316 29809 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:38.424279 29809 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:38.421584 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.429322 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.429327 28231 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:38.432081 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.432020 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.433413 28230 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:38.435253 29747 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:38.432220 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.436285 29747 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:38.439751 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.435245 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.442401 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.448112 28235 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:38.451097 28233 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:38.452176 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.454953 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.456365 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.457056 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.463605 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.464915 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.467228 28230 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:38.468104 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.470556 28227 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:38.471762 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.472424 28227 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:38.453019 28233 raft_consensus.cc:2376] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2. Re-sending same reply.
I20260709 13:11:38.468592 28230 raft_consensus.cc:2376] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2. Re-sending same reply.
I20260709 13:11:38.474135 28232 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:38.474689 28229 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:38.476212 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.424614 29809 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:11:38.479960 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.482436 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.482348 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.483605 28230 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:38.486665 28229 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:11:38.473717 29833 tablet_copy_client.cc:323] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:11:38.486943 28234 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:38.490655 28245 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet 030bc6df8e7e4f2da3b18613d20d9bdc from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:11:38.491472 28245 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet 030bc6df8e7e4f2da3b18613d20d9bdc from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-030bc6df8e7e4f2da3b18613d20d9bdc
I20260709 13:11:38.497860 29809 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:38.499467 29809 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:38.478891 28232 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c654c99561449cb823642a1b85879ac"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39698
I20260709 13:11:38.508360 28232 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:38.508858 28232 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:38.509135 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.512962 28226 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:38.514413 28230 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:38.521670 28231 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:38.525074 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:38.550717 28227 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:38.527815 28161 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:38.532224 28159 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:11:38.543658 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.555018 28233 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:38.555616 29780 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:38.545264 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.556260 28233 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:38.557741 28234 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:38.545877 28245 tablet_copy_source_session.cc:215] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:38.547441 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:38.561106 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:38.561487 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:11:38.548394 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.554623 28231 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:38.563690 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:38.563961 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:38.564348 28152 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:38.565127 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:38.565270 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:38.566036 28232 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.566841 28235 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:11:38.568528 29833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 030bc6df8e7e4f2da3b18613d20d9bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:38.568831 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:38.570709 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:38.545691 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:38.569394 28230 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:38.573038 28232 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:38.573989 29773 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:38.574566 29773 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.569077 29825 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.569761 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:38.575475 29773 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.555750 29784 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:38.581794 28229 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:11:38.599516 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.583269 29785 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:11:38.619293 29784 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:11:38.648803 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.656811 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:38.658517 29631 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:38.705715 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49010.7us
I20260709 13:11:38.716069 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:38.718082 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:38.724480 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:38.725853 29763 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:38.724382 29825 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:38.727977 29631 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.741401 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.736720 29763 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.739336 29825 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.748200 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.754532 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:38.754853 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:38.755206 28158 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:38.755595 28158 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.744796 29808 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:38.756278 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:38.757513 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:38.758952 29773 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:38.757356 29631 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.759618 29773 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.760576 29773 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.761718 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:38.762283 28080 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:38.763684 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:38.758162 29711 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:38.746008 29814 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:11:38.780351 29808 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:38.780798 29808 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:38.780866 29763 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.781688 29808 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.783183 29808 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:38.786657 28158 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.791049 28080 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
W20260709 13:11:38.793715 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44556.4us
I20260709 13:11:38.794828 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:38.799935 28158 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:38.802264 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.776492 29825 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.806350 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:38.807883 29711 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.808043 29857 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:38.808885 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47808.2us
W20260709 13:11:38.828760 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.796602 29814 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.812801 28127 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:38.811590 29812 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.844900 29711 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.806535 29806 raft_consensus.cc:2764] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:11:38.873703 28232 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:38.833628 29747 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.874485 29833 tablet_copy_client.cc:806] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:38.857854 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:38.861939 28082 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:38.866641 29857 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:38.855919 28763 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:38.875996 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:38.814033 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:38.880329 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66395.4us
I20260709 13:11:38.881130 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.832615 29853 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:38.884474 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:38.882323 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:38.874414 29747 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.878150 29631 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.903055 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:38.883231 29809 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.882711 29806 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:38.886106 29833 tablet_copy_client.cc:670] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:38.883147 29767 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:11:38.905423 29856 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:38.888833 29857 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.886716 29780 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:11:38.910725 29747 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:38.892590 29814 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:38.912914 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:38.891335 29820 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:11:38.888510 29773 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:38.915923 28158 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:38.937440 28232 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:38.945426 29857 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:38.946488 28158 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:38.926008 29812 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:38.950630 29812 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:38.954653 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:38.955381 28158 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:38.965907 28227 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:38.919803 29631 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:11:38.974578 29853 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:38.975750 29853 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:38.986394 28080 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:38.991412 28232 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:38.958076 29747 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.929049 29711 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:11:39.013943 29765 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:11:39.015892 29711 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:39.016331 29711 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:39.017235 29711 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:39.012931 29747 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.020287 29833 tablet_copy_client.cc:538] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:39.028564 28227 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:39.061383 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:39.062318 28229 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:39.063850 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:39.070384 29855 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.071218 29855 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:39.073750 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 99699.5us
I20260709 13:11:39.055773 27984 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 2, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 2 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:39.044601 29773 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:39.001683 29812 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:39.034557 29631 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:11:39.080479 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:39.072430 29767 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:39.093340 29856 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:39.089802 29767 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:39.101233 29706 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:39.101774 29706 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:39.092099 29765 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:11:39.102180 29706 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:39.083590 29820 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:11:39.099798 29767 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:39.111313 29773 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:39.116699 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 114293us
I20260709 13:11:39.095695 29856 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:39.135807 28159 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:39.138789 28081 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:39.140421 29858 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:39.140964 29858 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:39.141971 29858 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:39.143500 29858 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:39.144512 29871 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.146487 28082 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:39.155115 29855 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:39.145113 29871 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.158116 29871 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:39.166268 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 85870.5us
I20260709 13:11:39.164887 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:39.167953 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:39.168129 28230 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:39.170174 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:39.170888 28228 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:39.159483 29631 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:11:39.169637 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:39.168774 28082 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:39.178418 28230 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:39.177170 28161 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:39.181305 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:39.215190 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:39.215013 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:39.195154 29772 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:39.187075 29631 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:11:39.226786 29745 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:39.249377 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:39.184413 29872 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260709 13:11:39.222824 29801 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:11:39.237186 29854 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:39.250278 28763 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:39.241793 29871 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.274333 29706 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.245271 29745 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:39.270090 29854 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:39.293152 29854 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:39.266435 29864 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.297868 27982 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 3, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 3 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:39.271010 29808 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.242266 29772 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.297999 29745 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:39.291036 29871 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:39.330127 29878 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:11:39.330686 29876 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:39.273517 29767 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.320525 29864 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.339210 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:39.340087 28763 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:11:39.337087 29706 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.344704 29833 tablet_bootstrap.cc:492] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:11:39.337805 29876 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.324882 29808 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.363919 28763 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:39.361037 29765 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.365727 29765 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.366318 29864 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.368244 29765 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.373713 29747 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.374339 29876 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.378862 28230 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:39.379077 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:39.384207 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:39.391577 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:39.368494 29871 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:39.391119 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:39.400482 28233 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:11:39.418079 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:39.423056 28226 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:39.389475 29801 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.389740 29876 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.426843 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:39.418111 29706 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:39.399225 29880 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.421149 29882 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.439172 29864 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:39.450986 29854 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.439553 29808 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.429222 29825 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:11:39.444082 29785 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:39.474536 29881 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.240s	user 0.003s	sys 0.000s
I20260709 13:11:39.446857 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:39.476871 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:39.491605 28229 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:39.447844 29706 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:39.445240 29801 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.499455 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:39.464316 29820 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.445050 29876 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.464993 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:39.475994 29785 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:39.487031 28159 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:39.474521 29854 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:39.500797 28226 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:11:39.562029 28763 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:39.458806 29882 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.449456 29880 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.606492 28085 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
W20260709 13:11:39.561028 29840 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:11:39.537519 29772 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:39.635946 29896 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:11:39.641026 29767 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.659137 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:11:39.659899 29858 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.003s
I20260709 13:11:39.659972 28755 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:39.667577 29882 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.670181 29801 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:39.673697 28755 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:39.678344 29876 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.687071 29880 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.694876 28161 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:39.697963 29808 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:39.564575 28759 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:39.702041 29706 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.623041 28229 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:39.557735 29785 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:39.726140 28227 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:39.730247 28084 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:39.738147 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:39.739096 28161 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:39.740466 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:39.741390 28155 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 2: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:39.742096 28233 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:39.710392 29877 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.753878 29869 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.558466 29820 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.729773 29711 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:39.757296 28083 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.006s	sys 0.018s
I20260709 13:11:39.765786 28049 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:11:39.766189 28083 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.007s	sys 0.018s
W20260709 13:11:39.773386 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 288195us
I20260709 13:11:39.792986 28077 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:39.762972 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:39.805317 28083 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:39.747721 29745 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:39.791000 29771 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.035s	sys 0.069s
I20260709 13:11:39.773592 29882 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:11:39.818213 29771 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.035s	sys 0.070s
I20260709 13:11:39.763891 29877 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.762560 29869 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:39.812680 29772 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.893370 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:39.894415 28235 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:39.815685 29814 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:11:39.824329 29767 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:11:39.868333 29882 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:39.906701 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:39.865923 29857 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.909912 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:39.920488 28227 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:39.885664 29772 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:39.966831 29902 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.973240 29902 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:39.975620 29902 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:39.984421 28161 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:39.938412 29857 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:39.966473 29880 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.961825 29870 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:39.993749 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:39.987080 29833 tablet_bootstrap.cc:700] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Time spent opening tablet: real 0.219s	user 0.002s	sys 0.001s
I20260709 13:11:39.969288 29890 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.996590 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:39.997473 28757 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:39.996450 29890 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.002518 29890 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.003790 29785 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.005563 29814 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.005774 29869 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.006080 29814 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.006536 29814 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:39.989626 29882 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.007608 29857 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.008359 29745 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.009699 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.010521 29785 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.011274 28226 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:40.012004 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.012903 28758 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:40.013484 29907 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.013892 29907 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.014235 29907 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:39.995476 29820 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:39.995235 29880 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.019544 29880 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:39.984716 29767 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.994108 29870 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.025156 29767 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.025666 29870 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.027390 29767 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.006609 29812 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.028414 29812 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.030659 29812 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:40.031232 29812 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.031652 29812 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.046248 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.049844 29869 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.050305 29869 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.050671 29869 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.055048 29877 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.062902 28082 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:40.070668 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.071540 28080 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:40.080542 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.095145 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.096051 28082 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
W20260709 13:11:40.099779 28728 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.010s	sys 0.011s
W20260709 13:11:40.100569 28728 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.010s	sys 0.011s
I20260709 13:11:40.097533 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:40.078864 29877 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.111445 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.113035 28161 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:40.085767 29905 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:40.112953 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 203120us
I20260709 13:11:40.091003 29895 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:40.151955 29912 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:40.153635 29780 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.143646 29913 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.155304 29913 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.154320 29780 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.158313 29780 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.159837 29808 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.160609 29808 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.166469 29772 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.169202 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.170176 28755 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:40.175870 29833 tablet_bootstrap.cc:492] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:11:40.112914 29899 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.178633 29899 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.179896 29833 tablet_bootstrap.cc:492] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:11:40.180327 27983 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 2 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:40.158401 29913 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.059325 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:40.185496 28049 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:11:40.193218 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 90594.2us
I20260709 13:11:40.082755 29814 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.108172 29825 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.198104 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.149150 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:40.200596 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.108490 29767 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.203593 29870 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.204972 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:40.206784 29899 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.208864 29812 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.149302 29905 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.219273 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.225770 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.226166 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:40.201377 29895 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:40.117352 29877 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.228999 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.201382 29706 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.114883 29871 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.239441 28232 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:40.271992 28227 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
W20260709 13:11:40.273025 28234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.013s	sys 0.023s
I20260709 13:11:40.273680 28730 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.274937 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.245788 29771 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:40.276767 28127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.027s
W20260709 13:11:40.277247 28127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.008s	sys 0.027s
W20260709 13:11:40.240891 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 91854us
I20260709 13:11:40.192433 29833 ts_tablet_manager.cc:1403] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.848s	user 0.023s	sys 0.015s
I20260709 13:11:40.246526 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.192523 29808 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.253284 28763 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:40.258538 28235 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:40.258963 28228 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:11:40.282171 28764 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:40.247237 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.242480 29825 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:40.273567 28234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.013s	sys 0.023s
I20260709 13:11:40.279862 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.253013 28226 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:11:40.281863 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.306425 28229 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:40.284642 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.308266 28232 raft_consensus.cc:2410] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:40.262928 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.287461 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.309834 28230 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:40.310202 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:40.293939 28756 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:11:40.298794 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.275969 29706 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:40.277618 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 533965us
I20260709 13:11:40.303188 28227 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:40.282606 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.311664 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:40.313154 28235 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:40.261273 29785 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:40.327528 29785 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.328362 29880 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.328858 29880 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.329200 29880 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.323719 28755 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:40.332201 29785 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.281991 29771 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.334048 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:40.334632 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.335588 28235 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:40.335901 28228 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:40.337744 28730 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.322206 29833 raft_consensus.cc:359] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f [term 3 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.308537 28759 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:40.287395 29892 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:40.291810 29899 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.288022 29745 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.290474 29812 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:40.340344 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.300705 29772 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.301856 29808 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.255513 29767 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.347656 28233 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:40.334273 29706 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.360721 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.362872 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.288237 29871 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.325246 29916 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:40.339766 29771 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:40.344231 29883 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:40.343472 29785 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:40.374241 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.369413 28155 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:40.377652 28156 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:40.345610 29745 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:40.381000 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 283558us
I20260709 13:11:40.382855 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:40.385242 28161 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:40.386457 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 272097us
I20260709 13:11:40.388959 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.358572 29812 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.358487 29780 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.392994 28762 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:11:40.363166 29905 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.395887 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.374858 29784 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:40.399489 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:40.345335 29858 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:11:40.401800 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.349431 29899 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.387113 29745 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.426237 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.427376 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.393378 29812 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:40.377864 29864 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.383935 29785 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:40.340245 29907 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.441148 29877 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.441926 28082 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:11:40.442314 29917 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:11:40.442459 29747 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.442664 29907 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.346748 29892 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.443382 29785 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:40.443845 29876 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.444388 29876 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.445461 29892 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.430215 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.446285 29876 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.446581 28155 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:40.430604 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.354107 29869 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.449069 28160 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:40.449620 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:40.450088 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.392313 29885 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.450378 29869 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.360670 29808 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.450839 29885 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.452401 29808 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:40.452945 29808 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.453365 29808 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.379276 29883 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.454057 29814 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.454335 29883 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:40.455734 28083 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:40.377092 29916 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.456573 29916 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:40.438805 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 45714.6us
I20260709 13:11:40.404522 29780 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.377048 29763 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:11:40.461517 29908 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:40.461907 29780 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.462327 29763 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:40.440344 29864 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.443037 29747 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.428376 29925 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.465015 29925 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:40.465518 29825 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.465683 29747 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.450237 28756 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.438196 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.377612 29895 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.460951 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.435585 28153 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:40.472150 28759 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:40.440882 29857 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:40.468984 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:40.469194 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:40.482064 28081 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:40.482278 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:40.453529 29767 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.495939 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.496678 28078 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:40.472465 28161 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:40.474067 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:11:40.483261 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 80358.6us
I20260709 13:11:40.491870 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.499387 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:40.497264 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:40.497929 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:40.500283 28227 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:40.498255 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.501189 28228 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:40.434120 29784 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:40.455372 29808 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.434343 29928 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.511250 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.539142 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.539868 28228 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:40.542088 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.542858 28235 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:40.464195 29916 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.536428 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:40.547840 28231 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:40.493120 29921 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.059s	user 0.002s	sys 0.000s
I20260709 13:11:40.452945 29885 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.536078 29869 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:40.548251 28198 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:11:40.545691 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 84808.2us
W20260709 13:11:40.608541 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 60362.3us
W20260709 13:11:40.554198 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54890.6us
I20260709 13:11:40.439682 29833 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:11:40.558815 29825 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:40.504653 29870 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.566598 29880 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.630820 29772 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.631336 29772 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.541313 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.632591 28227 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:40.633859 28082 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:40.591569 29899 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.592151 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:40.592578 29784 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.594033 29864 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.594892 29871 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:40.618960 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.503499 29747 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.641458 29747 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.534382 29814 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.628114 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.630398 29825 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.558952 29892 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:40.650091 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.673978 28078 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:40.513155 29872 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.674791 29872 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.569016 29867 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.677006 29867 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.677417 29867 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.680193 28756 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:40.672744 29825 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:40.626907 29631 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.472004 29863 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.186s
I20260709 13:11:40.599131 29869 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:40.683342 29922 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.131s	user 0.000s	sys 0.002s
I20260709 13:11:40.683419 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:40.707459 28761 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:40.563230 29883 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.664713 29814 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.546978 29808 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:40.664141 29820 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:40.699132 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.699841 29773 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.705494 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.684199 29919 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:40.710309 28081 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:40.714479 28078 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:40.715982 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.718298 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.720654 28082 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:40.721808 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.722693 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.686916 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:40.688897 29871 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:40.742697 28086 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:40.727344 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.727963 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.750523 28762 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:40.750451 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.744549 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.742775 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.752945 28081 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:40.752822 28157 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:40.740960 29864 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.741277 28160 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:40.709779 29869 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:40.705346 29882 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.770362 29882 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.771600 28078 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:40.733264 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:40.690958 29890 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.773104 29763 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:40.773940 29890 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.754338 28231 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:40.782076 29872 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.783293 29745 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.783892 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1729) took 1077 ms. Trace:
I20260709 13:11:40.723824 29814 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.553319 29928 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.785431 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 252) took 1028 ms. Trace:
I20260709 13:11:40.788132 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.792606 29911 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.742535 29820 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.793048 29923 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:40.794222 29747 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.743829 29785 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.709671 29631 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.800310 28158 rpcz_store.cc:276] 0709 13:11:39.735294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.735499 (+   205us) service_pool.cc:224] Handling call
0709 13:11:39.735976 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:39.969087 (+233111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:39.969120 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:39.988944 (+ 19824us) log.cc:844] Serialized 26 byte log entry
0709 13:11:40.339816 (+350872us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:40.377058 (+ 37242us) log.cc:844] Serialized 14 byte log entry
0709 13:11:40.377278 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.377358 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:40.763823 (+386465us) raft_consensus.cc:1685] finished
0709 13:11:40.763842 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.763966 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50064,"prepare.queue_time_us":37140,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":63961,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":16842,"thread_start_us":522174,"threads_started":2,"wal-append.queue_time_us":616550}
I20260709 13:11:40.749583 29801 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.749820 29871 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.803985 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.870047 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.934818 28154 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:40.934942 28160 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:40.742379 29925 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:40.944977 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.736197 29808 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.870131 28757 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:40.872170 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.955534 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:40.789906 29929 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:11:40.946008 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.719702 29833 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:11:40.944504 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 138427us
W20260709 13:11:40.958772 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 215343us
I20260709 13:11:40.962541 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:40.809131 29913 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:40.802326 29882 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:40.878091 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.808530 28226 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:40.763293 29919 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:40.935478 29933 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.367s	user 0.007s	sys 0.002s
I20260709 13:11:40.809664 29930 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:40.749850 29773 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:41.031637 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 92998.4us
W20260709 13:11:40.976512 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 188428us
I20260709 13:11:41.035430 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:40.785553 29864 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.931548 29801 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.975128 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:41.041831 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.877152 29936 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:41.048950 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 87180.1us
I20260709 13:11:40.976179 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:41.045145 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 313171us
I20260709 13:11:41.077642 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:41.047641 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:41.037262 29899 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:41.055217 29833 ts_tablet_manager.cc:1434] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.765s	user 0.009s	sys 0.016s
I20260709 13:11:40.974912 28758 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:40.969954 29925 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:40.971426 29883 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.063853 29882 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:41.067111 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.945291 29867 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:40.930909 29911 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:40.946241 29871 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:41.122675 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.076411 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:41.124719 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:40.973413 29747 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:41.090031 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:41.037027 28152 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:41.058220 29930 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.095086 29876 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:41.070411 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:41.075256 29801 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:41.114986 28235 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:40.968432 29923 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.126981 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:41.143704 28227 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:41.128084 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:41.089174 29899 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.087002 29765 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:11:41.134279 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:41.092360 29833 ts_tablet_manager.cc:1467] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Tablet startup took 803ms
I20260709 13:11:41.045753 29913 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.096582 29858 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:11:41.140854 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 1710) took 1066 ms. Trace:
W20260709 13:11:41.172091 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 485223us
I20260709 13:11:41.139979 29905 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:41.146544 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:41.111466 29931 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:11:41.148139 28161 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:41.108830 29916 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.117614 29892 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:41.194798 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 108680us
W20260709 13:11:41.168919 29771 consensus_meta.cc:287] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.780s	user 0.013s	sys 0.000s
I20260709 13:11:41.123304 29872 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:41.131314 29882 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:41.189329 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:41.172174 28080 rpcz_store.cc:276] 0709 13:11:40.074848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.084994 (+ 10146us) service_pool.cc:224] Handling call
0709 13:11:40.086032 (+  1038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:40.089342 (+  3310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:40.089384 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:40.099470 (+ 10086us) log.cc:844] Serialized 26 byte log entry
0709 13:11:40.402736 (+303266us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:40.402807 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.403015 (+   208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:40.796324 (+393309us) spinlock_profiling.cc:243] Waited 67.6 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:41.140463 (+344139us) raft_consensus.cc:1685] finished
0709 13:11:41.140491 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.140773 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2845,"raft.queue_time_us":5639,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":19113,"spinlock_wait_cycles":155552128,"thread_start_us":218685,"threads_started":1,"wal-append.queue_time_us":432450}
W20260709 13:11:41.194144 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 146622us
I20260709 13:11:41.174525 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:41.146546 29918 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:41.180717 28227 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:41.138168 29930 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:41.159732 29899 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:41.116598 29925 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.160943 29876 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:41.145285 29747 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.212853 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:41.235525 28081 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:41.172911 29928 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:41.237111 29928 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:41.237596 29928 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.238022 29928 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:41.218426 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:41.085232 29936 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.243148 29936 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:41.243983 29936 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:41.187898 29767 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:41.232161 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:41.245512 28078 raft_consensus.cc:606] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:41.197530 29892 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.248929 29892 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:41.209444 29772 consensus_meta.cc:287] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.578s	user 0.012s	sys 0.000s
I20260709 13:11:41.253041 29772 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.255697 29772 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.244514 29936 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.261746 29936 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:41.262473 29936 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.277328 29936 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.226200 29908 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:41.280344 29908 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:11:41.281445 29939 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:11:41.288200 29937 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:11:40.784329 28159 rpcz_store.cc:276] 0709 13:11:39.706713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.721620 (+ 14907us) service_pool.cc:224] Handling call
0709 13:11:39.796218 (+ 74598us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:39.876584 (+ 80366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:39.883211 (+  6627us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:39.948906 (+ 65695us) log.cc:844] Serialized 26 byte log entry
0709 13:11:40.317870 (+368964us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:40.317943 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.318070 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:40.783601 (+465531us) raft_consensus.cc:1685] finished
0709 13:11:40.783630 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.783790 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37299,"raft.queue_time_us":46287,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":4108,"thread_start_us":308155,"threads_started":1,"wal-append.queue_time_us":435247}
I20260709 13:11:41.296902 29767 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:41.298350 29767 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.204489 29905 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:41.206151 29918 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.293473 29916 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:41.216327 29765 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:41.271341 29892 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.340124 29785 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.320824 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:41.441277 28755 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:41.322716 29765 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:41.207823 29771 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:41.325446 29706 consensus_meta.cc:287] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.950s	user 0.010s	sys 0.001s
I20260709 13:11:41.443405 29706 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.443717 29930 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:41.168478 29833 ts_tablet_manager.cc:1469] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Trace:
0709 13:11:38.263788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:38.266665 (+  2877us) service_pool.cc:224] Handling call
0709 13:11:38.361130 (+ 94465us) ts_tablet_manager.cc:934] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
0709 13:11:38.566939 (+205809us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:11:38.647535 (+ 80596us) tablet_metadata.cc:665] Metadata flushed
0709 13:11:38.647665 (+   130us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:11:38.873080 (+225415us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:11:38.873444 (+   364us) tablet_replica.cc:206] Creating consensus instance
0709 13:11:38.873815 (+   371us) ts_tablet_manager.cc:989] Replica registered.
0709 13:11:38.873892 (+    77us) inbound_call.cc:177] Queueing success response
0709 13:11:39.344209 (+470317us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:11:40.290301 (+946092us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:11:40.318583 (+ 28282us) tablet_replica.cc:261] Starting instrumentation
0709 13:11:40.319517 (+   934us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":51091,"raft.queue_time_us":4471,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":9977,"tablet-copy.queue_time_us":86321,"tablet-download-pool.queue_time_us":23638,"tablet-download-pool.run_cpu_time_us":1231,"tablet-download-pool.run_wall_time_us":70041,"thread_start_us":121798,"threads_started":2}
W20260709 13:11:41.325906 29924 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
I20260709 13:11:41.524281 28077 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:41.361299 29936 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:41.293924 29940 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.525653 29940 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.526551 29745 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.526655 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 294) took 1138 ms. Trace:
W20260709 13:11:41.326064 29869 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.000s
I20260709 13:11:41.527259 29745 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:41.206815 29872 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.530395 29869 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.530485 29745 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.531901 29872 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.312145 29918 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:41.322016 29808 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:41.407243 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 203328us
W20260709 13:11:41.374163 29938 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:11:41.349776 29943 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:41.340577 29631 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:41.438513 29892 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.323098 29820 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.447291 28085 raft_consensus.cc:2368] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3.
I20260709 13:11:41.422767 29763 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:41.576856 29763 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.579713 29763 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.457214 28758 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:41.445475 29864 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:41.464327 29947 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:11:41.580932 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 296) took 1185 ms. Trace:
I20260709 13:11:41.472023 29773 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:41.472173 29825 consensus_meta.cc:287] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.761s	user 0.007s	sys 0.000s
I20260709 13:11:41.581390 28085 rpcz_store.cc:276] 0709 13:11:40.395324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.395505 (+   181us) service_pool.cc:224] Handling call
0709 13:11:41.580661 (+1185156us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000c85d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:41.580863 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2401869568}
I20260709 13:11:41.473709 29923 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:41.474645 29747 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.475360 29913 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:41.479553 29812 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.003s	user 0.008s	sys 0.000s
I20260709 13:11:41.593235 29812 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.592589 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:41.586254 29825 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.594967 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:41.481278 29925 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:41.454322 29919 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.472568 29905 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.600602 29825 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.574481 29940 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:41.425745 29882 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.487819 29880 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.492409 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:11:41.494720 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 320301us
I20260709 13:11:41.468108 29871 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.486272 29706 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:41.515856 29899 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.518882 29908 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.527256 28077 rpcz_store.cc:276] 0709 13:11:40.388475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.390018 (+  1543us) service_pool.cc:224] Handling call
0709 13:11:41.526376 (+1136358us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002add54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:41.526553 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2503944064}
I20260709 13:11:41.360718 29876 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:41.527803 28124 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.196:46855 timed out after 1.758s (SENT) [suppressed 34 similar messages]
I20260709 13:11:41.524575 29911 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.549942 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 341) took 1306 ms. Trace:
I20260709 13:11:41.550491 29771 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:41.556166 29890 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:41.532687 29869 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.498776 29765 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.563814 29780 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.466001 29928 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:41.585541 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 529) took 1081 ms. Trace:
I20260709 13:11:41.472776 29867 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:41.582489 27984 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:41.650481 27984 catalog_manager.cc:5697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:11:41.624431 29940 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:41.596937 29884 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:41.598207 29895 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:41.598344 29812 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:41.603823 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:41.618234 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:41.619844 29925 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:41.623859 28761 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 13:11:41.603896 29814 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.000s
I20260709 13:11:41.605846 29745 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:41.633143 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 100136us
W20260709 13:11:41.630432 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 155166us
I20260709 13:11:41.634234 28158 raft_consensus.cc:2368] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:41.605424 29943 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:41.675565 29943 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.620265 28755 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:11:41.635921 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.634167 28728 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:41.663553 29918 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.652339 27984 catalog_manager.cc:5697] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:11:41.674790 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:41.648680 29908 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:41.678392 28200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.087s	sys 0.154s
W20260709 13:11:41.703092 28200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.088s	sys 0.154s
W20260709 13:11:41.678584 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 74832.9us
W20260709 13:11:41.707073 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72978.3us
W20260709 13:11:41.681427 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 63309.9us
I20260709 13:11:41.708149 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:41.710898 28235 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:41.712423 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.713327 28227 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:41.630934 28757 rpcz_store.cc:276] 0709 13:11:40.243452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.245509 (+  2057us) service_pool.cc:224] Handling call
0709 13:11:41.549681 (+1304172us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b540043ab54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:41.549844 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131553152}
I20260709 13:11:41.660287 29895 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:41.690750 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 472029us
I20260709 13:11:41.649504 29856 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:41.722370 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:41.744980 28230 raft_consensus.cc:2368] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3.
I20260709 13:11:41.688768 29808 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:41.704706 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:41.724756 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 196969us
I20260709 13:11:41.651224 29780 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:41.738998 28125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.016s	sys 0.065s
I20260709 13:11:41.742903 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.649045 28758 rpcz_store.cc:276] 0709 13:11:40.504372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.513722 (+  9350us) service_pool.cc:224] Handling call
0709 13:11:41.585317 (+1071595us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b5400531e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:41.585465 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":732947712}
I20260709 13:11:41.752225 28230 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:41.733410 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.754321 28234 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:41.660094 29940 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.748678 28154 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:41.686935 29960 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:41.746775 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73882.7us
I20260709 13:11:41.756863 28235 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:11:41.752282 28125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.016s	sys 0.065s
I20260709 13:11:41.727644 27984 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 3, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 3 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:41.758975 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.754299 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:41.759718 28234 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:41.720355 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:41.739277 29895 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:41.745304 29884 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:41.673456 29925 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.749409 29855 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:41.755118 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.678047 29814 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.725653 29867 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:41.760928 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.757959 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.765476 28230 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:41.765652 28229 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:41.766417 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.766469 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:41.766541 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.761631 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:41.752792 29812 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:41.772538 28229 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:11:41.773630 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:41.777057 28158 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
W20260709 13:11:41.773692 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 93716.6us
I20260709 13:11:41.774428 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:41.779840 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:41.780953 28228 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:11:41.782642 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:41.783010 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:41.758087 29957 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:41.787889 28233 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:11:41.767268 29801 consensus_meta.cc:287] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.000s
I20260709 13:11:41.750003 29856 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.802701 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:41.776026 29960 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:41.765333 29959 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:11:41.775837 28245 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-030bc6df8e7e4f2da3b18613d20d9bdc received from {username='slave'} at 127.0.0.1:44070
I20260709 13:11:41.846551 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:41.786594 29907 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:41.857513 28086 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:41.856981 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:11:41.876215 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.854724 29890 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:41.784898 29961 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:41.784196 29812 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.803390 29871 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:41.860307 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:41.776455 29867 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.805331 29771 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:41.799422 29820 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:41.798808 29936 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:42.049372 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 189154us
I20260709 13:11:41.791500 29885 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.152s
I20260709 13:11:41.803896 29801 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.947682 28245 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-030bc6df8e7e4f2da3b18613d20d9bdc on tablet 030bc6df8e7e4f2da3b18613d20d9bdc with peer 70996712c3c942468a6006602de6477f
I20260709 13:11:41.968783 29858 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:42.059995 28154 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:42.018078 29631 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.017859 29808 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.043413 29890 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:41.861430 29960 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:41.801402 29957 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:42.122035 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 363655us
I20260709 13:11:42.053238 29771 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:42.147174 29955 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:11:42.175191 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:42.145901 29954 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.451s	user 0.001s	sys 0.000s
I20260709 13:11:42.178110 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:42.075598 29864 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.183804 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:42.185984 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:11:42.187315 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 423872us
I20260709 13:11:42.189186 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:42.191159 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:42.192844 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:42.194819 28764 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:11:42.059018 29871 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.203181 29931 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.114321 29780 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.063949 29820 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:42.075424 29711 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.171403 29825 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.116720 29801 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.165483 29858 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:42.170485 29928 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:42.212297 29962 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:42.182401 29923 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:42.212425 29958 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
I20260709 13:11:42.173312 29773 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.288127 28085 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:42.385185 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:42.287774 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:42.177189 29771 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:42.238435 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:42.216715 29825 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:42.215988 29965 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:11:42.383919 28233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 1567) took 1833 ms (client timeout 2138 ms). Trace:
I20260709 13:11:42.289918 28077 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:42.330711 28232 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
W20260709 13:11:42.419047 28229 consensus_meta.cc:287] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.639s	user 0.009s	sys 0.000s
I20260709 13:11:42.064416 29936 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:42.427161 28730 leader_election.cc:400] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3.
W20260709 13:11:42.422194 28233 rpcz_store.cc:269] 0709 13:11:40.502408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.510977 (+  8569us) service_pool.cc:224] Handling call
0709 13:11:42.330190 (+1819213us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003b4554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:42.335907 (+  5717us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2871857408}
I20260709 13:11:42.433189 28730 leader_election.cc:403] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:42.430824 28229 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:42.433835 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:42.435292 28233 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:42.287863 29858 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:42.228415 29916 raft_consensus.cc:993] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:42.427227 29880 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:42.430841 29962 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:42.235707 29872 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.182585 29631 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:42.448010 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.361478 29870 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:42.414796 29867 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:42.318724 29885 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:42.322757 29959 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.423997 29814 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:42.444482 29812 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:42.287642 29711 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:42.276955 29918 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.436234 29928 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:42.182794 29808 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.496614 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:42.497306 28160 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:42.498111 29808 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:42.498278 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:42.498831 28160 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:42.448843 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:42.500234 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:42.459703 29870 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:42.501456 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:42.503376 29870 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:42.472707 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:42.511348 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:42.473480 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:42.474150 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:42.425973 29923 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.478927 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:42.424316 29773 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.208978 29864 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.516444 28156 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:42.517740 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:42.495424 29928 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:42.495939 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:42.519546 28155 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
W20260709 13:11:42.456497 29908 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.730s	user 0.009s	sys 0.003s
I20260709 13:11:42.504963 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:42.520938 28231 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:42.454686 29962 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:42.453694 29880 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.514329 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:42.512509 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:42.522815 28154 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:42.214849 29871 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:42.493479 29892 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:42.220417 29931 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.457273 29872 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.524821 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:42.510129 29942 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:42.526599 29871 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:42.526687 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:42.463950 29885 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.527370 28761 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:42.455742 29970 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3.
I20260709 13:11:42.531955 29923 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.515329 29936 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:42.493978 29812 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:42.507445 29814 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.526172 29928 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:42.547214 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.025s	sys 0.023s
W20260709 13:11:42.577975 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.025s	sys 0.023s
I20260709 13:11:42.548116 29957 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:42.513038 29883 raft_consensus.cc:1081] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:11:42.511217 29918 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:42.521401 28755 consensus_meta.cc:287] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.820s	user 0.010s	sys 0.002s
I20260709 13:11:42.545888 28231 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:42.554837 29931 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.465348 29959 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:42.556396 28050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.020s	sys 0.030s
W20260709 13:11:42.563335 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.460317 29820 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.571731 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:42.546422 29808 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.583602 28154 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:42.558528 28159 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:42.595072 29885 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.616587 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:42.588258 29936 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.609120 29918 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.609863 28083 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:11:42.610677 28050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.020s	sys 0.030s
I20260709 13:11:42.609921 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:42.612265 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:42.595863 29880 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.555837 29892 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:42.619059 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47425.4us
I20260709 13:11:42.593029 29870 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:42.595860 29966 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.011s	sys 0.019s
I20260709 13:11:42.620213 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:42.620301 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:42.557138 29942 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.592180 29814 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:42.627689 28231 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:42.682705 28160 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
W20260709 13:11:42.648913 29966 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.011s	sys 0.020s
W20260709 13:11:42.633448 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.620957 29975 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:42.647997 29959 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.608597 28755 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:42.592715 29812 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:42.626643 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 843609us
I20260709 13:11:42.650154 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:42.662516 29872 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.621605 29808 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:42.632774 29908 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:11:42.641517 29931 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:42.633260 29957 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:42.633894 29825 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.699918 29814 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:42.740931 28233 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:42.641044 29983 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.633770 29928 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.607517 29771 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:42.716228 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 66177.8us
I20260709 13:11:42.654405 29863 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:42.725998 29942 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.728578 28231 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:42.722261 29892 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.729310 29711 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:42.760793 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:42.652915 29870 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.732014 28050 leader_election.cc:400] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:42.630393 29936 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.747370 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:42.769930 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:42.747226 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:42.723043 29867 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:42.751525 28234 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:42.727567 29960 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:42.727613 28755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 536) took 1263 ms (client timeout 1624 ms). Trace:
I20260709 13:11:42.776806 29812 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
W20260709 13:11:42.777031 28755 rpcz_store.cc:269] 0709 13:11:41.463431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.492030 (+ 28599us) service_pool.cc:224] Handling call
0709 13:11:42.722456 (+1230426us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:42.731799 29957 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.778625 29936 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:42.778851 29957 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:42.772977 29867 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.779759 29867 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:42.781423 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:42.782631 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:42.783174 28233 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:42.783581 29918 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.776408 28159 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:42.784601 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1366) took 1034 ms. Trace:
I20260709 13:11:42.785502 29771 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:42.788208 29895 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 1.015s	user 0.003s	sys 0.008s
I20260709 13:11:42.790043 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 270) took 1085 ms. Trace:
I20260709 13:11:42.773818 28079 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:42.747210 29814 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.719044 29975 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:42.832329 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.751952 29928 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.751368 29983 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.790530 28152 rpcz_store.cc:276] 0709 13:11:41.704629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.744321 (+ 39692us) service_pool.cc:224] Handling call
0709 13:11:41.762031 (+ 17710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:42.043277 (+281246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:42.049923 (+  6646us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:42.058078 (+  8155us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:42.062134 (+  4056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:42.789751 (+727617us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400051c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:42.789804 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:42.789978 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431613312}
I20260709 13:11:42.885864 29812 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.888501 29945 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:42.922775 29765 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:42.926239 29808 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:42.947594 29812 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:42.950580 29812 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:42.957444 28157 raft_consensus.cc:2410] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:42.958737 29870 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:42.788720 29895 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:42.785161 28226 rpcz_store.cc:276] 0709 13:11:41.750567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.944019 (+193452us) service_pool.cc:224] Handling call
0709 13:11:42.054052 (+110033us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:42.206140 (+152088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:42.206183 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:42.208880 (+  2697us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:42.212087 (+  3207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:42.784269 (+572182us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54001a0254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:42.784343 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:42.784527 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338876800}
I20260709 13:11:42.885282 29989 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:42.998327 29895 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:42.926777 29931 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:43.005594 29975 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:43.007153 29858 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.001s
I20260709 13:11:42.904801 29995 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.041685 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:43.008837 28231 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:43.042564 28234 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.043543 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:43.019580 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:11:43.033777 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 263886us
W20260709 13:11:43.046922 28728 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.023s	sys 0.025s
W20260709 13:11:43.047272 28728 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.023s	sys 0.025s
I20260709 13:11:42.881117 29814 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:43.035957 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 288817us
I20260709 13:11:43.055960 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:43.037197 29863 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:43.040387 29928 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:43.041112 29808 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.041504 29983 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:43.049723 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 630726us
I20260709 13:11:43.004182 29957 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.057793 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.058135 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 272) took 1352 ms. Trace:
I20260709 13:11:43.059417 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.077486 29925 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:43.059969 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:43.058130 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.061147 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:43.033936 29975 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:43.060252 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1355) took 1338 ms. Trace:
I20260709 13:11:43.061596 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:43.051607 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.048828 29812 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.072528 29928 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.059676 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1359) took 1327 ms. Trace:
I20260709 13:11:43.063759 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:42.952229 29945 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.075572 29808 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.048913 29858 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.042131 29995 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:43.285530 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 224443us
I20260709 13:11:43.285758 28761 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:43.136591 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.286602 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:43.287233 29995 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.287918 29987 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:43.070904 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:11:43.289062 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 229143us
W20260709 13:11:43.289430 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 229172us
I20260709 13:11:43.290112 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:43.076623 28161 rpcz_store.cc:276] 0709 13:11:41.705352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.706647 (+  1295us) service_pool.cc:224] Handling call
0709 13:11:43.043373 (+1336726us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:43.058072 (+ 14699us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7084416}
I20260709 13:11:43.111016 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:43.292562 28233 raft_consensus.cc:2368] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:43.137728 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.294238 28083 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:11:43.296617 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.060626 29863 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:43.068542 29936 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.288542 28232 rpcz_store.cc:276] 0709 13:11:41.718729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.746733 (+ 28004us) service_pool.cc:224] Handling call
0709 13:11:42.330397 (+583664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:42.424581 (+ 94184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.045884 (+621303us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b54003b4550. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:43.045997 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1341589248}
I20260709 13:11:43.303158 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:43.303644 28232 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:43.137157 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:43.299026 28200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.094s	sys 0.142s
I20260709 13:11:43.298062 28078 raft_consensus.cc:2410] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:43.299321 28198 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:43.304391 28200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.094s	sys 0.142s
I20260709 13:11:43.296746 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.304904 28085 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:11:43.301591 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.296562 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.300742 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:43.305956 28077 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:43.306540 28083 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:43.309315 28086 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:43.287714 28228 rpcz_store.cc:276] 0709 13:11:41.707474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.716809 (+  9335us) service_pool.cc:224] Handling call
0709 13:11:41.717138 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:41.781388 (+ 64250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.045360 (+1263972us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b5400305f54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:43.045565 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116954880}
W20260709 13:11:43.290514 29867 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.001s
I20260709 13:11:43.312187 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.312700 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:11:43.125797 29985 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 13:11:43.330152 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:43.077625 29979 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.320303 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.109472 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.333613 28233 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:43.325155 28085 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:43.237526 29994 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.313313 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.286631 29869 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.297871 29995 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.371531 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.372334 28083 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:43.197340 29961 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:43.326076 28228 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:43.299218 29925 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.120092 29983 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:43.342865 28086 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:43.330055 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:43.344079 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:43.345470 29988 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:43.346287 29963 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.562s	user 0.017s	sys 0.012s
W20260709 13:11:43.347150 28080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 306) took 2375 ms (client timeout 2052 ms). Trace:
W20260709 13:11:43.394680 28080 rpcz_store.cc:269] 0709 13:11:40.972009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.218073 (+246064us) service_pool.cc:224] Handling call
0709 13:11:43.346840 (+2128767us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b54001af4d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:43.347031 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146533248}
I20260709 13:11:43.396425 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.348946 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:43.398337 28230 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:43.361261 29884 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:43.300631 29857 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:43.073792 29785 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.235s
I20260709 13:11:43.339979 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.326326 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:43.369335 28234 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:43.307121 29871 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:43.326960 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:43.302394 29812 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.319525 29936 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.378029 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.467661 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.468972 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.469532 28086 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:43.469759 29952 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.470229 29952 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.320006 29765 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.471410 29765 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:43.471807 29765 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.370177 28226 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:43.308250 29863 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:43.399909 29994 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:43.480538 29998 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:11:43.480934 29814 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:43.418913 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.418983 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:43.482709 28230 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:43.483914 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:43.301604 29928 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.314993 29975 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:43.458693 28228 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:43.459246 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.300451 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:43.373111 29869 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:43.465963 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:43.494633 28234 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:43.467669 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:43.495591 28233 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:43.415215 29931 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.484580 28230 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:43.418001 29957 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:43.488719 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:43.489079 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:43.495452 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.336287 29867 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.473176 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 1585) took 1234 ms. Trace:
I20260709 13:11:43.462962 29905 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:43.486456 29962 consensus_meta.cc:287] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.937s	user 0.014s	sys 0.000s
I20260709 13:11:43.381950 29979 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.510057 28228 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:43.508972 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:43.523370 28234 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:43.379271 29995 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.486316 29871 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 3 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:43.512346 28230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:53020
I20260709 13:11:43.513104 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:43.537977 28230 raft_consensus.cc:606] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:43.394102 29925 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.390558 29961 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.465763 29857 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:43.514987 29785 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:43.516547 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:43.549432 28233 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:43.503131 28152 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:43.465505 29988 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.535676 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 1582) took 1297 ms. Trace:
I20260709 13:11:43.509387 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:11:43.485095 29977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.033s	sys 0.063s
I20260709 13:11:43.502815 29814 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.487957 29994 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.538491 28230 raft_consensus.cc:614] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:11:43.508347 29820 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.036s	sys 0.061s
I20260709 13:11:43.503074 29869 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.464387 29983 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.515970 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.536206 29957 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.550997 28234 rpcz_store.cc:276] 0709 13:11:42.237878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.508660 (+1270782us) service_pool.cc:224] Handling call
0709 13:11:43.535615 (+ 26955us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:43.572515 29993 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:43.581756 29977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.034s	sys 0.063s
I20260709 13:11:43.517283 29706 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.536005 29979 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.545929 28755 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:11:43.550573 29998 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.069s	user 0.002s	sys 0.000s
W20260709 13:11:43.567143 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 496278us
I20260709 13:11:43.511435 29931 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.510878 28226 rpcz_store.cc:276] 0709 13:11:42.238713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.033021 (+794308us) service_pool.cc:224] Handling call
0709 13:11:43.473099 (+440078us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:43.577037 28078 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
W20260709 13:11:43.581849 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 72573.4us
I20260709 13:11:43.585604 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:53020 (request call id 16) took 1012 ms. Trace:
I20260709 13:11:43.585820 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.534795 29962 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:43.584100 29820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.036s	sys 0.061s
I20260709 13:11:43.587203 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:43.568960 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:43.590595 29858 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.591089 29857 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.593658 29928 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.593978 29988 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:43.594964 29812 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.595506 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1369) took 1844 ms. Trace:
I20260709 13:11:43.586499 29814 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.645973 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.596792 29867 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:43.577999 29785 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:43.597886 29869 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.602196 29995 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.603210 29945 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:43.516314 29905 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.606735 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:43.582615 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.610384 28230 rpcz_store.cc:276] 0709 13:11:42.570404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.512172 (+941768us) service_pool.cc:224] Handling call
0709 13:11:43.582745 (+ 70573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:43.656571 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.657917 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.658152 28230 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:43.658627 28161 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:11:43.659066 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 272) took 1900 ms. Trace:
I20260709 13:11:43.659334 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.621284 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.638186 28158 raft_consensus.cc:2410] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:11:43.660159 28161 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:43.638332 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.661432 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:43.638520 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 340118us
I20260709 13:11:43.604884 29957 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:43.607986 29993 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.664456 29957 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.645964 28233 rpcz_store.cc:276] 0709 13:11:41.751333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.581996 (+1830663us) service_pool.cc:224] Handling call
0709 13:11:43.594935 (+ 12939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.595121 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.595147 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:43.595191 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:43.595234 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.595292 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.595435 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:43.700843 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1375) took 1918 ms. Trace:
I20260709 13:11:43.612843 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:43.649746 28124 connection.cc:256] RPC call timeout handler was delayed by -1.0704755289999639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
I20260709 13:11:43.608533 29952 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.608528 29706 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.644107 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:43.610558 29808 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.661170 29863 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.671968 29869 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:43.673498 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.676822 30004 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:43.677209 29936 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.679105 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.680117 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:11:43.682308 28083 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:43.587389 29975 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.653662 30000 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:43.612650 29871 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.734666 30000 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:43.737403 30000 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:43.642844 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.683429 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:43.681329 29785 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.740036 28155 raft_consensus.cc:2368] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:43.686650 29814 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.687539 29925 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.688020 29983 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.689109 28159 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:43.689729 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.691150 29931 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.691591 28156 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:43.692067 29961 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:43.727808 30006 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.694636 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.659580 28230 rpcz_store.cc:276] 0709 13:11:41.758697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.657295 (+1898598us) service_pool.cc:224] Handling call
0709 13:11:43.657724 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.658663 (+   939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.658993 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21248}
I20260709 13:11:43.698351 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.701310 28233 rpcz_store.cc:276] 0709 13:11:41.782304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.699826 (+1917522us) service_pool.cc:224] Handling call
0709 13:11:43.700255 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.700451 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.700475 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:43.700520 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:43.700553 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.700616 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.700769 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:43.703951 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.771257 28764 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:43.772406 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:43.706815 29962 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:43.708586 29989 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.700s	user 0.005s	sys 0.005s
W20260709 13:11:43.709007 29964 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.846s	user 0.013s	sys 0.020s
I20260709 13:11:43.773406 29925 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.713443 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.715616 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:43.765719 30006 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.774114 28760 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
W20260709 13:11:43.724581 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 112386us
I20260709 13:11:43.728570 28161 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:43.712292 29706 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.723065 29952 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:11:43.736502 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:43.759479 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 76118.2us
I20260709 13:11:43.780606 28049 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:43.773859 29925 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:43.775377 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.735523 29993 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.735746 30004 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.820958 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.773634 29989 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:43.777179 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 61654.7us
I20260709 13:11:43.759588 28152 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:43.777443 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1579) took 1540 ms. Trace:
I20260709 13:11:43.777935 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.725944 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1373) took 1973 ms. Trace:
I20260709 13:11:43.779016 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1379) took 1702 ms. Trace:
I20260709 13:11:43.762924 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.781153 28080 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:43.781514 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:43.826762 28230 rpcz_store.cc:276] 0709 13:11:42.076806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.778213 (+1701407us) service_pool.cc:224] Handling call
0709 13:11:43.778521 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.778681 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.778698 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:43.778804 (+   106us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:43.778825 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.778863 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.778970 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:43.781879 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:43.827646 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.828130 28161 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:43.819584 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.828238 28080 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:43.828626 28155 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:11:43.826114 28233 rpcz_store.cc:276] 0709 13:11:42.237087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.776562 (+1539475us) service_pool.cc:224] Handling call
0709 13:11:43.776964 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.777131 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.777155 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:43.777199 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:43.777230 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.777289 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.777392 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:43.820958 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.769980 29814 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:43.823875 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:11:43.827672 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46242.9us
I20260709 13:11:43.830614 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:43.829222 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.797585 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:43.828568 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1580) took 1591 ms. Trace:
I20260709 13:11:43.832393 28230 rpcz_store.cc:276] 0709 13:11:42.237360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.827828 (+1590468us) service_pool.cc:224] Handling call
0709 13:11:43.828094 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.828261 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.828287 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:43.828331 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:43.828359 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.828410 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.828521 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:43.834327 29871 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:43.831085 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:43.835714 29871 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:43.836136 30002 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:43.826560 28234 rpcz_store.cc:276] 0709 13:11:41.752017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.606551 (+1854534us) service_pool.cc:224] Handling call
0709 13:11:43.606927 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.607121 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:43.704960 (+ 97839us) log.cc:844] Serialized 14 byte log entry
0709 13:11:43.725576 (+ 20616us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:43.725633 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:43.725670 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.725739 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.725880 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":117807,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":95,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":115608,"threads_started":2,"wal-append.queue_time_us":20670}
I20260709 13:11:43.726722 30009 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:43.838574 30009 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.840565 30009 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:43.843436 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:43.843670 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:43.844539 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:11:43.845311 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 47882.8us
I20260709 13:11:43.845844 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:43.848541 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:43.850004 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:43.783674 29706 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:43.819726 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.822959 29925 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.853848 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.780755 29911 consensus_queue.cc:1059] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:11:43.859444 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.831784 29814 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:43.860414 28080 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:43.860845 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.861373 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:43.862464 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.863410 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.832155 29952 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:43.783185 29962 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:43.869110 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.870995 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:43.847594 29983 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:43.849180 29808 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:43.886112 28155 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:43.855368 29919 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:43.888432 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.889901 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.891186 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.855741 29997 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:43.892632 30006 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:43.859445 29765 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.894923 29890 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.895520 29890 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.853967 29905 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:43.860033 29706 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:43.897853 29706 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.898533 30000 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.880872 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.883102 29911 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:43.923974 29911 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:43.924168 29930 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.849498 28756 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:43.925124 29911 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:43.853019 29945 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:43.926111 28762 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:43.926378 29945 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:43.926631 29911 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.926791 29945 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:43.864328 28159 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:11:43.852715 29993 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.928748 29993 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:43.929189 29993 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.929266 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.930004 28159 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:43.930621 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.930984 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.931769 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:43.932422 28158 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:43.855326 29812 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:43.935487 29812 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:43.935879 29812 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:43.855127 29995 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.903501 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.880586 29814 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:43.914068 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 138918us
I20260709 13:11:43.901130 28156 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:43.925640 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.854399 29977 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:43.896018 29890 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.932511 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:43.933667 28763 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:43.934108 28159 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:11:43.904618 29863 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:43.947973 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:43.953585 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:43.968919 28080 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.957180 28756 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 1.
I20260709 13:11:43.959293 29890 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.960041 28086 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:43.907258 29962 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:43.961931 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:43.961412 29979 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:43.882212 29983 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:43.935042 28762 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:43.971124 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:43.979384 28158 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:11:43.972766 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 134296us
I20260709 13:11:43.946135 29977 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:43.945096 29995 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:43.971817 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:43.981050 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:43.978834 29983 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:43.977311 29930 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:43.977660 29962 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:43.985088 29863 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:43.981069 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:43.998775 28080 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:44.001021 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:44.005133 29960 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:44.005965 29960 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:44.010805 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:44.011679 29988 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:43.993307 30006 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:43.993403 29997 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:11:43.996402 29941 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.197s	user 0.013s	sys 0.025s
W20260709 13:11:44.048120 29941 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.261s	user 0.013s	sys 0.031s
I20260709 13:11:44.012194 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:44.050268 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:44.054286 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:44.055634 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:44.042285 28052 leader_election.cc:400] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:44.009749 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:44.057556 28052 leader_election.cc:403] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:11:43.983135 29995 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:43.982296 29977 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:44.012387 29808 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:44.041611 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:44.008555 29962 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.058993 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:44.067026 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1587) took 1485 ms. Trace:
I20260709 13:11:44.087632 28233 rpcz_store.cc:276] 0709 13:11:42.581315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.829858 (+1248543us) service_pool.cc:224] Handling call
0709 13:11:43.830145 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:43.830800 (+   655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.830827 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:43.843123 (+ 12296us) log.cc:844] Serialized 26 byte log entry
0709 13:11:43.993524 (+150401us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:43.993589 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.993680 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:44.066683 (+ 73003us) raft_consensus.cc:1685] finished
0709 13:11:44.066716 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:44.066930 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16447,"raft.queue_time_us":1366,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":10496,"thread_start_us":130289,"threads_started":1,"wal-append.queue_time_us":184030}
I20260709 13:11:44.090381 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:44.072256 29864 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:44.074932 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:44.078511 29899 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:43.996318 29930 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.051232 29936 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:44.047106 29997 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:44.058610 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:44.093648 28233 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:44.094772 28082 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:44.046782 29940 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:44.014122 29911 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:44.083827 29928 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:44.105582 29997 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.084028 29994 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:44.013592 29945 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:44.084128 30001 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:44.086457 30021 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:44.035630 29890 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:11:44.099277 29930 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:44.108418 28159 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:44.139477 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:44.139806 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1720) took 1003 ms. Trace:
I20260709 13:11:44.131075 29892 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:44.086895 29993 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:44.238668 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:44.086508 29808 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:44.239602 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 181046us
I20260709 13:11:44.291107 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:44.107674 29988 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:44.108597 29936 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:44.131690 29979 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:44.273327 28082 rpcz_store.cc:276] 0709 13:11:43.136380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.137879 (+  1499us) service_pool.cc:224] Handling call
0709 13:11:44.074700 (+936821us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:44.139748 (+ 65048us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4484352}
I20260709 13:11:44.252736 29870 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:44.287482 30001 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:44.288519 30027 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:11:44.257318 29940 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:44.377053 30022 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:44.345393 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:44.267218 29911 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:44.267058 29928 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:44.300997 29808 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:44.321110 29943 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:44.308423 29856 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:44.274812 29765 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:44.386535 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:44.275033 30004 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:44.362241 30000 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:44.350256 29870 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:44.302515 29892 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:44.433492 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:44.546247 28085 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:11:44.321609 29988 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:44.398736 29940 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:44.383183 29812 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:44.391937 29936 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:44.642879 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1361) took 2894 ms. Trace:
I20260709 13:11:44.662519 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:44.616271 28158 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.008s	sys 0.003s
W20260709 13:11:44.729768 28158 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.008s	sys 0.008s
I20260709 13:11:44.731719 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 281) took 1662 ms. Trace:
I20260709 13:11:44.732278 28229 rpcz_store.cc:276] 0709 13:11:43.069378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.730920 (+1661542us) service_pool.cc:224] Handling call
0709 13:11:44.731237 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.731429 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.731452 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:44.731493 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:44.731520 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.731568 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:44.731667 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:44.728214 28227 rpcz_store.cc:276] 0709 13:11:41.748662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.751715 (+  3053us) service_pool.cc:224] Handling call
0709 13:11:41.751977 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:41.753202 (+  1225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:41.753230 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:41.762580 (+  9350us) log.cc:844] Serialized 26 byte log entry
0709 13:11:42.778289 (+1015709us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b48000600d8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:11:43.983995 (+1205706us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b180025bdc0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:11:44.004887 (+ 20892us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:44.004944 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.007304 (+  2360us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:44.642621 (+635317us) raft_consensus.cc:1685] finished
0709 13:11:44.642642 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:44.642814 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106761,"raft.queue_time_us":3662,"raft.run_cpu_time_us":34392,"raft.run_wall_time_us":1020419,"spinlock_wait_cycles":11369088,"thread_start_us":11514,"threads_started":1,"wal-append.queue_time_us":2268587}
I20260709 13:11:44.396999 30022 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:44.744187 28049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.005s	sys 0.062s
W20260709 13:11:44.745632 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 686629us
I20260709 13:11:44.751252 28227 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:44.756256 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:44.757725 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 282) took 1458 ms. Trace:
I20260709 13:11:44.757839 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 283) took 1386 ms. Trace:
I20260709 13:11:44.758148 28227 rpcz_store.cc:276] 0709 13:11:43.298746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.750135 (+1451389us) service_pool.cc:224] Handling call
0709 13:11:44.750575 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.756436 (+  5861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.756704 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:44.758598 28229 rpcz_store.cc:276] 0709 13:11:43.371551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.754946 (+1383395us) service_pool.cc:224] Handling call
0709 13:11:44.757375 (+  2429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.757546 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.757571 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:44.757608 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:44.757639 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.757690 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:44.757792 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:44.760325 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:44.755198 28049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.005s	sys 0.062s
I20260709 13:11:44.764634 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1603) took 1248 ms. Trace:
I20260709 13:11:44.766817 28227 rpcz_store.cc:276] 0709 13:11:43.516132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.763357 (+1247225us) service_pool.cc:224] Handling call
0709 13:11:44.763809 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.764043 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.764080 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:44.764137 (+    57us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:44.764180 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.764251 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:44.764391 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:44.551414 29785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.028s	sys 0.044s
I20260709 13:11:44.784650 28233 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:44.551136 29870 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.787529 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:44.792486 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:44.793573 28760 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:44.740826 29940 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:44.783072 29785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.033s	sys 0.053s
I20260709 13:11:44.801777 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:44.759733 30022 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:44.796622 29945 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:44.795708 29993 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:44.805063 30034 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:44.754571 29997 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:44.396297 30001 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:44.643354 29911 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:44.787449 30016 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:11:44.749445 29928 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:44.831713 30034 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:44.849056 29925 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.967s	user 0.013s	sys 0.003s
W20260709 13:11:44.847898 29930 consensus_meta.cc:287] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.546s	user 0.003s	sys 0.006s
I20260709 13:11:44.833400 30032 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:44.844658 29945 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.865054 30034 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:44.869184 30032 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:44.875068 29957 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.175s	user 0.007s	sys 0.007s
W20260709 13:11:44.878980 29994 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.010s	sys 0.010s
I20260709 13:11:44.881058 30032 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:44.882086 29994 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.010s	sys 0.010s
I20260709 13:11:44.882443 29994 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:44.883123 29997 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:44.883625 29994 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:44.886831 29962 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.789s	user 0.008s	sys 0.000s
I20260709 13:11:44.887310 29962 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:44.891709 30001 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:44.895907 29995 consensus_meta.cc:287] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.802s	user 0.007s	sys 0.003s
W20260709 13:11:44.898408 30021 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.597s	user 0.006s	sys 0.004s
I20260709 13:11:44.900501 29995 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:44.900610 30021 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
W20260709 13:11:44.899724 29983 consensus_meta.cc:287] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.891s	user 0.009s	sys 0.000s
I20260709 13:11:44.902637 29995 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:44.902150 29983 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:44.934105 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.005s	sys 0.011s
W20260709 13:11:44.935545 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.006s	sys 0.011s
W20260709 13:11:44.938609 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 647526us
W20260709 13:11:44.934311 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 196117us
I20260709 13:11:44.941007 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:44.943665 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:44.948876 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:44.950949 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1354) took 1215 ms. Trace:
I20260709 13:11:44.951532 28080 rpcz_store.cc:276] 0709 13:11:43.736041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.947935 (+1211894us) service_pool.cc:224] Handling call
0709 13:11:44.950843 (+  2908us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:44.955617 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1767) took 1250 ms. Trace:
I20260709 13:11:44.956326 28159 rpcz_store.cc:276] 0709 13:11:43.704455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.370986 (+666531us) service_pool.cc:224] Handling call
0709 13:11:44.381474 (+ 10488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.381712 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:44.585088 (+203376us) log.cc:844] Serialized 14 byte log entry
0709 13:11:44.954491 (+369403us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:44.954636 (+   145us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:44.954779 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.954965 (+   186us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:44.955300 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":458360,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"thread_start_us":571314,"threads_started":2,"wal-append.queue_time_us":307225}
I20260709 13:11:44.962831 28085 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:44.970443 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:44.972155 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1366) took 1143 ms. Trace:
I20260709 13:11:44.974988 28085 rpcz_store.cc:276] 0709 13:11:43.828358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.433092 (+604734us) service_pool.cc:224] Handling call
0709 13:11:44.972048 (+538956us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:44.976509 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:44.977902 28125 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:44.992885 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1357) took 1229 ms. Trace:
I20260709 13:11:44.993364 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:44.993497 28085 rpcz_store.cc:276] 0709 13:11:43.763869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.976048 (+1212179us) service_pool.cc:224] Handling call
0709 13:11:44.992792 (+ 16744us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:44.994849 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:44.997093 28085 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:44.996097 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:44.975417 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1339) took 1281 ms. Trace:
I20260709 13:11:44.998504 28080 rpcz_store.cc:276] 0709 13:11:43.694228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.969665 (+1275437us) service_pool.cc:224] Handling call
0709 13:11:44.975306 (+  5641us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.000794 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.001854 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1744) took 1322 ms. Trace:
I20260709 13:11:45.005115 28080 rpcz_store.cc:276] 0709 13:11:43.679857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.000331 (+1320474us) service_pool.cc:224] Handling call
0709 13:11:45.001767 (+  1436us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.006577 29765 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:45.006927 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:45.007355 29765 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:45.007779 28080 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.009699 29765 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.017767 29812 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.020264 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:45.025379 29773 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:45.026983 28079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1334) took 3236 ms (client timeout 2045 ms). Trace:
W20260709 13:11:45.027632 28079 rpcz_store.cc:269] 0709 13:11:41.790421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.790586 (+   165us) service_pool.cc:224] Handling call
0709 13:11:45.026442 (+3235856us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54003aa7d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.026696 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115728128}
I20260709 13:11:45.029897 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:45.030299 28085 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:45.031066 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1341) took 1336 ms. Trace:
I20260709 13:11:45.031209 28081 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:11:45.031301 29814 consensus_meta.cc:287] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.064s	user 0.002s	sys 0.016s
W20260709 13:11:45.032074 28081 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1722) took 1832 ms (client timeout 1760 ms). Trace:
I20260709 13:11:45.032022 29814 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.032368 28760 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:45.031441 28085 rpcz_store.cc:276] 0709 13:11:43.694765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.994347 (+1299582us) service_pool.cc:224] Handling call
0709 13:11:45.030945 (+ 36598us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.036877 28080 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:45.037528 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1364) took 1271 ms. Trace:
I20260709 13:11:45.038024 28080 rpcz_store.cc:276] 0709 13:11:43.765904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.006534 (+1240630us) service_pool.cc:224] Handling call
0709 13:11:45.037465 (+ 30931us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:45.032593 28081 rpcz_store.cc:269] 0709 13:11:43.200177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.200604 (+   427us) service_pool.cc:224] Handling call
0709 13:11:45.029444 (+1828840us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:45.031956 (+  2512us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3305216}
I20260709 13:11:44.867141 29925 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.049309 28234 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
W20260709 13:11:45.041085 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 97833.3us
I20260709 13:11:45.058437 30036 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.056954 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.059021 30039 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:45.061945 30039 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:45.062628 30039 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:45.063032 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 261421us
I20260709 13:11:45.062538 29876 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:45.065912 29812 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:44.868430 29930 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.065793 29957 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.066612 29812 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:45.067168 29812 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:45.060106 30036 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.070013 28228 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.064602 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.077760 28079 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:45.078449 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1348) took 1379 ms. Trace:
I20260709 13:11:45.078783 28079 rpcz_store.cc:276] 0709 13:11:43.699222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.038229 (+1339007us) service_pool.cc:224] Handling call
0709 13:11:45.078350 (+ 40121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.074685 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:45.071099 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.080986 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.085492 28159 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:45.085611 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:45.087139 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.091317 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1338) took 1397 ms. Trace:
I20260709 13:11:45.092851 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 271) took 3339 ms. Trace:
W20260709 13:11:45.095050 29808 consensus_meta.cc:287] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.582s	user 0.002s	sys 0.011s
I20260709 13:11:45.092973 28080 rpcz_store.cc:276] 0709 13:11:43.693973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.088174 (+1394201us) service_pool.cc:224] Handling call
0709 13:11:45.088594 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.090918 (+  2324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.090946 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.091016 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:45.091054 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.091109 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.091222 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:44.870831 29945 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.100098 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 539) took 1332 ms. Trace:
I20260709 13:11:45.100715 28763 rpcz_store.cc:276] 0709 13:11:43.767597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.770447 (+  2850us) service_pool.cc:224] Handling call
0709 13:11:43.933478 (+163031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.945964 (+ 12486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.099603 (+1153639us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400722250. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.099849 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346772224}
I20260709 13:11:45.101192 29983 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.102052 29945 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.103116 29962 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.104080 29911 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.105427 29911 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.107479 29925 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.100329 29814 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.087143 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:45.097132 29808 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.109685 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1345) took 1412 ms. Trace:
I20260709 13:11:45.111301 28079 rpcz_store.cc:276] 0709 13:11:43.697110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.086832 (+1389722us) service_pool.cc:224] Handling call
0709 13:11:45.108973 (+ 22141us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:45.094204 28231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1380) took 2454 ms (client timeout 1716 ms). Trace:
W20260709 13:11:45.112452 28231 rpcz_store.cc:269] 0709 13:11:42.639960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.752373 (+112413us) service_pool.cc:224] Handling call
0709 13:11:45.093749 (+2341376us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54003bf6d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.093938 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":568415488}
W20260709 13:11:45.122388 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.081924 29957 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.110754 28154 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
W20260709 13:11:45.116888 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 52497us
I20260709 13:11:45.116780 30036 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.131148 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.131991 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.135694 29931 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:45.136300 29931 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:45.135748 29870 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.137213 28229 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:45.141055 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1627) took 1546 ms. Trace:
I20260709 13:11:45.125138 29957 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:45.143959 29930 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.141734 28154 rpcz_store.cc:276] 0709 13:11:43.579736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.579842 (+   106us) service_pool.cc:224] Handling call
0709 13:11:45.070605 (+1490763us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:45.125819 (+ 55214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6607104}
I20260709 13:11:45.110913 29876 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:11:45.146322 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 90575us
I20260709 13:11:45.143980 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1342) took 1447 ms. Trace:
I20260709 13:11:45.163460 28079 rpcz_store.cc:276] 0709 13:11:43.696349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.125888 (+1429539us) service_pool.cc:224] Handling call
0709 13:11:45.143256 (+ 17368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.143571 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.143596 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.143663 (+    67us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:45.143701 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.143763 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.143896 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.146826 30036 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:45.163975 29988 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.169178 30036 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:45.150674 28159 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.137907 29931 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.174124 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.143985 29808 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.175190 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1751) took 1524 ms. Trace:
I20260709 13:11:45.183490 28155 rpcz_store.cc:276] 0709 13:11:43.650270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.852582 (+202312us) service_pool.cc:224] Handling call
0709 13:11:45.174690 (+1322108us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b5400376ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.174953 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50688768}
I20260709 13:11:45.184451 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.184914 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.186002 28155 raft_consensus.cc:2368] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:45.186975 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.187628 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.149816 29960 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:45.176513 28156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1762) took 1479 ms (client timeout 1851 ms). Trace:
I20260709 13:11:45.108387 28226 rpcz_store.cc:276] 0709 13:11:41.753354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.648178 (+1894824us) service_pool.cc:224] Handling call
0709 13:11:45.091315 (+1443137us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b58001c0598. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552e9c 00007f59cb552a5b 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:11:45.091646 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.092163 (+   517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.092192 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.092271 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:45.092372 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.092440 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.092593 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11141888}
I20260709 13:11:45.134891 30034 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:45.179045 28083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1731) took 1847 ms (client timeout 1798 ms). Trace:
I20260709 13:11:45.178251 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1367) took 3427 ms. Trace:
I20260709 13:11:45.150444 29812 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.178185 29911 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.191393 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:45.198561 28761 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:45.200276 28230 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.200759 30047 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:45.189296 28156 rpcz_store.cc:269] 0709 13:11:43.696784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.780256 (+ 83472us) service_pool.cc:224] Handling call
0709 13:11:45.176138 (+1395882us) spinlock_profiling.cc:243] Waited 71.1 ms on lock 0x7b540033e354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.176325 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163546880}
I20260709 13:11:45.201258 30047 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:45.201706 30047 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:45.201643 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.202050 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.202476 30034 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.193480 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:11:45.195163 28078 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1730) took 1881 ms (client timeout 2073 ms). Trace:
W20260709 13:11:45.197113 28083 rpcz_store.cc:269] 0709 13:11:43.331551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.379768 (+ 48217us) service_pool.cc:224] Handling call
0709 13:11:45.178642 (+1798874us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400207e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.178878 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460903296}
I20260709 13:11:45.181807 30036 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:45.202677 28084 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
W20260709 13:11:45.205523 28084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1727) took 1990 ms (client timeout 1888 ms). Trace:
W20260709 13:11:45.205971 28084 rpcz_store.cc:269] 0709 13:11:43.215475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.287495 (+ 72020us) service_pool.cc:224] Handling call
0709 13:11:45.201346 (+1913851us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:45.205446 (+  4100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7457152}
W20260709 13:11:45.206447 28086 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1743) took 1526 ms (client timeout 1891 ms). Trace:
I20260709 13:11:45.206864 30022 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.207993 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1346) took 1510 ms. Trace:
I20260709 13:11:45.208132 28080 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:45.206905 28086 rpcz_store.cc:269] 0709 13:11:43.679564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.932285 (+252721us) service_pool.cc:224] Handling call
0709 13:11:45.206079 (+1273794us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b54000c99d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.206247 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32208256}
I20260709 13:11:45.209542 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:45.210672 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.197180 28228 rpcz_store.cc:276] 0709 13:11:41.750823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.537328 (+1786505us) service_pool.cc:224] Handling call
0709 13:11:43.537658 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.070586 (+1532928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.177972 (+107386us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400305554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:45.178197 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3521489664}
I20260709 13:11:45.208433 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.146359 29870 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:45.196161 29977 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.103s	user 0.006s	sys 0.010s
W20260709 13:11:45.203477 28078 rpcz_store.cc:269] 0709 13:11:43.313694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.339485 (+ 25791us) service_pool.cc:224] Handling call
0709 13:11:45.194756 (+1855271us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54001ad454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.194987 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276440448}
I20260709 13:11:45.214143 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.208428 28084 rpcz_store.cc:276] 0709 13:11:43.697317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.206909 (+1509592us) service_pool.cc:224] Handling call
0709 13:11:45.207322 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.207624 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.207647 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.207717 (+    70us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:45.207746 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.207800 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.207914 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.218750 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.220341 28078 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.221282 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1360) took 1456 ms. Trace:
I20260709 13:11:45.221670 28078 rpcz_store.cc:276] 0709 13:11:43.764802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.215578 (+1450776us) service_pool.cc:224] Handling call
0709 13:11:45.219846 (+  4268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.220925 (+  1079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.221165 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.224242 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.225244 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.226562 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.226931 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.228134 29942 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:45.228236 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.228629 29942 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:45.238564 29942 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:45.218288 27983 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 257) took 5458 ms. Trace:
I20260709 13:11:45.229097 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1355) took 1488 ms. Trace:
I20260709 13:11:45.173854 29876 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.230333 29870 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:11:45.230810 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49832.4us
I20260709 13:11:45.234082 29977 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.235318 30022 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:45.235090 29882 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:11:45.234082 28084 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.236106 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.206390 29931 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:45.250332 28077 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:45.212143 30032 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.237118 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.239912 28083 rpcz_store.cc:276] 0709 13:11:43.740838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.212045 (+1471207us) service_pool.cc:224] Handling call
0709 13:11:45.212543 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.218204 (+  5661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.218252 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.228794 (+ 10542us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:45.228840 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.228905 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.229038 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.243340 28154 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:45.253407 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.257350 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:45.257795 28199 connection.cc:256] RPC call timeout handler was delayed by -1.3572817859999304s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
I20260709 13:11:45.258001 28154 raft_consensus.cc:2410] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:11:45.258333 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2933817859999408s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
I20260709 13:11:45.258632 28155 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
W20260709 13:11:45.258958 30024 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.912s	user 0.003s	sys 0.007s
I20260709 13:11:45.259593 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:45.246621 28124 leader_election.cc:336] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Timed out: RequestConsensusVote RPC to 127.26.83.196:46855 timed out after 1.758s (SENT)
W20260709 13:11:45.255033 30017 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.331s	user 0.015s	sys 0.031s
I20260709 13:11:45.260330 28124 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:45.247587 29812 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.256752 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.261485 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1359) took 1486 ms. Trace:
I20260709 13:11:45.255364 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 317) took 1169 ms. Trace:
I20260709 13:11:45.261863 28084 rpcz_store.cc:276] 0709 13:11:43.764478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.216943 (+1452465us) service_pool.cc:224] Handling call
0709 13:11:45.233831 (+ 16888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.250592 (+ 16761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.250792 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.261917 28083 rpcz_store.cc:276] 0709 13:11:44.085734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.254144 (+1168410us) service_pool.cc:224] Handling call
0709 13:11:45.254652 (+   508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.255020 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.255050 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.255092 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:45.255120 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.255174 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.255292 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.259723 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.263677 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1594) took 2212 ms. Trace:
I20260709 13:11:45.264083 28234 rpcz_store.cc:276] 0709 13:11:43.051354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.838949 (+787595us) service_pool.cc:224] Handling call
0709 13:11:43.839223 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.049851 (+1210628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.260783 (+210932us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54001f54d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:45.263614 (+  2831us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2781599104}
I20260709 13:11:45.266721 29952 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:45.266706 29940 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.268221 28227 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:45.268316 29989 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:45.268844 29989 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:45.269932 29989 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.265822 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1365) took 1466 ms. Trace:
I20260709 13:11:45.270848 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.271366 29989 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:45.271977 28081 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:45.273206 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:45.273402 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.274129 28764 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:11:45.274441 28153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1629) took 1694 ms (client timeout 2008 ms). Trace:
I20260709 13:11:45.272544 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 320) took 1174 ms. Trace:
W20260709 13:11:45.274926 28153 rpcz_store.cc:269] 0709 13:11:43.580353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.580463 (+   110us) service_pool.cc:224] Handling call
0709 13:11:45.273098 (+1692635us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:45.274329 (+  1231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5502720}
I20260709 13:11:45.275329 28081 rpcz_store.cc:276] 0709 13:11:44.098342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.270459 (+1172117us) service_pool.cc:224] Handling call
0709 13:11:45.272480 (+  2021us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.276212 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.278206 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.279096 28160 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:11:45.279536 28160 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1626) took 1700 ms (client timeout 1547 ms). Trace:
W20260709 13:11:45.281261 28157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1736) took 3507 ms (client timeout 1839 ms). Trace:
W20260709 13:11:45.281888 28157 rpcz_store.cc:269] 0709 13:11:41.773564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.774216 (+   652us) service_pool.cc:224] Handling call
0709 13:11:45.280781 (+3506565us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400053554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.281035 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2708810496}
I20260709 13:11:45.282822 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:45.283416 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.284981 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.285758 28161 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:45.286731 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.288029 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:45.286696 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.239514 27983 rpcz_store.cc:276] 0709 13:11:39.742267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.748384 (+  6117us) service_pool.cc:224] Handling call
0709 13:11:45.200489 (+5452105us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:40.795891 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:40.796606 (+   715us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:40.796652 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:11:40.796676 (+    24us) tablet.cc:662] Decoding operations
0709 13:11:40.797089 (+   413us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:40.797149 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:40.797185 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:11:40.797368 (+   183us) tablet.cc:701] Row locks acquired
0709 13:11:40.797385 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:11:40.797515 (+   130us) write_op.cc:270] Start()
0709 13:11:40.797628 (+   113us) write_op.cc:276] Timestamp: P: 1783602700797477 usec, L: 0
0709 13:11:40.797650 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:11:41.039378 (+241728us) log.cc:844] Serialized 541 byte log entry
0709 13:11:41.945600 (+906222us) op_driver.cc:464] REPLICATION: finished
0709 13:11:42.673460 (+727860us) write_op.cc:301] APPLY: starting
0709 13:11:42.673610 (+   150us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:42.673755 (+   145us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:42.674079 (+   324us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:42.674483 (+   404us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:42.674716 (+   233us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:11:42.674817 (+   101us) write_op.cc:312] APPLY: finished
0709 13:11:42.754238 (+ 79421us) log.cc:844] Serialized 23 byte log entry
0709 13:11:43.275340 (+521102us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:43.297008 (+ 21668us) write_op.cc:454] Released schema lock
0709 13:11:43.300199 (+  3191us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:43.384351 (+ 84152us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:43.394777 (+ 10426us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":354424,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":632647,"apply.run_cpu_time_us":30628,"apply.run_wall_time_us":669355,"mutex_wait_us":359,"num_ops":1,"prepare.queue_time_us":612478,"prepare.run_cpu_time_us":47204,"prepare.run_wall_time_us":635346,"raft.queue_time_us":210690,"raft.run_cpu_time_us":33532,"raft.run_wall_time_us":772982,"replication_time_us":1145436,"thread_start_us":1910917,"threads_started":6,"wal-append.queue_time_us":1195756,"wal-append.run_cpu_time_us":43576,"wal-append.run_wall_time_us":2593568}]]}
I20260709 13:11:45.270996 28078 rpcz_store.cc:276] 0709 13:11:43.799876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.222554 (+1422678us) service_pool.cc:224] Handling call
0709 13:11:45.238009 (+ 15455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.261384 (+ 23375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.261410 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:45.265477 (+  4067us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:45.265522 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.265594 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.265705 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:45.293336 29858 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.293968 29858 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.294821 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.295637 28762 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:45.294821 29940 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:45.279944 28160 rpcz_store.cc:269] 0709 13:11:43.579365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.579576 (+   211us) service_pool.cc:224] Handling call
0709 13:11:45.277907 (+1698331us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:45.279469 (+  1562us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6438272}
I20260709 13:11:45.304854 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.304030 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1600) took 2251 ms. Trace:
W20260709 13:11:45.313390 28127 connection.cc:256] RPC call timeout handler was delayed by -1.0617915319999156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
I20260709 13:11:45.311393 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.318329 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:45.312597 28230 rpcz_store.cc:276] 0709 13:11:43.053161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.841920 (+788759us) service_pool.cc:224] Handling call
0709 13:11:45.093977 (+1252057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.200867 (+106890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.297168 (+ 96301us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54003bf6d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:45.298071 (+   903us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003bf6d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.303956 (+  5885us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3121917568}
I20260709 13:11:45.296417 29977 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.296396 29858 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:45.253434 30039 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.322012 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:45.326628 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.260298 30047 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.332723 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.323446 29952 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:11:45.346726 29631 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:45.347137 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:45.347748 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:45.349419 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:45.350382 29962 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:45.350806 29962 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:45.351177 29962 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.353211 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.353951 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 280) took 2297 ms. Trace:
I20260709 13:11:45.354025 28127 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:45.354476 28233 rpcz_store.cc:276] 0709 13:11:43.056219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.294097 (+1237878us) service_pool.cc:224] Handling call
0709 13:11:44.303629 (+  9532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.785126 (+481497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.353250 (+568124us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54002dafd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:45.353750 (+   500us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105821568}
I20260709 13:11:45.355557 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.356573 28085 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:45.358486 30039 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:45.359437 28079 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.361483 30004 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:45.362362 30004 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:45.363559 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.363284 29942 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.346935 30047 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.305387 28154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:50460
I20260709 13:11:45.365147 30032 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.365317 28154 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:45.328397 29940 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:45.365861 28154 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.366014 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.366205 29940 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.367393 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:45.387251 28085 raft_consensus.cc:2376] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5. Re-sending same reply.
I20260709 13:11:45.368256 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.354809 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.368790 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1363) took 3619 ms. Trace:
I20260709 13:11:45.365042 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.370358 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1604) took 1503 ms. Trace:
I20260709 13:11:45.371071 28153 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:45.373171 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:45.376884 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47109.6us
I20260709 13:11:45.380158 29942 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:45.379850 29962 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.380307 28757 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:45.325438 29911 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.344928 28152 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:45.363713 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.397843 28160 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:45.388664 28235 rpcz_store.cc:276] 0709 13:11:41.749275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.768142 (+ 18867us) service_pool.cc:224] Handling call
0709 13:11:41.772455 (+  4313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:42.065095 (+292640us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:43.711950 (+1646855us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b18003df840. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf
0709 13:11:43.712102 (+   152us) log.cc:844] Serialized 14 byte log entry
0709 13:11:43.772994 (+ 60892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:43.773041 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:43.773078 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.368123 (+1595045us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54000dd4d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:45.368230 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.368560 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":444687,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":194,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":702630016,"thread_start_us":531333,"threads_started":2,"wal-append.queue_time_us":59408,"wal-append.run_cpu_time_us":1172,"wal-append.run_wall_time_us":1005787}
I20260709 13:11:45.398334 28082 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:45.391865 28083 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:45.399423 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.346774 29952 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.366493 29857 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.400315 29952 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:45.400520 29857 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:45.401396 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:45.402190 28081 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.401886 29857 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.403848 29857 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.385066 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.392537 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.389767 28227 rpcz_store.cc:276] 0709 13:11:43.866655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.783428 (+916773us) service_pool.cc:224] Handling call
0709 13:11:44.783752 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.268674 (+484922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.369989 (+101315us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b54001f59d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:45.370269 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1113171840}
I20260709 13:11:45.398430 28153 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:11:45.399502 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:45.406556 28153 pending_rounds.cc:85] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 3
I20260709 13:11:45.336053 28161 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:45.392309 28154 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.407729 28082 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:45.406879 28153 pending_rounds.cc:107] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:11:45.414568 28153 raft_consensus.cc:2889] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:45.421095 28154 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:45.424072 28159 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.420858 28157 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.399374 29911 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.428491 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.433169 29911 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:45.435693 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:45.436096 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:45.437562 29942 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:45.438251 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1344) took 1741 ms. Trace:
I20260709 13:11:45.439126 28227 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:45.438737 28079 rpcz_store.cc:276] 0709 13:11:43.696899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.168345 (+1471446us) service_pool.cc:224] Handling call
0709 13:11:45.173096 (+  4751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.359950 (+186854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.437968 (+ 78018us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54000c9ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:45.438150 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428081920}
I20260709 13:11:45.438328 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.441012 28235 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:45.442786 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:45.446030 28763 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:45.446204 28156 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.449218 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:45.443907 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.451449 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1343) took 1754 ms. Trace:
I20260709 13:11:45.451921 28080 rpcz_store.cc:276] 0709 13:11:43.696643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.130963 (+1434320us) service_pool.cc:224] Handling call
0709 13:11:45.131467 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.208621 (+ 77154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.451202 (+242581us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b54003a98d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:45.451388 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175887104}
I20260709 13:11:45.401419 29952 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:45.453044 30057 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.002s
I20260709 13:11:45.453353 29941 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:45.453711 29952 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.452584 28728 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.454471 29892 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.458392 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:45.459357 28757 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:45.463165 28152 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1765) took 1765 ms (client timeout 2093 ms). Trace:
I20260709 13:11:45.465093 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:45.465174 29975 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:11:45.466605 29977 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:45.467110 29977 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.467552 29977 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:45.467984 30037 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.734s	user 0.000s	sys 0.004s
W20260709 13:11:45.465193 28152 rpcz_store.cc:269] 0709 13:11:43.697479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.890922 (+193443us) service_pool.cc:224] Handling call
0709 13:11:45.462901 (+1571979us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540011b054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.463085 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3340838656}
I20260709 13:11:45.470870 30016 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:45.477648 28081 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:45.484602 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1638) took 1814 ms. Trace:
I20260709 13:11:45.484961 29961 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:45.485533 29961 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:45.484197 29975 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:11:45.486688 29975 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:45.485087 28158 rpcz_store.cc:276] 0709 13:11:43.669729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.011586 (+341857us) service_pool.cc:224] Handling call
0709 13:11:44.011881 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:44.017868 (+  5987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.017914 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:44.043411 (+ 25497us) log.cc:844] Serialized 26 byte log entry
0709 13:11:44.730176 (+686765us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:45.129341 (+399165us) log.cc:844] Serialized 14 byte log entry
0709 13:11:45.130584 (+  1243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.130720 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:45.467114 (+336394us) spinlock_profiling.cc:243] Waited 7.47 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:45.484131 (+ 17017us) raft_consensus.cc:1685] finished
0709 13:11:45.484254 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.484396 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5537,"prepare.queue_time_us":141942,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":24006,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":8850,"spinlock_wait_cycles":17180928,"thread_start_us":957355,"threads_started":2,"wal-append.queue_time_us":675442}
I20260709 13:11:45.487111 29975 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:45.486881 29961 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:45.488229 28077 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1726) took 2273 ms (client timeout 1959 ms). Trace:
I20260709 13:11:45.488866 30035 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:45.488888 29941 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:45.489334 28086 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.489441 30035 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:45.490005 30035 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:45.490789 29941 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.491123 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1349) took 1781 ms. Trace:
I20260709 13:11:45.491346 29941 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:45.491700 28086 rpcz_store.cc:276] 0709 13:11:43.709341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.209579 (+1500238us) service_pool.cc:224] Handling call
0709 13:11:45.489157 (+279578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.489836 (+   679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.490845 (+  1009us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54001aead0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.491040 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641947008}
I20260709 13:11:45.494341 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 321) took 1374 ms. Trace:
W20260709 13:11:45.488776 28077 rpcz_store.cc:269] 0709 13:11:43.215173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.314737 (+ 99564us) service_pool.cc:224] Handling call
0709 13:11:45.487873 (+2173136us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54001aead4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:45.488127 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4122470656}
W20260709 13:11:45.499034 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54985.7us
I20260709 13:11:45.503922 30006 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:11:45.504379 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 238700us
I20260709 13:11:45.504601 30006 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:45.505908 30006 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.492684 29941 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.510003 29942 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:45.510864 29942 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.513681 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:45.515275 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:45.516083 28086 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.510073 30036 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.517880 30032 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:45.518391 30032 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:45.518810 30032 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:45.519387 30036 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:45.519627 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 69459.8us
I20260709 13:11:45.522476 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:45.523790 29911 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:45.524356 28233 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:45.488519 29961 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:45.529610 28081 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.533591 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:45.534700 29812 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:45.535171 29812 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.535370 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:45.535605 29812 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:45.535823 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.535904 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.537052 28227 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:45.537045 30034 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:45.537696 30034 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:45.538079 30034 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.539342 27982 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:45.547263 29940 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.549224 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.550348 28160 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:45.549016 27982 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:45.555366 28227 raft_consensus.cc:2410] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:45.557279 29942 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.557389 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.559732 27984 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 274) took 5025 ms. Trace:
I20260709 13:11:45.557000 28233 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:45.560858 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.556865 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.555200 27982 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:45.494966 28084 rpcz_store.cc:276] 0709 13:11:44.105480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.266328 (+1160848us) service_pool.cc:224] Handling call
0709 13:11:45.271950 (+  5622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:45.273042 (+  1092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:45.273074 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:45.273656 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:11:45.409141 (+135485us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:45.409180 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.409269 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:45.479587 (+ 70318us) raft_consensus.cc:1685] finished
0709 13:11:45.479607 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.479847 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":240,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":194,"thread_start_us":135072,"threads_started":1,"wal-append.queue_time_us":184494}
I20260709 13:11:45.562295 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.561270 28198 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:45.568579 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.569881 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.569438 27982 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:45.488281 29975 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.571424 27982 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 3 to 4, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:45.510076 29941 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.578804 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:45.563807 30016 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:45.578573 27982 catalog_manager.cc:5697] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 2, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:45.581583 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.584064 28082 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.583678 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:45.582911 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.582628 29940 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:45.560389 27984 rpcz_store.cc:276] 0709 13:11:40.533787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.534024 (+   237us) service_pool.cc:224] Handling call
0709 13:11:45.559429 (+5025405us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:41.805050 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:42.139868 (+334818us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:42.141092 (+  1224us) write_op.cc:435] Acquired schema lock
0709 13:11:42.142168 (+  1076us) tablet.cc:662] Decoding operations
0709 13:11:42.205091 (+ 62923us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:42.205145 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:42.205178 (+    33us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:11:42.205611 (+   433us) tablet.cc:701] Row locks acquired
0709 13:11:42.205628 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:11:42.205818 (+   190us) write_op.cc:270] Start()
0709 13:11:42.205955 (+   137us) write_op.cc:276] Timestamp: P: 1783602702205764 usec, L: 0
0709 13:11:42.205975 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:11:42.870767 (+664792us) log.cc:844] Serialized 1696 byte log entry
0709 13:11:43.646340 (+775573us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce672e28 00007f59d20ad581 00007f59d20acd58 00007f59d20b19f1 00007f59d20b19a9 00007f59d20b1939 00007f59d20b1901 00007f59d20b0b2d 000000000038f354 000000000038f199
0709 13:11:43.721144 (+ 74804us) op_driver.cc:464] REPLICATION: finished
0709 13:11:43.721782 (+   638us) write_op.cc:301] APPLY: starting
0709 13:11:43.729677 (+  7895us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:43.774997 (+ 45320us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:43.778001 (+  3004us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:43.818754 (+ 40753us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:43.819026 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:11:43.819120 (+    94us) write_op.cc:312] APPLY: finished
0709 13:11:43.819437 (+   317us) log.cc:844] Serialized 41 byte log entry
0709 13:11:43.819633 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:43.819785 (+   152us) write_op.cc:454] Released schema lock
0709 13:11:43.823984 (+  4199us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:43.865142 (+ 41158us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:43.882602 (+ 17460us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1368199,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":172,"apply.run_cpu_time_us":3404,"apply.run_wall_time_us":131732,"num_ops":4,"prepare.queue_time_us":2661,"prepare.run_cpu_time_us":21796,"prepare.run_wall_time_us":1847101,"replication_time_us":1514961,"spinlock_wait_cycles":1139813632}]]}
I20260709 13:11:45.594751 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:45.592064 27982 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 3 to 4, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:45.596029 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.598917 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.597121 29943 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:45.599844 29943 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.588516 30034 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.600309 29943 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:45.592904 29931 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:45.615206 29931 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.615624 29931 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:45.619637 28227 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:45.620011 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.602964 29983 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:45.621409 29983 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:45.621850 29983 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.622157 28235 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.621088 30004 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:11:45.624398 30004 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.624775 30004 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:45.619927 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.625833 28230 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.603586 30014 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:45.597147 29772 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:45.605049 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48327.9us
I20260709 13:11:45.626535 30014 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:45.626222 30004 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.626950 30014 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:45.627014 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.627913 29772 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:45.618361 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.632481 29975 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:45.607407 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:45.611349 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.637439 29882 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:45.637955 29882 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:45.638469 29882 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.640142 29892 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:45.640898 29919 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:45.642695 29907 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:45.643253 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.644838 29869 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:45.645370 29869 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:45.628000 30014 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.646492 29869 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.647228 30014 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:45.613248 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.642733 30036 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.613587 28159 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:11:45.649648 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 48811us
I20260709 13:11:45.613965 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.614187 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.614635 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:45.621073 28233 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.641903 29812 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.656376 29919 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.657068 29919 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:45.657579 29812 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:45.658452 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.612383 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:45.658242 29919 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.649271 30036 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:45.628371 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.656539 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.650650 27982 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 3 to 4, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:45.659853 29919 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.596572 29962 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:45.627842 30004 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.666450 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:45.667914 30016 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:45.648121 29869 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:45.667976 28230 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:45.669327 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.678159 28233 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.682684 28152 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.683355 28155 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:45.684913 29911 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:45.685478 29977 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.686094 29911 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.686436 29911 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:45.687615 29977 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.687374 29911 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.689165 28158 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.690858 28154 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:11:45.691236 28154 pending_rounds.cc:85] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 6
I20260709 13:11:45.691488 28154 pending_rounds.cc:107] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:11:45.691726 28154 raft_consensus.cc:2889] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:45.694629 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.694943 29812 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:45.695453 29812 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:45.696213 29808 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:45.679767 29904 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:11:45.696906 29808 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.698129 29801 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:45.698570 29801 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:45.699560 29808 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.699697 29801 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.705039 28227 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
W20260709 13:11:45.708352 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 136730us
I20260709 13:11:45.710011 28230 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.711510 28235 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.712908 29943 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.714593 29882 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.669957 30036 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.717247 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.724283 28234 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:11:45.727213 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.728678 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.680378 30032 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.720147 29911 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:45.733187 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.696480 29812 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.736354 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.742372 28228 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.745877 28227 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.746061 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:45.734172 29801 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:45.741397 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.747869 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.751236 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.754508 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.753847 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.756676 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.757535 28227 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:11:45.756601 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69992.1us
I20260709 13:11:45.760815 28233 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 2: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.762041 28227 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:45.762324 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.764252 28228 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.736239 29631 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:11:45.750923 29892 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:45.757064 30036 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.769170 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.768819 29916 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:45.770637 29989 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:11:45.765405 30016 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:45.758790 29904 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:11:45.787022 30062 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:11:45.764326 29943 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.788111 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.747727 29931 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:45.790514 28227 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.795357 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:45.799847 28233 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:45.770557 30034 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:45.765076 29983 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.829067 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.829829 28227 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:45.803308 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:45.804301 29882 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:45.812041 29631 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:45.813231 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.815788 29820 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:45.816525 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:45.817073 29808 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.817814 28235 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:45.818162 29989 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:45.818668 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.825736 29952 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.802968 29904 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.833933 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.792953 30016 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:45.845691 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:45.845651 28230 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:45.861367 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 44905.5us
I20260709 13:11:45.864679 28233 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.842494 29820 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:45.867048 29820 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.868459 29820 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:45.869091 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 96140.6us
I20260709 13:11:45.844336 29952 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:45.829671 30038 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
W20260709 13:11:45.874433 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 107838us
I20260709 13:11:45.848143 29882 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:45.878037 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.870894 28226 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 2: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.846398 29904 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:45.865880 30016 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:45.885644 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.890324 29908 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:45.890515 30039 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:45.887940 29904 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:45.903043 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.863803 29808 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:45.902412 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:11:45.908481 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 112983us
W20260709 13:11:45.909627 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 106400us
I20260709 13:11:45.913029 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.889243 30032 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.889518 29772 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:11:45.900908 29892 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:45.927407 28231 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:45.929211 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:45.961254 28763 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:45.944607 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:45.969085 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59690.3us
I20260709 13:11:45.931305 28234 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:45.944456 28232 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.916604 28233 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:45.972673 28077 raft_consensus.cc:2410] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:45.962378 28763 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:11:45.981395 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 71012.1us
I20260709 13:11:45.928886 30039 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.943856 29892 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:45.961082 29983 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.993112 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:45.995690 28228 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:45.964715 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:45.979954 30038 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.930830 29974 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 13:11:45.927000 29808 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:45.975733 29904 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:45.990167 30034 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.005165 28231 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:45.916918 30014 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:11:46.020705 30034 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.022455 29772 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:46.024691 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.024937 30004 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:11:46.025792 29892 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.028977 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:45.921291 29854 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
W20260709 13:11:46.037613 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.993158 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:46.042969 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 52723.4us
W20260709 13:11:46.049870 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 56847.7us
I20260709 13:11:46.058094 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.061993 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:45.925112 29952 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.047866 29916 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:46.046314 30068 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:46.048005 29931 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:46.076495 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47637.8us
I20260709 13:11:46.061144 29631 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:46.061486 29983 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:11:46.086475 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 102506us
I20260709 13:11:46.088165 28125 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.069924 29857 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:46.048138 30038 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.091939 30014 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:46.095990 30014 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:46.097002 30038 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:46.072106 29908 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:11:46.097028 30014 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.099453 30014 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.103993 29892 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:46.110759 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:46.111874 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.097316 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:46.094760 30006 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:46.124836 30067 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.149s
I20260709 13:11:46.117543 30068 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.128228 28050 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:46.129091 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 76294.9us
I20260709 13:11:46.129978 30068 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:46.130337 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.132120 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.132995 29931 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.133652 29931 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:11:46.137231 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 65795.2us
I20260709 13:11:46.124631 29854 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:11:46.140107 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.142957 29857 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:46.144598 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:46.117334 30006 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:46.147593 30006 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.154506 29952 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.092217 30004 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:46.156998 30004 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.157439 30004 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.158447 29908 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:46.158823 29908 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:46.159204 29908 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.159157 28763 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.140352 29854 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:46.160912 29854 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:46.161808 29962 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:46.161900 29854 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.163434 29854 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.087582 29983 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:46.164978 29983 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.169656 30034 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:46.168138 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.152473 29941 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:46.170750 29941 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:11:46.171353 30035 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.681s	user 0.013s	sys 0.000s
W20260709 13:11:46.171981 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 43829.1us
I20260709 13:11:46.171805 30035 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.172065 29941 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.172894 28080 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:46.173092 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.173552 29941 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:46.173939 30035 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:46.175107 30035 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.170986 28230 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:46.157150 30006 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.162554 29962 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.202530 28763 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.207659 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.187795 28227 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:46.204031 29908 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:46.217832 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49810.2us
W20260709 13:11:46.221460 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 84133.5us
I20260709 13:11:46.204228 29857 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:46.204160 29907 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.187548 29961 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:11:46.229745 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:46.231508 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:46.232915 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:46.233906 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 101923us
I20260709 13:11:46.235339 28229 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:46.234534 28761 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:46.239465 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.242470 28762 raft_consensus.cc:2393] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:46.242864 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.243079 28228 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:46.244918 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.246359 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:46.247017 28226 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.248162 29908 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:46.236443 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.223435 30074 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:11:46.241080 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:46.254731 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:11:46.239653 29911 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.001s
I20260709 13:11:46.261536 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:46.262115 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:46.263314 29911 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:46.264243 29907 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:46.264623 29974 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:11:46.256264 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.265432 29911 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.256526 30038 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:46.266083 29908 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:46.266469 30038 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:46.267414 30038 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.267160 29911 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.260928 28234 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:46.264648 29907 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:46.269842 29907 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.259815 28233 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:46.289178 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 54117.7us
I20260709 13:11:46.289464 28230 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:46.289530 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.291608 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.292402 29931 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:46.292868 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.295102 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.295831 28760 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:46.296633 28235 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:46.297415 28198 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.241377 29857 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:11:46.310690 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.306784 28198 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:46.309865 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:46.313434 28757 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Advancing to term 4
I20260709 13:11:46.289237 29820 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:46.318431 29820 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:46.319516 29820 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.317610 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.320797 28158 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:46.256901 30078 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:11:46.321496 28233 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:46.296712 30034 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.249637 29962 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:46.327358 30069 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.211s	user 0.005s	sys 0.000s
I20260709 13:11:46.315194 29808 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.349354 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.305226 30014 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:11:46.322119 29961 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:46.331044 29977 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:46.357215 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.340197 29916 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:11:46.351390 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:46.374135 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 81357.8us
I20260709 13:11:46.375278 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.374727 28154 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:46.377005 28159 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:46.345906 30085 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:11:46.378239 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.347829 29911 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:11:46.384090 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 45767.5us
I20260709 13:11:46.338934 30032 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:11:46.391564 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.372696 30039 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.390906 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:46.394131 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.366618 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.369123 29961 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.395753 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.375245 29977 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:46.375191 29962 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:46.397535 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:46.350380 29975 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:11:46.372334 29892 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:11:46.417964 28080 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:46.402913 28160 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:46.415539 28760 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:46.404014 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.417723 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.421931 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.391659 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.422458 28157 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 1.
I20260709 13:11:46.424645 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.394637 30006 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:46.432305 30078 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:46.433154 30078 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:46.433525 30078 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.434952 28159 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:46.434532 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:46.435657 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 45826.6us
I20260709 13:11:46.436293 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.403769 29977 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.456178 28156 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:46.415895 30074 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:46.375665 30034 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.436786 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:46.408943 30077 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:11:46.397233 29961 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.439100 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.424185 29882 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.445438 29908 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:46.447325 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:46.489455 28759 raft_consensus.cc:2410] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:46.453814 29892 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:11:46.454144 30076 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:11:46.455437 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.460400 28763 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:46.436887 28157 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:46.415561 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.463114 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.410030 29904 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:11:46.467608 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.496294 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.496673 28160 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.442772 30072 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:11:46.478144 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 81272.5us
I20260709 13:11:46.478598 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.436968 28756 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.456001 30078 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:46.498428 28234 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:46.497800 30014 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:11:46.509032 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.509694 30089 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:46.501924 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.509780 28082 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:46.504979 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.505134 28153 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:46.460863 29911 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:11:46.506395 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:46.500926 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 109997us
I20260709 13:11:46.514771 28082 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:46.518092 28081 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:46.492070 29892 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:46.518899 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.509142 28152 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:46.522460 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:46.502895 28757 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f [term 4 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:46.499404 30072 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:46.527091 28082 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:46.524133 30016 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.622s	user 0.008s	sys 0.003s
I20260709 13:11:46.507156 29975 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:46.533576 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 96924.8us
I20260709 13:11:46.534955 29904 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:46.541527 29904 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:46.541642 29812 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:46.541952 29942 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:46.542480 29942 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:46.542620 29904 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.543919 29904 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.543511 29942 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.541577 30092 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:11:46.544881 30004 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.546626 30093 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:11:46.547299 30004 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:46.548105 29931 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:46.550134 28161 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:46.548811 29892 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.556118 28157 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:46.557245 29962 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:46.558877 29962 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:46.559306 29857 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:46.559847 29857 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.560082 29962 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:46.561898 30034 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:46.562580 30034 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:46.563962 28084 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:46.566342 30086 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:46.566936 28226 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:46.567374 29631 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:46.549647 30016 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.570035 28230 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:46.534503 30032 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:46.574064 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:46.575896 28082 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.576386 28762 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:46.578186 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.578261 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:46.579137 28764 raft_consensus.cc:2410] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:46.575268 30032 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:46.580996 30032 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.560559 29962 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.593178 30075 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.577594 30006 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.595151 30006 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.597059 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:46.598865 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:11:46.601444 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.589862 29983 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.606101 30085 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:46.606562 30085 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:46.607825 29983 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:46.607589 30085 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.603355 29942 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:46.615653 28233 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:46.616837 28758 raft_consensus.cc:2410] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:46.585290 29857 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.619484 29977 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.624595 28125 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.625612 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.627420 28050 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:46.628794 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:46.628954 28082 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:46.608500 30088 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:11:46.631330 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.632894 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.633108 28083 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
W20260709 13:11:46.633143 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 92582.3us
I20260709 13:11:46.636536 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.590447 29812 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:46.639876 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.642010 28084 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:46.642355 28082 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.641098 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.647032 28081 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:46.648504 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:46.654628 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.655391 28764 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:11:46.630195 29975 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:46.658295 28761 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:46.659893 30085 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.659672 29962 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.660940 29962 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.661012 29931 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:46.662128 29931 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:46.662472 29931 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.663554 28078 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:46.659734 30036 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:46.633080 29908 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.666231 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:46.666131 29908 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.638312 30084 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:11:46.671672 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:46.667927 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:11:46.668251 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.631222 29869 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.665542 30006 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:46.631709 30075 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:46.623490 30092 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
W20260709 13:11:46.637908 29916 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:11:46.672606 30035 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:11:46.646210 29974 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.633930 30088 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.669097 28077 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:46.586748 30004 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:11:46.674249 29808 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:46.675226 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.675165 30081 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:46.646379 29631 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:46.659662 29907 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.720961 29908 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:46.721625 29907 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:46.630537 29989 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:46.654467 29952 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:46.722795 28079 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:46.722961 29857 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:46.723196 28080 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:46.724485 28085 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:46.717218 29631 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.725517 29631 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:46.725692 30016 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:46.710672 29974 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.680810 28083 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:46.694875 28086 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:46.732614 29974 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.735005 29974 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:46.673697 29892 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:46.737326 28160 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:46.737860 30081 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:46.738072 28156 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:46.738305 30081 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.695225 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.689393 29961 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.739022 28084 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.739419 30081 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.740401 29961 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:46.698134 29983 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:11:46.686780 30075 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:46.697975 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:46.688819 30072 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.680653 28082 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:46.703796 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:46.694559 29820 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.690526 30078 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.756075 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:46.705533 30032 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.706218 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:46.710443 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:46.688349 28756 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:46.764083 28077 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:46.711020 29808 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:46.713511 28152 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:46.695331 29812 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:46.701988 30034 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:46.741036 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:46.743762 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.744498 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:46.747031 28084 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:46.666765 29931 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.759398 28233 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.768374 28762 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:11:46.764715 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56279.7us
W20260709 13:11:46.778358 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 123180us
I20260709 13:11:46.785089 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.785941 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:46.786633 28160 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:46.787182 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.752312 29904 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:46.780495 28760 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:46.760002 30081 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:46.766335 29952 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.779150 29808 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:46.822575 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 93349.8us
I20260709 13:11:46.829890 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:46.831003 28081 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:46.833451 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:46.833900 28198 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.784960 30038 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:46.777690 30093 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:11:46.822175 29942 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:46.801549 29908 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:46.843935 30006 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:46.846601 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:46.873505 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.849107 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.874418 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:46.848302 30088 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:46.849555 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.849947 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.852211 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.854624 29931 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:46.880548 28077 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:46.861290 29952 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.804597 29904 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.865041 29911 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:46.866356 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:46.886476 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.869768 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:46.887676 28079 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 2: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:46.835059 29962 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.874164 28085 raft_consensus.cc:2410] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:46.848276 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.875320 28084 raft_consensus.cc:2368] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4.
I20260709 13:11:46.875785 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.894558 28078 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:46.895145 28081 raft_consensus.cc:2410] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:46.881453 28233 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:46.837815 29977 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:46.889225 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:11:46.891876 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 59194.5us
I20260709 13:11:46.900074 28077 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 2: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:46.835270 30086 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:11:46.909219 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:46.910702 29916 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:46.911406 29916 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:46.908156 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:46.873656 30006 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.868495 29631 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.916951 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:46.922673 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:46.923357 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 89576.1us
I20260709 13:11:46.905791 30074 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:46.928745 30074 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:46.929183 30074 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:46.912631 29916 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:46.932886 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 234947us
I20260709 13:11:46.933729 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.898914 29974 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:46.926633 30101 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:46.925413 30088 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:46.933048 30004 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:46.927753 29904 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:46.909379 29983 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:46.943125 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.916795 29962 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.917917 30104 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:46.953611 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:46.952126 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:46.800594 30075 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.959908 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:46.961216 28077 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:46.938015 29820 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.934345 30034 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:46.880947 29812 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.962591 28234 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:46.937453 29974 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.937254 29801 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.965417 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:46.937860 30101 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:46.941936 30078 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:46.972538 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.977505 28079 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:46.958642 29977 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:46.956573 29983 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:46.978529 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.941860 29772 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:11:46.958128 30004 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.962815 29908 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:46.980834 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:46.958649 30086 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:11:46.966980 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:46.970001 29974 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:46.980093 28081 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:46.993528 28083 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:46.964367 29962 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:46.985476 30104 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:47.000849 30100 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:11:47.001704 29911 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:47.002106 30016 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:47.002739 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:47.003631 30086 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:46.992053 30078 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:11:46.989403 29989 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.006786 29808 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.980048 30103 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:11:47.013585 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.043385 28081 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:47.014317 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:11:47.019081 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53734.5us
I20260709 13:11:47.021373 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:46.969841 29820 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:46.985502 30072 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:47.022977 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:11:47.051509 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.975605 29801 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.024415 29892 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:11:47.029551 30097 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:11:47.024389 30074 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.030057 28084 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:47.024583 30109 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:11:47.031143 28077 raft_consensus.cc:2410] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:47.031697 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:47.093740 28082 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:47.035003 30086 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:46.994339 30084 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:47.040901 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.052086 30038 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.044909 28762 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:47.048166 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:47.110669 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 62631.6us
I20260709 13:11:47.054230 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:46.970320 30006 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.061084 28230 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:47.022277 29631 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:47.065097 30032 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:47.065565 30074 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:47.066751 29801 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:47.069185 29808 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:47.069796 29892 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:47.071693 30098 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.146s	user 0.002s	sys 0.000s
I20260709 13:11:47.071028 30101 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.057385 28152 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:47.074193 29962 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:47.078595 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.034741 30016 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:11:47.082194 30110 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:11:47.119033 30081 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.032514 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:47.101212 28083 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:47.045737 29904 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:47.108883 28078 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:47.046720 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:47.048036 29911 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:47.111513 28198 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:47.117040 28077 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:11:47.156817 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 45433.1us
W20260709 13:11:47.117024 28199 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.517s (SENT) [suppressed 43 similar messages]
I20260709 13:11:47.110030 28755 raft_consensus.cc:2393] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:47.120222 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:47.125305 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.052253 29989 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.127592 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.160027 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.131606 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:47.118251 29908 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.120366 30004 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.165634 30016 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.166970 29989 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:47.167299 30016 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:47.134255 27981 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 2 to 4, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 4 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:47.116482 30038 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.137605 30086 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.140203 29892 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:47.141554 28762 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:47.146574 29801 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:47.180649 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.144336 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 221750us
I20260709 13:11:47.085201 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.146678 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.146654 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:47.058717 30104 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:11:47.156150 28085 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:47.151026 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:47.157415 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.158747 28082 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:47.159108 28080 raft_consensus.cc:2410] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:47.160567 28081 raft_consensus.cc:2410] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:47.161317 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:47.174031 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.141831 27982 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58554 (request call id 280) took 7054 ms. Trace:
I20260709 13:11:47.173295 28758 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:47.175602 29911 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.149698 29820 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:47.171824 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:47.168934 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.189774 28233 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:47.115572 30103 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:47.181910 28234 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:47.151003 29904 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.182934 28125 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:47.176582 30086 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:47.160820 29808 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:47.191648 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:47.182176 28760 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:11:47.195005 28227 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:47.196406 28229 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:47.164847 30101 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:47.203095 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:47.210670 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:47.210320 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:47.222010 28157 raft_consensus.cc:2368] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6.
I20260709 13:11:47.222849 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:47.223109 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:47.223927 28157 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:47.157194 30081 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.220047 28049 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:47.220283 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:47.188372 30006 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:47.223742 28156 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:47.229425 30120 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.230618 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:47.186975 27982 rpcz_store.cc:276] 0709 13:11:40.087570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.087734 (+   164us) service_pool.cc:224] Handling call
0709 13:11:47.141765 (+7054031us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:46.222109 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:46.313820 (+ 91711us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:46.313873 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:11:46.313891 (+    18us) tablet.cc:662] Decoding operations
0709 13:11:46.319637 (+  5746us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:46.319753 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:46.319797 (+    44us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:11:46.320564 (+   767us) tablet.cc:701] Row locks acquired
0709 13:11:46.320614 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:11:46.320820 (+   206us) write_op.cc:270] Start()
0709 13:11:46.320965 (+   145us) write_op.cc:276] Timestamp: P: 1783602706320746 usec, L: 0
0709 13:11:46.321002 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:11:46.385276 (+ 64274us) log.cc:844] Serialized 3238 byte log entry
0709 13:11:46.892132 (+506856us) op_driver.cc:464] REPLICATION: finished
0709 13:11:47.033748 (+141616us) write_op.cc:301] APPLY: starting
0709 13:11:47.033881 (+   133us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:47.034219 (+   338us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:47.084184 (+ 49965us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:47.085812 (+  1628us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:47.086368 (+   556us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:11:47.086456 (+    88us) write_op.cc:312] APPLY: finished
0709 13:11:47.086988 (+   532us) log.cc:844] Serialized 65 byte log entry
0709 13:11:47.087270 (+   282us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:47.087445 (+   175us) write_op.cc:454] Released schema lock
0709 13:11:47.087528 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:47.088082 (+   554us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:47.088111 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":64540,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":141264,"apply.run_cpu_time_us":5046,"apply.run_wall_time_us":54342,"num_ops":8,"prepare.queue_time_us":282704,"prepare.run_cpu_time_us":30626,"prepare.run_wall_time_us":427869,"raft.queue_time_us":172113,"raft.run_cpu_time_us":29711,"raft.run_wall_time_us":93140,"replication_time_us":570952,"spinlock_wait_cycles":12786048,"thread_start_us":496485,"threads_started":4,"wal-append.queue_time_us":309885}]]}
I20260709 13:11:47.226918 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.233146 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:47.233784 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:47.234186 28158 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:11:47.235162 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:47.238202 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55402.3us
W20260709 13:11:47.246932 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.218783 30014 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:47.237690 28159 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:47.199144 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:47.249989 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:47.251492 28158 pending_rounds.cc:85] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 1
I20260709 13:11:47.262048 28158 pending_rounds.cc:107] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:11:47.262308 28158 raft_consensus.cc:2889] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:47.256387 28157 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:47.253336 28125 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:47.193130 29892 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:11:47.217090 30103 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:47.285004 28161 raft_consensus.cc:2368] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:47.285112 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.297533 28082 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:47.285422 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:47.286368 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:47.298317 28083 raft_consensus.cc:2410] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:47.188781 29801 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.196620 29942 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:11:47.237254 30035 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.250200 29907 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.330963 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:47.325186 28079 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:47.234547 29808 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:47.332824 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.333627 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:47.284622 29820 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:47.261073 30075 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:47.261374 30014 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:47.288570 29961 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:47.358479 28077 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:47.364266 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:47.291667 29962 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:47.336531 28050 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:47.342008 28084 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:47.294787 28152 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:47.354991 28082 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:47.355764 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:47.325888 29857 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.367962 28156 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:47.371564 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:47.372254 28157 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:47.378361 29857 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.378939 30105 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:47.379690 30105 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.380218 30105 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:47.381305 29857 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:47.382606 28156 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:11:47.387151 29801 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:47.409302 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:47.336830 30085 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:11:47.354550 30094 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.267s	user 0.001s	sys 0.003s
I20260709 13:11:47.253866 30120 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:47.335732 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 115512us
I20260709 13:11:47.435413 28124 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:47.357090 30014 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:47.344832 29820 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:47.351495 30075 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.356597 29961 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.431763 28086 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:47.337100 29907 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.503599 28084 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:47.508354 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:47.434419 30085 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:47.460811 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.514086 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:47.335294 30035 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.509213 29961 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:47.439225 29962 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:47.453799 29820 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:47.453001 30075 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:47.543081 30123 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.526161 30108 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:47.517681 30092 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:47.573004 28125 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:47.547053 28757 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:47.535005 30103 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:47.557710 30116 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
W20260709 13:11:47.539789 30096 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:11:47.558585 30117 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
I20260709 13:11:47.556169 28153 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:47.525889 29908 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:47.527513 30085 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:47.447955 30093 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:47.576192 28161 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
W20260709 13:11:47.575357 30088 consensus_meta.cc:287] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.625s	user 0.014s	sys 0.000s
W20260709 13:11:47.576933 29931 consensus_meta.cc:287] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.001s
W20260709 13:11:47.655514 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.659663 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.664572 28229 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:47.589047 30109 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:11:47.678929 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 342408us
I20260709 13:11:47.663838 30093 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.665927 30081 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:47.687346 29974 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.677s	user 0.007s	sys 0.005s
I20260709 13:11:47.663796 29857 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:47.584878 30092 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.723982 28050 leader_election.cc:400] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6.
I20260709 13:11:47.575129 29808 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.670480 29908 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.734601 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.747725 28229 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:47.711455 30109 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:47.731737 30072 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:47.706192 30038 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:47.710086 30093 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:47.577579 30108 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:47.749868 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 68862.9us
W20260709 13:11:47.731940 30122 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:11:47.719682 29904 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:47.576043 30123 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:47.764814 28729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.010s	sys 0.011s
W20260709 13:11:47.770440 28729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.010s	sys 0.011s
W20260709 13:11:47.754268 30101 consensus_meta.cc:287] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.002s
I20260709 13:11:47.768358 28050 leader_election.cc:400] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:47.671675 29983 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.599727 30014 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.724401 29974 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.728653 29857 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.762483 29892 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.725289 29962 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.785002 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:11:47.785154 28199 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.012s	sys 0.016s
W20260709 13:11:47.785759 28199 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.013s	sys 0.016s
W20260709 13:11:47.786175 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 669120us
I20260709 13:11:47.789331 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:47.790571 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:47.791209 28234 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:11:47.793668 28230 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:47.785509 29983 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:47.772100 30072 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.805020 30135 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:47.805187 28226 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:47.809185 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.792146 30038 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:11:47.790750 30089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.016s	sys 0.013s
I20260709 13:11:47.820369 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:47.799774 30120 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:47.784556 29892 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.807554 29907 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:47.823508 28230 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:47.826002 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.823951 28050 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:47.841625 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:11:47.823495 30089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.017s	sys 0.014s
I20260709 13:11:47.849112 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:47.851948 28081 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:47.855003 28080 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:47.828395 29857 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:47.859687 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.860805 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:47.861439 30132 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:11:47.861758 30126 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
W20260709 13:11:47.862501 30124 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:11:47.865070 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.901605 28235 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:47.865720 28226 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:11:47.869992 30006 consensus_meta.cc:287] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.633s	user 0.004s	sys 0.006s
I20260709 13:11:47.870007 30072 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:47.870584 30108 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:47.871712 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:47.828383 29974 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:47.882740 28230 raft_consensus.cc:2410] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:11:47.921275 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.799031 30109 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.922557 28078 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:47.922017 28235 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:47.873116 28228 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:47.847954 29907 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:47.900233 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:47.822467 30135 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:47.905618 28231 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:11:47.847437 30140 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.930629 28231 pending_rounds.cc:85] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 4
I20260709 13:11:47.826339 30138 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:11:47.930934 28231 pending_rounds.cc:107] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:11:47.931354 28231 raft_consensus.cc:2889] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:47.930842 30140 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:47.932631 28199 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.839s (SENT)
I20260709 13:11:47.933393 30140 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:47.944937 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:47.947191 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.952548 28160 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
W20260709 13:11:47.913699 30105 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.002s
I20260709 13:11:47.955277 28232 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:47.913426 28758 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:47.921272 28152 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:47.967561 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:11:47.971123 30139 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:11:47.955350 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:47.959052 29907 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:47.975052 30123 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:47.976346 30135 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:47.905573 30128 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.194s
I20260709 13:11:47.960495 29908 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:47.960892 30092 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:47.985050 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:47.995883 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:47.999682 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:48.000869 30032 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:48.002650 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.003823 30032 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:48.004428 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.005574 30032 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:48.006278 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.007231 30032 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:48.007735 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.010326 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.010516 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.009989 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.011538 28083 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:48.011968 28231 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:47.987506 30108 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:48.017138 28229 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:47.992393 28755 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:47.963303 30105 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:47.996613 29911 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:48.028137 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:47.998972 29983 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:48.009938 29812 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:48.030153 28229 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:48.033751 28154 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:48.016948 30103 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:48.016886 30014 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.039059 30105 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.043953 30084 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.044706 30084 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.044870 29911 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.047391 29911 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.048087 30084 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.047802 30093 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.049400 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.050225 28227 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:48.053582 29820 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.055719 29820 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.033354 29907 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.058076 28234 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:48.064975 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:11:48.057580 30016 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.890s	user 0.013s	sys 0.000s
I20260709 13:11:48.058610 29808 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:48.066218 29983 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:11:48.057323 29892 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.069658 29916 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:11:48.070152 29961 consensus_meta.cc:287] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.529s	user 0.012s	sys 0.000s
I20260709 13:11:48.070703 29961 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.070130 29916 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:48.073375 30085 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.073876 29916 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.075631 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.056516 30075 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.078107 29961 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.077971 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.079051 28078 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:48.075980 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.079624 30075 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.080073 28077 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:48.077436 29916 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:48.076668 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.081489 28080 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:48.082391 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:48.082782 29989 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:48.083159 29989 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:48.083266 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.032404 30108 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:48.084149 30016 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:48.084568 30128 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:48.084020 29989 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.084988 30128 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.085392 30128 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:48.077832 30085 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:48.077265 30086 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.851s	user 0.012s	sys 0.000s
I20260709 13:11:48.086591 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.076284 28083 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:48.090309 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.090466 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.093891 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.097553 30105 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.098137 30105 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.098345 29983 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:48.101670 28125 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:48.101774 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:48.103717 28153 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:48.094188 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.107957 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:48.108768 28153 raft_consensus.cc:2368] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 3. Current term is 4.
I20260709 13:11:48.109251 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.110064 28080 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:48.122282 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:48.123059 28153 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:48.133767 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.136888 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:11:48.137682 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 43874.7us
I20260709 13:11:48.141970 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:48.142118 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:48.143129 28161 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.147557 28728 leader_election.cc:400] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 3. Current term is 4.
I20260709 13:11:48.147921 28728 leader_election.cc:403] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:11:48.149798 28161 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:48.150386 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:48.151017 28153 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:48.151749 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:48.151857 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:48.152254 28153 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.156052 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.156282 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:48.156873 28161 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.161170 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:48.161895 28155 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:48.167203 28080 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:11:48.168303 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.104007 29908 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.170125 28156 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:48.171561 29904 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:48.133466 30085 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.165433 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.176239 30135 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.161006 30084 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:48.161274 30014 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.169256 30123 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:48.136233 30092 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.136199 29983 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:48.155215 30016 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.145315 29801 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.138285 30086 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.187983 28153 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:48.172977 28161 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:48.175551 29857 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.176120 29974 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:48.175910 30120 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:48.136564 30006 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:11:48.181056 30085 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:48.183171 30123 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.213919 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:48.184859 30092 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.220572 28158 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.185719 29983 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:48.185262 29904 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:48.209687 29812 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:48.212134 30086 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.210916 30084 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:48.209486 29801 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.213737 30106 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:48.215689 29974 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.181571 30135 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.218384 28152 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:48.213267 28755 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:48.224383 30074 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.182788 29908 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.221692 30123 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:48.225064 30092 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.227279 30152 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:48.229018 29983 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.223654 30085 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.247879 30035 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.233522 30086 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.253396 28231 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:48.254223 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.254526 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.255138 28231 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.255506 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.211791 30016 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.216147 29857 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.235172 29974 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:48.241251 30162 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:11:48.241647 30127 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:48.232064 29772 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.257668 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.290022 28230 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.258394 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.233795 29801 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.185266 30108 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.277798 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.292176 28235 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:48.278129 28228 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:48.237823 30105 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.291244 28232 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:48.287182 29857 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.304093 28226 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:48.305043 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:48.305747 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:48.239914 30074 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.306207 30109 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.306541 30133 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:11:48.307163 30164 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:11:48.307492 29904 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.305768 28755 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:11:48.284029 30085 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.326097 28230 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:48.277344 30100 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:48.325711 30075 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:48.333956 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 78006.3us
I20260709 13:11:48.331126 28758 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:48.302132 29772 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.262534 30152 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.349647 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:48.336674 30093 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.371109 28231 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
W20260709 13:11:48.365357 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 297252us
I20260709 13:11:48.329170 29911 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:11:48.369644 28756 raft_consensus.cc:2410] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:48.379536 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.381911 28231 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:48.334815 30109 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.373790 30092 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:48.384714 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 243087us
I20260709 13:11:48.417562 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:48.418303 29772 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.418787 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:48.419652 30100 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:11:48.420058 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.421000 30173 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:48.421734 30173 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:48.422034 30100 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:48.417843 30092 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.422947 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:48.423385 29974 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.424520 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.425740 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.426543 30175 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:48.427273 30175 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.427531 30103 raft_consensus.cc:2764] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:11:48.434136 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.438243 30104 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:48.438982 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.439247 28158 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:48.440387 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.437316 30175 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:48.443051 29974 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.442160 30175 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:48.444485 30137 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.445101 30137 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.445804 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.447069 30175 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:48.447494 30137 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.447741 30175 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:11:48.449934 30157 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.427s	user 0.004s	sys 0.000s
I20260709 13:11:48.450136 30175 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:48.451231 30175 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.453580 30105 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.454164 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:48.457042 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:11:48.458060 30158 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.436s	user 0.002s	sys 0.001s
I20260709 13:11:48.461328 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:48.463248 29962 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.463653 29962 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.463986 29962 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.466013 29911 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:48.466719 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:48.344612 29801 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.468051 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:48.468276 29977 raft_consensus.cc:1027] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: attempt to promote peer 70996712c3c942468a6006602de6477f: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:11:48.469501 29801 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.389029 29961 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.347816 30084 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.469156 30161 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:11:48.470527 29961 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.471767 30161 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:11:48.393133 30169 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:11:48.394333 30128 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.472812 29961 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.389644 30093 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.473502 30093 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.475348 30128 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.475740 30093 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.406008 29983 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.404871 30085 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.478047 29983 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:48.378876 29931 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:11:48.480118 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1692) took 1267 ms. Trace:
I20260709 13:11:48.480672 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.481585 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.482161 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:48.482913 30130 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.731s	user 0.000s	sys 0.010s
I20260709 13:11:48.483275 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:48.483593 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.483891 28155 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:48.484838 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:48.485414 28155 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:48.485822 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.486970 29857 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.472711 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1405) took 1119 ms. Trace:
I20260709 13:11:48.487525 29857 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:48.488083 29857 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.460641 30148 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:48.409147 30109 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.484838 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.489481 28078 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.481551 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:48.490099 28161 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:48.490566 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:48.409397 28755 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:48.410481 30086 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.411159 28757 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:48.356001 30075 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.414357 30152 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:48.350406 30133 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.495147 27981 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 188) took 6259 ms. Trace:
I20260709 13:11:48.495805 30133 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.496325 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 1783) took 1369 ms. Trace:
I20260709 13:11:48.412228 30016 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.496752 28079 rpcz_store.cc:276] 0709 13:11:47.126496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.344868 (+218372us) service_pool.cc:224] Handling call
0709 13:11:47.368090 (+ 23222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:47.746208 (+378118us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:47.746236 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:47.746285 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:47.746321 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.496073 (+749752us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54001af4d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:48.496118 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:48.496241 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":174842,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":15354,"spinlock_wait_cycles":464217856,"thread_start_us":175782,"threads_started":1}
I20260709 13:11:48.497988 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.466277 30129 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:48.498558 30129 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:48.498822 28083 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:48.486284 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:48.499459 30129 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.500188 28155 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:48.469940 29916 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:48.483032 28080 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 2: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:48.491475 30085 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.500705 30129 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:48.501642 30085 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:48.454216 30105 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.502553 30085 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.504285 30105 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.501055 29916 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.336292 29812 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.505363 29916 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.505862 30075 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:48.507612 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.508845 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1804) took 1237 ms. Trace:
I20260709 13:11:48.412288 30014 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 3. Current term is 4.
I20260709 13:11:48.509299 28154 rpcz_store.cc:276] 0709 13:11:47.271131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.410950 (+139819us) service_pool.cc:224] Handling call
0709 13:11:47.411443 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:48.034238 (+622795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.508548 (+474310us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b54001969d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:48.508784 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1430523136}
I20260709 13:11:48.510138 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:48.510630 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:48.510847 28758 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:48.511669 28154 raft_consensus.cc:2410] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:11:48.511871 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 43899.8us
I20260709 13:11:48.512261 28728 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.512523 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:48.410115 28152 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:11:48.513144 30136 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:48.513549 30136 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.513679 29974 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:48.513921 30136 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.514248 29974 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:48.514974 29907 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:48.515542 29907 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:48.515149 29974 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.511613 30014 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.516407 29907 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.516635 30092 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:48.534018 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.534637 28080 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.495539 27981 rpcz_store.cc:276] 0709 13:11:42.316706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.360326 (+ 43620us) service_pool.cc:224] Handling call
0709 13:11:43.965788 (+1605462us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e1090 00007f59ce67ff3a 00007f59d44361cc 00007f59d453437e 00007f59d29f52a4 00007f59d29f5232 00007f59d29f5161 00007f59d29f50dc 00007f59d29f4392 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:48.495060 (+4529272us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:47.818844 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:47.964952 (+146108us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:47.966676 (+  1724us) write_op.cc:435] Acquired schema lock
0709 13:11:47.968968 (+  2292us) tablet.cc:662] Decoding operations
0709 13:11:47.988879 (+ 19911us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:47.988970 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:47.989011 (+    41us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:11:47.989226 (+   215us) tablet.cc:701] Row locks acquired
0709 13:11:47.989243 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:11:47.989435 (+   192us) write_op.cc:270] Start()
0709 13:11:47.989592 (+   157us) write_op.cc:276] Timestamp: P: 1783602707989363 usec, L: 0
0709 13:11:47.989613 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:11:47.990393 (+   780us) log.cc:844] Serialized 536 byte log entry
0709 13:11:48.212904 (+222511us) op_driver.cc:464] REPLICATION: finished
0709 13:11:48.391412 (+178508us) write_op.cc:301] APPLY: starting
0709 13:11:48.401489 (+ 10077us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:48.401601 (+   112us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:48.401790 (+   189us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:48.402032 (+   242us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:48.402226 (+   194us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:11:48.402280 (+    54us) write_op.cc:312] APPLY: finished
0709 13:11:48.402478 (+   198us) log.cc:844] Serialized 23 byte log entry
0709 13:11:48.402672 (+   194us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:48.402767 (+    95us) write_op.cc:454] Released schema lock
0709 13:11:48.402820 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:48.403242 (+   422us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:48.403275 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1229293,"reported_tablets":2,"spinlock_wait_cycles":732410624,"child_traces":[["op",{"apply.queue_time_us":159677,"apply.run_cpu_time_us":1898,"apply.run_wall_time_us":20157,"num_ops":1,"prepare.queue_time_us":612369,"prepare.run_cpu_time_us":3853,"prepare.run_wall_time_us":187187,"replication_time_us":190881,"thread_start_us":516711,"threads_started":2}]]}
W20260709 13:11:48.537475 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.540639 29962 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:48.542614 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1697) took 1328 ms. Trace:
I20260709 13:11:48.543027 28156 rpcz_store.cc:276] 0709 13:11:47.214506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.384843 (+170337us) service_pool.cc:224] Handling call
0709 13:11:48.166670 (+781827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:48.176637 (+  9967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.542421 (+365784us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b5400196ed0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:48.542556 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1797349760}
I20260709 13:11:48.545338 28083 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:48.547421 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.548873 28156 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 4. Current term is 5. Ops: [2.3-4.4]
I20260709 13:11:48.549626 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.456344 30074 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.501410 30086 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:48.551246 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.516917 30014 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.551962 30014 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.501714 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:48.554206 29989 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.518953 30135 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.518040 29911 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:11:48.415220 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:11:48.563710 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.524327 28756 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:11:48.525761 29908 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.480587 28157 rpcz_store.cc:276] 0709 13:11:47.212569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.373288 (+160719us) service_pool.cc:224] Handling call
0709 13:11:47.373776 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:47.518735 (+144959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:47.518779 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:47.557956 (+ 39177us) log.cc:844] Serialized 26 byte log entry
0709 13:11:47.782469 (+224513us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:47.804145 (+ 21676us) log.cc:844] Serialized 14 byte log entry
0709 13:11:47.814557 (+ 10412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.477654 (+663097us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b1800415180. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419
0709 13:11:48.479720 (+  2066us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400254c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:48.479795 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:48.479842 (+    47us) raft_consensus.cc:1685] finished
0709 13:11:48.479857 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:48.480050 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25085,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":3055,"raft.queue_time_us":10398,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":51321,"spinlock_wait_cycles":337706624,"thread_start_us":125148,"threads_started":1,"wal-append.queue_time_us":276541}
I20260709 13:11:48.487654 28085 rpcz_store.cc:276] 0709 13:11:47.353165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.517730 (+164565us) service_pool.cc:224] Handling call
0709 13:11:47.529224 (+ 11494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:47.749041 (+219817us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:47.749071 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:47.770190 (+ 21119us) log.cc:844] Serialized 26 byte log entry
0709 13:11:47.992670 (+222480us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:48.347174 (+354504us) log.cc:844] Serialized 14 byte log entry
0709 13:11:48.347479 (+   305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.347565 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:48.457302 (+109737us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:48.472428 (+ 15126us) raft_consensus.cc:1685] finished
0709 13:11:48.472525 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:48.472650 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":902,"prepare.queue_time_us":30812,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":195279,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":26400,"spinlock_wait_cycles":8256384,"thread_start_us":757558,"threads_started":3,"wal-append.queue_time_us":244234}
I20260709 13:11:48.564761 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.569110 29908 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.569512 29908 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.569875 29908 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:48.572041 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 63238.2us
I20260709 13:11:48.577368 29908 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.577735 30123 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.579811 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.580073 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.533001 29857 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.552912 28080 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:48.582747 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.583431 29857 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.583710 28226 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:48.576606 28762 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:48.585003 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.586030 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.586221 28232 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:48.586608 30136 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.588305 29908 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.589634 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.595105 28230 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:48.595430 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.596021 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.596128 28084 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.591028 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.597054 28235 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.598333 29962 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.599920 29916 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.601161 28085 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:48.601318 28082 raft_consensus.cc:2368] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5.
I20260709 13:11:48.591264 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.602305 29916 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:48.602741 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.602670 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.590058 28728 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.603644 28233 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:48.603936 30135 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.604383 30135 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.604861 28083 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:11:48.590674 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.591926 28227 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:48.603642 28085 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:48.613166 28228 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:48.610877 29942 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:11:48.613785 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.590559 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.614197 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.592494 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.614506 28226 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:48.614991 28234 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:48.604526 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.616042 28228 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:48.616183 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:48.616307 28233 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.510608 30085 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.612658 29962 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:11:48.618772 30085 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:48.614449 28227 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:48.596606 28230 raft_consensus.cc:2376] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4. Re-sending same reply.
I20260709 13:11:48.625168 28233 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:48.604822 30135 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.626307 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:48.628669 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.612074 28235 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:48.630885 28161 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:11:48.629860 28083 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:48.388659 28231 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:11:48.537504 28078 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:48.630523 28124 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.645833 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.639139 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:48.650640 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.651496 28235 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.646931 28085 raft_consensus.cc:2376] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3. Re-sending same reply.
I20260709 13:11:48.657466 30104 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:48.659044 30101 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:48.659538 30101 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:48.659827 28226 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:11:48.660348 29942 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:48.606025 30014 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.659929 30101 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.661731 30104 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:11:48.661382 30100 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:11:48.666023 30104 consensus_queue.cc:1243] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:11:48.661037 29942 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:48.666975 30148 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:48.667366 30148 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:11:48.670109 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 292955us
I20260709 13:11:48.671104 28730 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:48.678759 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.703994 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:48.705189 28161 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:48.705580 28762 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:48.706161 28084 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:48.710278 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.710364 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.712910 28083 raft_consensus.cc:2410] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:11:48.728065 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.741753 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:11:48.757812 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 141691us
I20260709 13:11:48.758692 28762 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:48.661069 29962 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:48.758116 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:11:48.764360 28728 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.26.83.254:42913, user_credentials={real_user=slave}} blocked reactor thread for 52856.7us
I20260709 13:11:48.764555 28085 raft_consensus.cc:2376] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2. Re-sending same reply.
I20260709 13:11:48.667884 29942 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.762944 28083 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:48.773865 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.766232 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.767834 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:48.662802 30093 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:48.769234 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:48.806586 28762 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 1.
I20260709 13:11:48.812937 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.771682 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.813747 28083 raft_consensus.cc:2368] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4.
I20260709 13:11:48.813870 28228 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.772496 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:48.795535 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:48.814560 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.795513 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.814826 28230 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.815456 28080 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:48.817222 28081 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 4. Current term is 5. Ops: [2.3-4.4]
I20260709 13:11:48.819190 28083 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:48.794925 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.699283 30085 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:48.813233 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.820659 28082 raft_consensus.cc:2376] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3. Re-sending same reply.
I20260709 13:11:48.814862 28761 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.822120 28759 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:48.822386 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.820695 28078 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
W20260709 13:11:48.823261 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 57127us
I20260709 13:11:48.824787 28728 leader_election.cc:400] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5.
I20260709 13:11:48.825176 28728 leader_election.cc:403] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:11:48.831878 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.771237 30129 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.764127 29892 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.763989 30016 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.771689 29820 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.907490 28235 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:48.758066 30072 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:48.909797 29904 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.009s	sys 0.015s
I20260709 13:11:48.915038 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.916283 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.844578 30152 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:48.711783 29911 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:48.764222 29962 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:48.926419 28230 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:48.903299 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:11:48.934733 30176 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:11:48.935545 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:48.936364 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.937014 28235 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:48.938122 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:48.941407 28761 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:48.942373 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:48.933493 29911 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.946725 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:48.947316 28763 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:11:48.947592 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:48.794579 30032 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.123s
W20260709 13:11:48.807653 30165 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.009s	sys 0.006s
I20260709 13:11:48.668213 30148 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:48.822674 29977 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:11:48.967845 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:48.910773 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:48.806782 29808 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.898896 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:48.917462 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:48.823012 28082 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:48.920223 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.972278 28085 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.972649 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.973290 28082 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:48.901225 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.901993 30093 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:48.929033 28762 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:48.982609 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:48.910586 30137 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.926026 30086 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:48.931512 28080 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:48.904999 30136 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:48.932415 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:48.914490 29904 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.009s	sys 0.016s
I20260709 13:11:48.955740 28078 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:11:48.912151 29857 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.911454 30035 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:48.794925 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:48.915964 30016 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.963349 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:48.965427 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 62202.9us
I20260709 13:11:49.006059 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:48.966441 28049 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:48.974393 28079 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 2: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:48.975996 29989 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:48.928319 30123 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:48.948863 30152 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:48.977319 30148 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:48.983111 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:48.904757 30129 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:48.987900 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 70568.9us
I20260709 13:11:48.921476 30072 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.999383 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:48.915674 29892 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.919265 29820 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.006682 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:48.970026 30133 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.989017 29931 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:49.023229 28082 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:49.025075 30136 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:11:48.954257 30165 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.009s	sys 0.006s
I20260709 13:11:48.985911 29808 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:48.997282 29916 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.001897 29801 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:48.985975 30120 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:11:48.995623 30086 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.039445 28762 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:49.057498 28084 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:49.016105 28226 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:49.047812 28763 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:49.025926 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:49.019613 29989 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.050365 28079 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:49.012279 30035 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.035763 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:49.024461 29911 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:49.032714 28085 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:49.016601 30143 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:49.045979 30089 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:49.068540 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:49.016274 29857 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:49.037709 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54672us
I20260709 13:11:49.036648 29974 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:49.038928 30129 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:49.070554 28756 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:49.008656 29962 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.100162 28762 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:11:49.091435 28077 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:49.060101 30136 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:11:49.075783 30108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.004s	sys 0.020s
W20260709 13:11:49.040014 30006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.007s	sys 0.017s
W20260709 13:11:49.075062 30175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.011s	sys 0.015s
I20260709 13:11:49.070797 29989 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:49.129794 30120 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:11:49.138427 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.041488 30014 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:49.083379 29907 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.021s	sys 0.021s
I20260709 13:11:49.075906 29931 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:49.085804 30147 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.050606 30133 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.097812 29911 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.069459 30075 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:11:49.189016 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:49.151338 30108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.004s	sys 0.020s
I20260709 13:11:49.098207 30143 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:11:49.173139 30175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.015s	sys 0.019s
I20260709 13:11:49.068084 29801 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:49.193271 28157 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.009s	sys 0.015s
I20260709 13:11:49.056469 29916 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.041095 30105 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.107s
I20260709 13:11:49.053964 30093 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:49.156371 30006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.008s	sys 0.023s
W20260709 13:11:49.198441 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 299642us
W20260709 13:11:49.200843 28157 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.009s	sys 0.016s
W20260709 13:11:49.192835 29907 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.021s	sys 0.021s
I20260709 13:11:49.189579 29962 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:49.206144 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:49.192410 29931 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.194020 29911 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:49.268868 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 62795.9us
I20260709 13:11:49.269985 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:49.271466 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:49.231452 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 211217us
W20260709 13:11:49.237795 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 217043us
I20260709 13:11:49.248600 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:49.278791 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:49.279146 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:49.100695 30152 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.290907 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:49.221350 30114 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:49.298329 28228 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:49.199625 30147 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.247633 30100 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:49.320812 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:49.276401 28199 leader_election.cc:400] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4.
I20260709 13:11:49.267689 29931 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:49.283743 29911 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.282714 29977 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:49.290474 30175 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:49.296439 30108 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:49.319131 30084 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:49.376091 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.387375 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:11:49.378439 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 99748.5us
I20260709 13:11:49.383339 28083 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
W20260709 13:11:49.367710 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 89689.3us
I20260709 13:11:49.366786 28199 leader_election.cc:403] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:11:49.375402 30167 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:11:49.376556 30192 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 13:11:49.377184 30191 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:11:49.381770 30084 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:49.427531 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:49.433036 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 152294us
I20260709 13:11:49.433917 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:49.468357 28085 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
W20260709 13:11:49.374958 30135 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.749s	user 0.009s	sys 0.002s
W20260709 13:11:49.381309 30179 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
I20260709 13:11:49.377009 30105 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:11:49.378131 30189 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
W20260709 13:11:49.399320 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 318685us
I20260709 13:11:49.294751 30136 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:49.381466 30188 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:11:49.465075 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 188878us
W20260709 13:11:49.466096 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 384990us
I20260709 13:11:49.490658 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:49.377465 30035 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:49.373657 29907 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:49.458954 29974 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:49.474324 30084 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.426192 30108 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:49.430469 29808 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:49.426683 29820 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:49.481920 30135 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.482158 30105 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.478710 30100 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4.
I20260709 13:11:49.499765 29801 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:49.500861 30147 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:49.503580 30208 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:49.367527 30143 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.392300 30072 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:49.505741 30183 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:11:49.507676 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:49.535979 30196 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:49.397872 29892 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:49.509229 28125 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:49.521085 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:49.524437 29942 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:49.525344 30035 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:11:49.526825 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 133431us
I20260709 13:11:49.426985 29857 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:49.530752 30133 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:49.499498 30193 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:11:49.457617 30006 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:11:49.540704 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 149517us
I20260709 13:11:49.541088 29907 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:49.507791 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:49.564697 30085 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.721s	user 0.007s	sys 0.000s
W20260709 13:11:49.571664 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64059.9us
I20260709 13:11:49.510308 30175 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
W20260709 13:11:49.595439 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 86269.6us
I20260709 13:11:49.597049 30072 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:49.536186 30205 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:49.597640 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:49.572604 28761 raft_consensus.cc:2368] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:49.615492 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:49.626336 28125 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:49.548882 30105 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.637051 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:49.627188 30085 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:49.658056 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 137062us
I20260709 13:11:49.666242 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:49.628001 29857 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:49.630728 30072 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.635769 30100 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:49.664193 29808 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:49.709126 29911 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.659830 30133 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:49.634630 29974 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.551977 30108 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.634729 30196 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:49.710265 30216 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.120s
I20260709 13:11:49.620991 29942 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.713020 30135 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:49.721880 29808 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.721963 29907 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.714363 30175 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:11:49.770431 30175 consensus_queue.cc:1243] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:11:49.770685 30109 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:49.770915 30175 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:49.771394 30175 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:49.771246 30109 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.670408 29892 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.743122 30133 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:49.715332 30072 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:49.760154 30129 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.634s	user 0.013s	sys 0.000s
W20260709 13:11:49.762511 30101 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.003s	user 0.010s	sys 0.000s
I20260709 13:11:49.762856 30196 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.764806 30216 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:11:49.765442 30148 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.002s
I20260709 13:11:49.765192 30215 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:49.767079 29942 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:49.745645 29989 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.000s
I20260709 13:11:49.769313 30104 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:11:49.738976 30137 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.742s	user 0.008s	sys 0.004s
I20260709 13:11:49.798667 29931 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:49.770924 29962 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.503s	user 0.006s	sys 0.003s
W20260709 13:11:49.814097 28227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.049s	sys 0.097s
I20260709 13:11:49.772327 30100 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:11:49.841037 28227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.050s	sys 0.097s
I20260709 13:11:49.839996 29989 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.850322 29911 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:49.853992 29911 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:49.854445 29911 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.854897 29911 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:49.855783 30109 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:49.856586 30109 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.862890 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 621) took 1154 ms. Trace:
I20260709 13:11:49.863493 28757 rpcz_store.cc:276] 0709 13:11:48.708088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.708247 (+   159us) service_pool.cc:224] Handling call
0709 13:11:49.861737 (+1153490us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:11:49.862056 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.862396 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:49.862450 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:49.862508 (+    58us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:49.862556 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.862640 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.862808 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9137536}
I20260709 13:11:49.816426 30101 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.785498 30175 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.866456 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:49.867348 30175 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:49.867467 30101 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:49.833287 30104 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:11:49.755030 30188 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:11:49.849999 28161 raft_consensus.cc:2410] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:11:49.855942 30194 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.377s	user 0.000s	sys 0.002s
I20260709 13:11:49.852461 29989 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:49.815284 30129 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.897115 30104 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.898883 30129 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:49.900256 30180 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.605s	user 0.000s	sys 0.000s
I20260709 13:11:49.900837 29931 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:49.901746 28226 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:49.832096 30216 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:49.849924 30137 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5.
W20260709 13:11:49.902467 30184 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.693s	user 0.000s	sys 0.002s
I20260709 13:11:49.914618 30206 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.809089 29892 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.941742 30206 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.942414 30104 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:49.943257 30084 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.944159 30206 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:49.945735 30216 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.947651 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1703) took 1802 ms. Trace:
I20260709 13:11:49.949025 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35896 (request call id 618) took 1246 ms. Trace:
I20260709 13:11:49.948096 28154 rpcz_store.cc:276] 0709 13:11:48.145687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.512438 (+366751us) service_pool.cc:224] Handling call
0709 13:11:48.512825 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:48.531303 (+ 18478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:48.531333 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:48.531788 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:11:48.989232 (+457444us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:49.007453 (+ 18221us) log.cc:844] Serialized 14 byte log entry
0709 13:11:49.007675 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.007904 (+   229us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:49.947393 (+939489us) raft_consensus.cc:1685] finished
0709 13:11:49.947443 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.947591 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12866,"prepare.queue_time_us":23196,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":4087,"raft.queue_time_us":13114,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":11711,"thread_start_us":457027,"threads_started":1,"wal-append.queue_time_us":661279}
I20260709 13:11:49.949440 28756 rpcz_store.cc:276] 0709 13:11:48.702433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.763896 (+ 61463us) service_pool.cc:224] Handling call
0709 13:11:48.946540 (+182644us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:49.948702 (+1002162us) spinlock_profiling.cc:243] Waited 83.4 ms on lock 0x7b5400472cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:49.948925 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254634112}
I20260709 13:11:49.950898 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1438) took 1358 ms. Trace:
I20260709 13:11:49.950909 28762 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:49.951499 28234 rpcz_store.cc:276] 0709 13:11:48.592575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.615629 (+ 23054us) service_pool.cc:224] Handling call
0709 13:11:48.625370 (+  9741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:48.625744 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:48.625787 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:48.626379 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:11:49.051699 (+425320us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:49.051755 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.051854 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:49.739356 (+687502us) raft_consensus.cc:1685] finished
0709 13:11:49.739385 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.909602 (+170217us) spinlock_profiling.cc:243] Waited 9.43 ms on lock 0x7b54000dedd0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:49.950832 (+ 41230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3191,"raft.queue_time_us":19933,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":148,"spinlock_wait_cycles":21689600,"thread_start_us":424874,"threads_started":1,"wal-append.queue_time_us":580687}
I20260709 13:11:49.969588 28764 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
W20260709 13:11:49.816097 30032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.041s	sys 0.050s
I20260709 13:11:49.864493 30133 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:49.932888 30147 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:49.982914 30147 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.983644 30101 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:49.974943 28080 pending_rounds.cc:85] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 4
I20260709 13:11:49.974913 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:49.986150 28080 pending_rounds.cc:107] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:11:49.986552 28232 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:49.976042 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:49.950019 30105 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:49.986999 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:49.987809 28229 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:49.989493 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1814) took 1322 ms. Trace:
I20260709 13:11:49.989867 30084 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:49.990584 28235 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:49.992295 28080 raft_consensus.cc:2889] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:49.989907 28157 rpcz_store.cc:276] 0709 13:11:48.666860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.667035 (+   175us) service_pool.cc:224] Handling call
0709 13:11:48.667397 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:48.667727 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:48.667755 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:48.668240 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:11:49.216219 (+547979us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:49.216295 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.216407 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:49.855249 (+638842us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:49.985405 (+130156us) raft_consensus.cc:1685] finished
0709 13:11:49.985430 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.989430 (+  4000us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1104,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":41628,"spinlock_wait_cycles":28464896,"thread_start_us":547527,"threads_started":1,"wal-append.queue_time_us":707608}
I20260709 13:11:49.997236 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:49.984263 30101 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:49.970989 30032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.042s	sys 0.050s
I20260709 13:11:49.991925 29942 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.985478 30211 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.991952 29983 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.014040 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:49.995765 28758 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:11:50.016332 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.018220 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:50.004962 30214 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:50.028357 28229 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:50.042121 30101 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:50.007061 29911 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.045432 30072 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.047067 30214 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.049445 30214 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:50.050012 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:50.050689 28234 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
W20260709 13:11:50.049365 30185 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 13:11:50.065116 28229 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:50.066339 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:50.068869 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:50.069625 28228 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:50.025693 30105 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:50.069736 28232 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:50.013108 29942 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.066211 30006 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:50.019244 29983 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.099691 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:11:50.057103 30093 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.005s	sys 0.013s
W20260709 13:11:50.110469 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 602689us
I20260709 13:11:50.111864 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:11:50.108134 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 471152us
W20260709 13:11:50.112058 30093 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.005s	sys 0.013s
I20260709 13:11:50.105557 30129 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:50.016623 30211 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.121025 30224 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 13:11:50.119309 30213 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.229s	user 0.000s	sys 0.002s
I20260709 13:11:50.107219 30215 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:50.107858 30072 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:50.118821 30214 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:50.169843 30214 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.170342 30214 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:50.138083 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:50.122864 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:50.120263 28757 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:50.144706 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:11:50.174539 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48618.3us
I20260709 13:11:50.134370 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:50.144292 30093 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 13:11:50.122254 29857 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:50.120095 30143 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:11:50.134127 30227 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:50.450627 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 329693us
I20260709 13:11:50.574429 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:50.516162 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 381928us
I20260709 13:11:50.603260 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:50.485731 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 349022us
I20260709 13:11:50.252976 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:50.166648 30129 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:50.471410 30229 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.322s	user 0.000s	sys 0.000s
W20260709 13:11:50.471778 30104 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
I20260709 13:11:50.604595 28153 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:50.604782 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:50.450336 30140 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:50.605912 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:50.615873 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:50.472182 30226 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
W20260709 13:11:50.447814 30175 consensus_meta.cc:287] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.000s
I20260709 13:11:50.460177 29911 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:50.610468 28159 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:50.250813 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:50.543313 30211 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:50.251065 30072 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:50.605612 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 429844us
I20260709 13:11:50.458469 30209 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:50.331336 29907 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:11:50.597836 30240 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:11:50.616504 28155 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:50.612527 30133 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:11:50.447503 30231 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.280s	user 0.000s	sys 0.000s
I20260709 13:11:50.251549 30109 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:50.448987 30152 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:50.461053 30136 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:50.613576 28160 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:50.150024 30220 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:50.614367 28153 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:50.618969 28156 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:50.483891 29942 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:50.555078 29983 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:50.623855 30140 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.632488 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1666) took 1871 ms. Trace:
I20260709 13:11:50.618695 30105 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:11:50.630720 30224 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:50.477644 30233 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:50.644811 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:50.691205 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:50.334266 29989 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.283s
I20260709 13:11:50.611845 30129 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:50.697765 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:50.700295 28231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 322) took 1961 ms (client timeout 1601 ms). Trace:
W20260709 13:11:50.700901 28231 rpcz_store.cc:269] 0709 13:11:48.731293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.769920 (+ 38627us) service_pool.cc:224] Handling call
0709 13:11:50.691014 (+1921094us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b58001c0598. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:50.692183 (+  1169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3411840}
I20260709 13:11:50.652998 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:50.609032 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:50.703285 28229 raft_consensus.cc:2410] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:50.703224 28157 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:50.351018 30214 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.661131 28232 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:50.613046 30234 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:50.251740 30215 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.681114 30232 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.641422 30196 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:50.682160 28227 rpcz_store.cc:276] 0709 13:11:48.761258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.916289 (+155031us) service_pool.cc:224] Handling call
0709 13:11:48.916692 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:49.172444 (+255752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.172483 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:49.195157 (+ 22674us) log.cc:844] Serialized 26 byte log entry
0709 13:11:49.849251 (+654094us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:49.849306 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.849408 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:50.632192 (+782784us) raft_consensus.cc:1685] finished
0709 13:11:50.632221 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.632416 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103455,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":13277,"thread_start_us":813370,"threads_started":2,"wal-append.queue_time_us":895483}
I20260709 13:11:50.685431 30136 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.354696 30006 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.641894 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:50.601734 30227 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.653947 30216 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.651937 30104 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:50.674846 30133 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:50.683946 30152 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.705204 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:50.681459 28763 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.021s	sys 0.048s
W20260709 13:11:50.735738 28763 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.022s	sys 0.048s
I20260709 13:11:50.681844 30220 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.682063 30209 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:50.713490 28153 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:50.724053 30196 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.688027 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:50.741276 28084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1419) took 2034 ms (client timeout 1525 ms). Trace:
I20260709 13:11:50.741915 30209 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:50.741704 28084 rpcz_store.cc:269] 0709 13:11:48.707293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.831493 (+124200us) service_pool.cc:224] Handling call
0709 13:11:50.741107 (+1909614us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400209754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:50.741207 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358907264}
I20260709 13:11:50.743775 28759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 584) took 1837 ms. Trace:
I20260709 13:11:50.706599 30224 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.744243 28759 rpcz_store.cc:276] 0709 13:11:48.906585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.910883 (+  4298us) service_pool.cc:224] Handling call
0709 13:11:50.742964 (+1832081us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:11:50.743205 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.743414 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.743450 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:50.743498 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:50.743534 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.743595 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.743713 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3301120}
I20260709 13:11:50.745766 30224 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:50.745857 30215 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:50.719730 30232 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.747171 30140 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:50.737434 30220 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:50.748193 30232 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:50.655458 30211 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:11:50.718009 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:50.719513 28154 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:50.695240 30212 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.720880 30230 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.720938 30219 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.696084 29857 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.704867 30109 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:50.706287 30234 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.723426 30214 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:50.727607 30136 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:50.728562 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:50.709182 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:50.740485 30196 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:50.732184 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:50.763044 30212 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.765570 30230 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.736388 30104 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.735966 30133 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:50.660873 30072 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.769131 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:50.776979 28156 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:50.785997 29974 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:50.787125 30227 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:50.793229 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:50.797516 28084 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:11:50.798288 30152 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:50.798938 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:11:50.787849 30190 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.492s	user 0.004s	sys 0.009s
I20260709 13:11:50.767988 30219 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.864463 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1444) took 2187 ms. Trace:
I20260709 13:11:50.805799 30147 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:50.813340 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:50.864903 28153 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:50.865816 30219 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:50.866050 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:50.767318 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 596) took 1054 ms. Trace:
I20260709 13:11:50.867192 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:50.867841 30016 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:50.734043 30216 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.867340 28763 rpcz_store.cc:276] 0709 13:11:49.713124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.713288 (+   164us) service_pool.cc:224] Handling call
0709 13:11:49.717583 (+  4295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:49.731085 (+ 13502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.731116 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:49.731594 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:11:50.736350 (+1004756us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:50.736420 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.736520 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:50.767064 (+ 30544us) raft_consensus.cc:1685] finished
0709 13:11:50.767087 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.767251 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8945,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"thread_start_us":997551,"threads_started":1,"wal-append.queue_time_us":1026446}
I20260709 13:11:50.869690 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:50.824347 30196 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.817814 30232 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.870297 28763 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:50.827693 28160 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:11:50.808625 30105 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:50.867205 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:50.871171 30105 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.871543 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:50.871871 28157 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:50.873541 30105 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:50.840517 28085 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.864916 28233 rpcz_store.cc:276] 0709 13:11:48.677191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.888233 (+211042us) service_pool.cc:224] Handling call
0709 13:11:48.898419 (+ 10186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:49.071524 (+173105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:49.071552 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:49.072139 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 13:11:49.322043 (+249904us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:49.322097 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.864053 (+1541956us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400306454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:50.864111 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:50.864156 (+    45us) raft_consensus.cc:1685] finished
0709 13:11:50.864168 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.864400 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15,"raft.queue_time_us":134,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":11955,"spinlock_wait_cycles":396849536,"thread_start_us":249536,"threads_started":1,"wal-append.queue_time_us":926721}
W20260709 13:11:50.840005 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 60474.7us
I20260709 13:11:50.843449 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:50.879381 28080 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:50.844331 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:50.843452 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:50.847967 30204 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:50.854481 30230 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:50.854820 30220 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:50.862223 28761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 592) took 1926 ms (client timeout 1594 ms). Trace:
W20260709 13:11:50.900808 28761 rpcz_store.cc:269] 0709 13:11:48.935578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.942132 (+  6554us) service_pool.cc:224] Handling call
0709 13:11:50.861928 (+1919796us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b54004d21d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:50.862130 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1447624064}
I20260709 13:11:50.902525 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:50.818014 30006 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.904289 30006 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.905747 28156 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:50.782367 30234 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:50.868119 28153 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:50.841868 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:50.892035 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 125419us
I20260709 13:11:50.909250 28759 raft_consensus.cc:2368] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:50.897109 28157 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:11:50.897233 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:50.915655 28728 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:50.782027 30109 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.825918 28758 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:50.924364 28085 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:50.931291 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:50.932067 28084 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:50.924907 28049 leader_election.cc:400] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:50.921262 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:50.933768 28049 leader_election.cc:403] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:50.930895 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:50.934396 28234 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:50.926370 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:50.935595 28080 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:50.870297 30216 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:50.938647 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:50.891251 30204 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:50.913771 30006 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:50.914944 29907 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:50.926213 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:50.870693 30232 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.893808 30196 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:50.941489 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:50.979550 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1426) took 2034 ms. Trace:
I20260709 13:11:50.988487 28227 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:50.960495 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:50.990445 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:50.992245 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:50.941632 29911 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.975862 28234 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:50.939816 30215 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.978715 29907 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:50.984630 30214 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:50.988240 28081 rpcz_store.cc:276] 0709 13:11:48.945584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.961350 (+ 15766us) service_pool.cc:224] Handling call
0709 13:11:48.961799 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.101339 (+139540us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:49.345445 (+244106us) log.cc:844] Serialized 14 byte log entry
0709 13:11:49.527628 (+182183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:49.527703 (+    75us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:49.527767 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.979282 (+1451515us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540017d754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:50.979328 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:50.979477 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3373,"prepare.queue_time_us":262794,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":100625,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"spinlock_wait_cycles":374886528,"thread_start_us":345275,"threads_started":3,"wal-append.queue_time_us":519501}
I20260709 13:11:50.994168 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:51.014936 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:51.015573 28081 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:50.997179 28084 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:50.999286 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:50.999549 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:50.946549 29908 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.002789 28759 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:51.018858 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:51.019706 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:51.003702 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 65479.1us
I20260709 13:11:50.924894 30152 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:51.024151 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:51.011451 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:11:51.012485 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:51.025918 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.940431 29989 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.017868 28231 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:51.020625 28085 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:51.023017 29983 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.940203 30123 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:51.024049 30032 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:50.940021 30251 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:51.033480 28081 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:51.035144 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:51.036586 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:50.968151 28157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:51.054093 28234 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:51.054383 28157 raft_consensus.cc:606] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:51.054730 28157 raft_consensus.cc:614] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:51.054104 28227 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:51.056515 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:51.038883 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:51.039673 28230 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:51.018970 30103 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.018782 30216 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:51.066298 30216 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:51.067276 30216 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:51.039713 30143 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.042874 30224 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:51.021750 30215 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:51.047832 28125 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:50.960969 30212 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:51.081193 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:51.059206 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:51.084117 28160 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:51.085335 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:51.086759 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:51.060310 28231 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:11:51.088850 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:51.061959 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:51.052325 30251 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.091363 28227 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:51.062841 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:51.101577 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 53867.7us
I20260709 13:11:51.102480 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:51.103094 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:50.938441 30219 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.099s
I20260709 13:11:51.014168 29911 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:51.042851 28226 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:51.040069 30089 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:51.007061 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1428) took 2060 ms. Trace:
I20260709 13:11:51.069706 30123 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:51.114233 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:51.037220 30108 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.028501 29908 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:51.063210 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:51.119489 28157 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:51.049693 29983 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:51.100776 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:51.121471 28231 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:51.106600 28160 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:51.102207 28230 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:51.108043 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:51.059767 30227 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:51.111239 28078 rpcz_store.cc:276] 0709 13:11:48.946182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.001408 (+ 55226us) service_pool.cc:224] Handling call
0709 13:11:49.001709 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.001927 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:49.234826 (+232899us) log.cc:844] Serialized 14 byte log entry
0709 13:11:50.879926 (+1645100us) spinlock_profiling.cc:243] Waited 5.1 ms on lock 0x7b18003a9ea0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:11:51.005984 (+126058us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:51.006052 (+    68us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:51.006117 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.006217 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:51.006984 (+   767us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":292599,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":247565,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"spinlock_wait_cycles":11735424,"thread_start_us":334581,"threads_started":2}
I20260709 13:11:51.080534 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:51.114998 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 108445us
I20260709 13:11:51.071707 30224 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:51.040037 30204 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:51.115325 30120 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:51.107012 30251 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:51.091290 30263 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:51.061335 30032 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:51.152498 28157 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:51.166648 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:51.100327 30104 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:51.171816 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:51.072062 30103 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:51.188473 28227 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:51.092368 30143 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:51.120379 28152 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:51.110387 30253 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:51.117424 30072 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:51.194046 28230 raft_consensus.cc:2410] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:11:51.192576 30258 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:11:51.208711 28227 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:11:51.167570 30224 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.199645 28157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:51.211969 28157 raft_consensus.cc:606] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:51.212574 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:51.196717 30254 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:11:51.218094 28153 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:51.193113 30233 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:51.194000 30120 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:51.118011 30089 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:51.180804 30227 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:51.154114 30108 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:11:51.197649 30225 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.063s
W20260709 13:11:51.221220 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 118179us
I20260709 13:11:51.292066 28155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:51.386125 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:51.237427 30255 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 13:11:51.197860 30109 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:51.385869 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 130653us
I20260709 13:11:51.386149 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:51.427089 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:51.428557 30214 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:51.429003 30214 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.429100 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 402) took 2938 ms. Trace:
I20260709 13:11:51.228910 28234 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:51.429350 28764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 587) took 2499 ms (client timeout 2126 ms). Trace:
I20260709 13:11:51.412832 28125 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:51.409926 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 238124us
W20260709 13:11:51.253844 30261 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:11:51.415779 30275 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:51.256428 30218 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.391s	user 0.001s	sys 0.011s
I20260709 13:11:51.427057 28229 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:51.412530 28155 raft_consensus.cc:606] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:51.451583 28155 raft_consensus.cc:614] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:11:51.429768 28764 rpcz_store.cc:269] 0709 13:11:48.930429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.930655 (+   226us) service_pool.cc:224] Handling call
0709 13:11:51.429113 (+2498458us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54004371d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:51.429242 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2370631552}
I20260709 13:11:51.429584 28760 rpcz_store.cc:276] 0709 13:11:48.490260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.490389 (+   129us) service_pool.cc:224] Handling call
0709 13:11:48.498833 (+  8444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:48.499248 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:48.499275 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:48.499774 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:11:48.907042 (+407268us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:48.907103 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.907205 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:49.901770 (+994565us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:49.945995 (+ 44225us) raft_consensus.cc:1685] finished
0709 13:11:49.946017 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:51.428884 (+1482867us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b5400686fd0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:51.429010 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1420,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":66696320,"thread_start_us":406825,"threads_started":1,"wal-append.queue_time_us":626804,"wal-append.run_cpu_time_us":17388,"wal-append.run_wall_time_us":1850274}
I20260709 13:11:51.451368 30152 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:51.216035 30220 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:51.207721 30263 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:51.297497 29916 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:51.454339 28231 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:11:51.411592 30233 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:51.229295 30259 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:11:51.215518 30204 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.460692 28234 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:51.218914 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:51.214218 30253 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:51.491371 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53728.1us
I20260709 13:11:51.316294 30086 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:51.455593 30225 consensus_queue.cc:1243] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:11:51.522842 28231 pending_rounds.cc:85] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 2
I20260709 13:11:51.248549 29908 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:51.401165 30274 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.229854 30196 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:51.416610 30136 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:51.523219 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 89849.7us
W20260709 13:11:51.528198 28077 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1799) took 2653 ms (client timeout 1537 ms). Trace:
I20260709 13:11:51.446246 30214 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:51.425170 30272 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:51.465198 30275 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:51.453814 30152 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:51.221755 30123 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:51.463457 30109 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:11:52.038995 30251 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.842s	user 0.009s	sys 0.000s
I20260709 13:11:52.039815 30109 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:52.040438 30109 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:52.041061 30108 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:52.036640 30249 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.101s	user 0.000s	sys 0.001s
I20260709 13:11:52.046423 30152 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:51.537060 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:52.048561 28077 rpcz_store.cc:269] 0709 13:11:48.875145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.916056 (+ 40911us) service_pool.cc:224] Handling call
0709 13:11:51.528000 (+2611944us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001ae0d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:51.528126 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332936448}
I20260709 13:11:52.039819 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1439) took 1238 ms. Trace:
W20260709 13:11:51.599835 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 171284us
W20260709 13:11:52.059754 28762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 595) took 2446 ms (client timeout 3071 ms). Trace:
W20260709 13:11:52.060272 28762 rpcz_store.cc:269] 0709 13:11:49.614920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.615127 (+   207us) service_pool.cc:224] Handling call
0709 13:11:52.059567 (+2444440us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b5400436554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:52.059696 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724247936}
W20260709 13:11:51.699640 30234 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.792s	user 0.011s	sys 0.000s
I20260709 13:11:52.118273 30234 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.052744 28080 rpcz_store.cc:276] 0709 13:11:50.801274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.025131 (+223857us) service_pool.cc:224] Handling call
0709 13:11:51.026427 (+  1296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:51.044038 (+ 17611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:51.044065 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:51.049073 (+  5008us) log.cc:844] Serialized 26 byte log entry
0709 13:11:51.445535 (+396462us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:51.445598 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.451593 (+  5995us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:51.637996 (+186403us) raft_consensus.cc:1685] finished
0709 13:11:51.638016 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:52.039753 (+401737us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18627,"raft.queue_time_us":13283,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":14715,"thread_start_us":340036,"threads_started":1,"wal-append.queue_time_us":454360}
W20260709 13:11:52.131983 28199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.008s	sys 0.028s
W20260709 13:11:52.134598 28199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.009s	sys 0.029s
W20260709 13:11:52.137133 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 602564us
W20260709 13:11:52.162400 28199 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.537s (SENT) [suppressed 20 similar messages]
I20260709 13:11:52.200110 28158 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:51.462570 30208 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:51.524710 30233 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:52.204941 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.201174 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:52.208189 28077 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:52.219080 28077 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:51.655488 28226 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:11:52.038683 30267 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.756s	user 0.001s	sys 0.000s
W20260709 13:11:51.650691 30129 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.936s	user 0.009s	sys 0.004s
I20260709 13:11:52.040130 30251 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:52.235715 28232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.028s	sys 0.029s
I20260709 13:11:51.537521 28231 pending_rounds.cc:107] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.3
W20260709 13:11:52.245675 28232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.028s	sys 0.029s
I20260709 13:11:51.627868 30143 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:52.247700 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:51.539130 30215 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:51.538180 30103 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:52.248548 28085 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
W20260709 13:11:52.203841 30016 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.006s	sys 0.032s
I20260709 13:11:52.265089 28231 raft_consensus.cc:2917] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Aborting config change with OpId 2.3: Aborted: Op aborted by new leader
I20260709 13:11:51.539003 30253 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:51.582583 30075 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:52.300426 28231 raft_consensus.cc:2889] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:52.268937 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:52.137976 29812 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:52.293310 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.205194 30219 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:52.225240 30272 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:52.304837 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1436) took 1504 ms. Trace:
I20260709 13:11:52.312831 28079 rpcz_store.cc:276] 0709 13:11:50.800366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.802078 (+  1712us) service_pool.cc:224] Handling call
0709 13:11:50.802532 (+   454us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:50.827747 (+ 25215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.827794 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:50.837978 (+ 10184us) log.cc:844] Serialized 52 byte log entry
0709 13:11:50.990288 (+152310us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:50.990355 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.990444 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:52.304523 (+1314079us) raft_consensus.cc:1685] finished
0709 13:11:52.304547 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:52.304702 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24655,"raft.queue_time_us":34121,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":3330,"thread_start_us":147235,"threads_started":1,"wal-append.queue_time_us":205378}
I20260709 13:11:52.247797 29962 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:52.301407 30016 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.006s	sys 0.033s
W20260709 13:11:52.327442 28082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.109s	user 0.000s	sys 0.012s
I20260709 13:11:52.319896 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:52.340534 28085 raft_consensus.cc:2368] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3.
I20260709 13:11:51.628132 30120 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.289566 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1462) took 1092 ms. Trace:
I20260709 13:11:52.200380 30136 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.316001 30196 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:52.341573 28082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.123s	user 0.000s	sys 0.013s
W20260709 13:11:52.335762 28083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1425) took 3390 ms (client timeout 1518 ms). Trace:
I20260709 13:11:52.310451 30089 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:52.361140 28083 rpcz_store.cc:269] 0709 13:11:48.945236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.947365 (+  2129us) service_pool.cc:224] Handling call
0709 13:11:52.335461 (+3388096us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54000e1354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:52.335602 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1000554752}
W20260709 13:11:52.303704 30224 consensus_meta.cc:287] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.076s	user 0.008s	sys 0.004s
I20260709 13:11:52.349655 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1696) took 1334 ms. Trace:
I20260709 13:11:52.362483 28228 rpcz_store.cc:276] 0709 13:11:51.015353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.015507 (+   154us) service_pool.cc:224] Handling call
0709 13:11:51.015815 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:51.016024 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:51.016048 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:51.016501 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 13:11:51.113270 (+ 96769us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:51.113308 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.113378 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:52.349423 (+1236045us) raft_consensus.cc:1685] finished
0709 13:11:52.349448 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:52.349592 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":73460,"threads_started":1,"wal-append.queue_time_us":181733}
I20260709 13:11:52.235067 30208 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.248229 30129 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.338801 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.308435 29812 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:52.295500 30108 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:11:52.040164 30274 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.365342 28085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
W20260709 13:11:52.373214 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 210843us
I20260709 13:11:52.336760 30227 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:51.872686 30263 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.371861 29812 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.383654 28079 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:52.311767 30075 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.348703 30273 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:52.399413 28078 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.901s	user 0.017s	sys 0.023s
W20260709 13:11:52.399923 28078 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.017s	sys 0.023s
I20260709 13:11:52.341490 29962 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.405090 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:44054 (request call id 13) took 15544 ms. Trace:
I20260709 13:11:52.337476 30219 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.350102 28226 rpcz_store.cc:276] 0709 13:11:51.171440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.192855 (+ 21415us) service_pool.cc:224] Handling call
0709 13:11:52.264070 (+1071215us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:52.398036 30133 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.558s	user 0.013s	sys 0.000s
W20260709 13:11:52.410421 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.410943 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 166968us
I20260709 13:11:52.411891 28080 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:52.349257 30214 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.407277 28231 rpcz_store.cc:276] 0709 13:11:36.842617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.842819 (+   202us) service_pool.cc:224] Handling call
0709 13:11:37.029461 (+186642us) log.cc:844] Serialized 506 byte log entry
0709 13:11:37.310125 (+280664us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b1800250900. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:11:37.766641 (+456516us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b1800250900. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59ce5d0b0b 00007f59ce655b63 00007f59ce67d752
0709 13:11:52.373092 (+14606451us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106712,"raft.run_cpu_time_us":710,"raft.run_wall_time_us":709,"spinlock_wait_cycles":12617984,"thread_start_us":209783,"threads_started":2,"wal-append.queue_time_us":224868,"wal-append.run_cpu_time_us":2058,"wal-append.run_wall_time_us":1047441}
I20260709 13:11:52.394829 30274 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:52.368978 30224 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:52.409235 30270 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.681s	user 0.000s	sys 0.001s
I20260709 13:11:52.403636 29812 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:52.390074 30288 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:52.434779 28729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.027s	sys 0.015s
I20260709 13:11:52.369879 30196 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.408319 29808 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:52.451046 28729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.027s	sys 0.015s
I20260709 13:11:52.431916 30109 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.463331 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:52.470600 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1697) took 1357 ms. Trace:
I20260709 13:11:52.432785 30133 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.489004 30206 raft_consensus.cc:2764] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:11:52.487685 28232 rpcz_store.cc:276] 0709 13:11:51.113413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.456045 (+342632us) service_pool.cc:224] Handling call
0709 13:11:51.456536 (+   491us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:51.461973 (+  5437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:51.462014 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:51.496045 (+ 34031us) log.cc:844] Serialized 52 byte log entry
0709 13:11:52.246523 (+750478us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:52.246581 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:52.246672 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:52.470199 (+223527us) raft_consensus.cc:1685] finished
0709 13:11:52.470231 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:52.470482 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":843128,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":29378,"thread_start_us":658268,"threads_started":2,"wal-append.queue_time_us":855205}
I20260709 13:11:52.489291 28083 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:52.472496 30136 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.402848 30273 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.443677 30279 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:52.492252 30279 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.492712 30208 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.411614 30272 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.473851 28085 raft_consensus.cc:606] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:52.494915 28085 raft_consensus.cc:614] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:52.485047 30274 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:52.495343 28152 consensus_meta.cc:287] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.274s	user 0.007s	sys 0.004s
I20260709 13:11:52.495708 28152 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:52.435793 30108 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.495627 30274 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.496510 30108 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.485625 29808 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:52.497679 29808 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.499545 30214 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:52.491667 29962 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:52.515166 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.527786 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1711) took 1924 ms. Trace:
I20260709 13:11:52.528406 28154 rpcz_store.cc:276] 0709 13:11:50.603010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.603129 (+   119us) service_pool.cc:224] Handling call
0709 13:11:50.621723 (+ 18594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:50.761094 (+139371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:52.527416 (+1766322us) spinlock_profiling.cc:243] Waited 97.6 ms on lock 0x7b540033e854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:52.527692 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224434048}
I20260709 13:11:52.529904 28231 pending_rounds.cc:107] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:11:52.530537 28231 raft_consensus.cc:2889] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:11:52.539484 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 030bc6df8e7e4f2da3b18613d20d9bdc: Aborted: Op aborted by new leader
I20260709 13:11:52.539945 30224 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:52.541143 30290 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:11:52.535241 30291 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:52.549392 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 4c9b061a7ca84a61ac13d29946a28169: Illegal state: Replica 11f10c12c03e47dcac88b050d4c1431b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:11:52.550098 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1443) took 1747 ms. Trace:
I20260709 13:11:52.550642 28082 rpcz_store.cc:276] 0709 13:11:50.802255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.154401 (+352146us) service_pool.cc:224] Handling call
0709 13:11:51.154757 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:51.169628 (+ 14871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:51.169664 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:51.170218 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 13:11:52.361214 (+1190996us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:52.361285 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:52.361387 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:52.542034 (+180647us) raft_consensus.cc:1685] finished
0709 13:11:52.542064 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:52.550008 (+  7944us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":16707,"thread_start_us":1143185,"threads_started":1,"wal-append.queue_time_us":1199966}
I20260709 13:11:52.553750 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 363) took 1058 ms. Trace:
I20260709 13:11:52.554155 28078 rpcz_store.cc:276] 0709 13:11:51.494945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.496494 (+  1549us) service_pool.cc:224] Handling call
0709 13:11:51.496931 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:51.497227 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:51.497263 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:51.497774 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 13:11:52.400370 (+902596us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:52.400411 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:52.400488 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:52.553566 (+153078us) raft_consensus.cc:1685] finished
0709 13:11:52.553586 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:52.553702 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":902250,"threads_started":1,"wal-append.queue_time_us":937125}
W20260709 13:11:52.557803 29989 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.513s	user 0.008s	sys 0.004s
I20260709 13:11:52.567303 30196 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:52.552796 30136 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.556988 30251 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.578383 30219 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.568967 30234 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.569514 29989 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4.
I20260709 13:11:52.474427 30288 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.549412 30291 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.574915 30133 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:52.595604 28077 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
W20260709 13:11:52.587556 29907 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.017s	sys 0.018s
W20260709 13:11:52.597633 29907 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.017s	sys 0.018s
I20260709 13:11:52.598505 28086 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:11:52.598894 28156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 321) took 1784 ms (client timeout 1566 ms). Trace:
W20260709 13:11:52.599365 28156 rpcz_store.cc:269] 0709 13:11:50.814820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.844073 (+ 29253us) service_pool.cc:224] Handling call
0709 13:11:52.598688 (+1754615us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b5400051c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:52.598821 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58257792}
I20260709 13:11:52.600227 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1434) took 1807 ms. Trace:
I20260709 13:11:52.600652 28086 rpcz_store.cc:276] 0709 13:11:50.793132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.796337 (+  3205us) service_pool.cc:224] Handling call
0709 13:11:50.796722 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:52.599095 (+1802373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:52.599948 (+   853us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b5400267fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:52.600162 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4142950144}
I20260709 13:11:52.602438 30227 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:52.602859 30227 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:52.603734 30227 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.605094 30251 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.605680 30251 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:52.604280 30204 consensus_meta.cc:287] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.067s	user 0.005s	sys 0.003s
I20260709 13:11:52.608013 30251 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.606389 30227 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:52.575145 30136 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.612555 30136 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.614349 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:52.615772 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:52.617090 28728 leader_election.cc:400] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3.
I20260709 13:11:52.617411 28728 leader_election.cc:403] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:52.618036 30297 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:52.618800 30297 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:52.618722 30014 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:52.619596 30014 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:52.620028 30014 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:52.619858 30297 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.620609 30135 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:52.633239 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.638612 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:52.638831 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:52.639424 28228 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:52.643008 28229 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:52.647321 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:52.648341 28232 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:52.657522 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.663825 28085 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:52.664793 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:52.668165 28232 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:52.653091 30263 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:52.673187 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:11:52.676651 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46488us
I20260709 13:11:52.678601 28228 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:52.676070 28226 pending_rounds.cc:85] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 2
I20260709 13:11:52.681195 28226 pending_rounds.cc:107] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:11:52.681506 28226 raft_consensus.cc:2889] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:52.649101 30286 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:11:52.682859 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:52.679893 30263 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:52.702260 28228 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:52.705756 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 2.31743e+06us
W20260709 13:11:52.706688 28050 connection.cc:256] RPC call timeout handler was delayed by -2.3155053650000355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
W20260709 13:11:52.707284 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.684521 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:52.712754 30291 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:52.724627 28730 connection.cc:256] RPC call timeout handler was delayed by -2.4935778189999382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 13:11:52.725153 28730 connection.cc:256] RPC call timeout handler was delayed by -2.4827778189999208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:11:52.725437 28730 connection.cc:256] RPC call timeout handler was delayed by -2.3018778189999693s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18020000000000003s for a response
W20260709 13:11:52.725679 28730 connection.cc:256] RPC call timeout handler was delayed by -2.2676778190000277s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18400000000000002s for a response
W20260709 13:11:52.725936 28730 connection.cc:256] RPC call timeout handler was delayed by -2.1590633859999571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 13:11:52.726531 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53485.4us
I20260709 13:11:52.727258 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:52.702248 30263 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:52.729947 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:52.730521 28232 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
W20260709 13:11:52.731930 28050 connection.cc:256] RPC call timeout handler was delayed by -1.7160470349999741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
W20260709 13:11:52.732067 28730 connection.cc:256] RPC call timeout handler was delayed by -1.9064071949999288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
I20260709 13:11:52.732861 30233 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.735585 29892 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.737282 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 365) took 1241 ms. Trace:
I20260709 13:11:52.738238 30135 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3.
I20260709 13:11:52.741164 30135 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.731483 30224 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.743245 30234 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.743120 30224 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.743768 30234 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.745352 30224 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.745775 30234 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.737744 28077 rpcz_store.cc:276] 0709 13:11:51.495561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.247192 (+751631us) service_pool.cc:224] Handling call
0709 13:11:52.737073 (+489881us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54000c99d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:52.737207 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":661857664}
I20260709 13:11:52.746752 30120 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.746579 30263 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.747301 30120 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.748112 30281 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:52.748564 30281 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:52.749043 30281 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:52.749135 28728 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:52.750434 30263 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.751214 30120 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.742496 30135 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.752977 30219 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.755537 30219 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.757261 30251 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:52.757901 30251 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:52.757967 30219 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:52.753155 28050 connection.cc:256] RPC call timeout handler was delayed by -1.5261470350000081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 13:11:52.759177 28234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1454) took 1723 ms (client timeout 2044 ms). Trace:
W20260709 13:11:52.759598 28050 connection.cc:256] RPC call timeout handler was delayed by -1.4640470350000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 13:11:52.760011 28050 connection.cc:256] RPC call timeout handler was delayed by -1.4145470350000551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18400000000000002s for a response
I20260709 13:11:52.753383 30234 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:52.761008 30234 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.728360 29989 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:52.762981 29989 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.763167 30234 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:52.753865 30120 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:52.758903 30251 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.754518 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:52.763393 29989 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:52.753422 30206 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:52.761386 30109 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:52.760529 28050 connection.cc:256] RPC call timeout handler was delayed by -1.3115063010000085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
I20260709 13:11:52.747740 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:11:52.765983 28730 connection.cc:256] RPC call timeout handler was delayed by -1.7229846699999598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
W20260709 13:11:52.771739 28730 connection.cc:256] RPC call timeout handler was delayed by -1.2821778190000259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29350000000000004s for a response
I20260709 13:11:52.728406 28228 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
W20260709 13:11:52.759789 28234 rpcz_store.cc:269] 0709 13:11:51.036122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.100446 (+ 64324us) service_pool.cc:224] Handling call
0709 13:11:52.758994 (+1658548us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54003caad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:52.759095 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261833088}
I20260709 13:11:52.776785 28762 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:52.778671 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:52.779258 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.779979 28760 raft_consensus.cc:2410] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:52.785030 28760 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:11:52.786799 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:52.787380 28234 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:52.756615 30086 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.735658 30233 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:52.748368 30014 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.802677 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:52.803536 28228 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:52.791126 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:52.805820 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:11:52.797928 28050 connection.cc:256] RPC call timeout handler was delayed by -1.3061063009999998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
W20260709 13:11:52.807575 28050 connection.cc:256] RPC call timeout handler was delayed by -1.0982063010000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
I20260709 13:11:52.763909 30120 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.808501 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:52.815861 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.816140 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.797067 29892 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:52.816713 28086 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:52.817721 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:52.770985 30253 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.819262 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.823021 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.819855 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.819727 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.821770 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:52.824236 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.825055 28082 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:52.825762 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.826375 28085 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:52.811273 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:52.827253 28078 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:52.827769 28234 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:11:52.828684 28086 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:52.782595 30206 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.812665 30219 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:52.830587 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.829082 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:52.832249 28086 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:52.837116 28078 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:52.800223 30086 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:52.843911 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:52.835781 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:52.836089 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.846048 28082 raft_consensus.cc:2410] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:11:52.817566 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:52.846688 28228 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:52.841776 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:52.847204 30127 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
W20260709 13:11:52.849696 28756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 600) took 2796 ms (client timeout 1838 ms). Trace:
I20260709 13:11:52.842736 30093 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:11:52.847682 30304 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:11:52.850837 30093 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:52.851169 28758 raft_consensus.cc:2368] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6.
I20260709 13:11:52.851393 30093 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:52.825346 30233 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:52.821381 29812 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.853169 29812 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:52.858464 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:52.859064 30014 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:52.859248 28228 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:52.860714 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:52.860280 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:52.862231 28226 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:52.863950 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:52.865195 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:52.866806 28730 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:52.866962 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:52.867832 28086 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:52.815135 30234 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:11:52.874480 28085 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:52.876157 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:52.877998 28232 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:52.872071 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:52.833853 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:52.834389 30109 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:52.846167 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:52.893963 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47713.3us
I20260709 13:11:52.895222 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:52.856436 29812 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:52.836709 30219 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:52.826968 29892 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:52.873114 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:52.885754 28234 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:52.902587 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:52.985826 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:52.986322 28086 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:52.996631 30319 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:52.997363 30319 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.997788 30319 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:52.877687 30014 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:53.000885 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 127777us
W20260709 13:11:53.001902 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 106824us
I20260709 13:11:53.006857 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:52.892207 30089 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.892828 30233 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:52.896615 30285 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.892623 30109 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.902611 30263 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:52.992619 30280 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:53.011665 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:52.985543 30219 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:52.850289 28756 rpcz_store.cc:269] 0709 13:11:50.052895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.054381 (+  1486us) service_pool.cc:224] Handling call
0709 13:11:52.829855 (+2775474us) spinlock_profiling.cc:243] Waited 8.21 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:52.849569 (+ 19714us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18879360}
I20260709 13:11:52.875293 29907 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.129s
I20260709 13:11:52.842918 30256 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.153s
I20260709 13:11:53.000579 30272 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:53.026664 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:52.992179 29962 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:11:53.010452 30313 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:53.021137 30109 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.047317 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:53.028329 30250 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:53.026265 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:53.053437 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 187883us
I20260709 13:11:53.057641 30305 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:53.058820 28730 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:53.033550 30280 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:53.059878 29962 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:11:53.061697 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 72322.7us
I20260709 13:11:53.041786 30014 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:11:53.071017 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.566s (SENT)
I20260709 13:11:53.049284 30263 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:53.061704 30250 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.024044 30285 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.084311 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.054929 30306 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:53.022473 30089 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.086560 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:53.103380 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 44908us
I20260709 13:11:53.103542 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.080687 29962 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:53.060402 29892 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.087121 30120 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:53.088361 30086 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.088697 28154 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:53.115724 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:53.089480 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.117107 28156 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
I20260709 13:11:53.060674 30135 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.089919 30281 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.106274 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:53.090534 30273 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:53.092960 30250 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:53.102427 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:11:53.168529 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 66180us
I20260709 13:11:53.113911 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1674) took 3118 ms. Trace:
I20260709 13:11:53.169531 28235 rpcz_store.cc:276] 0709 13:11:49.995240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.995494 (+   254us) service_pool.cc:224] Handling call
0709 13:11:50.006989 (+ 11495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:50.007221 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.007248 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:52.806987 (+2799739us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b2c0004c898. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:11:52.807456 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:11:52.887010 (+ 79554us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:52.887076 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:52.887281 (+   205us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:53.090800 (+203519us) raft_consensus.cc:1685] finished
0709 13:11:53.090820 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:53.113835 (+ 23015us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6611840,"thread_start_us":60251,"threads_started":1,"wal-append.queue_time_us":238106}
I20260709 13:11:53.081544 30305 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:53.107708 28157 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:53.178514 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.108201 30306 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.141983 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:53.109570 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.163173 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:53.164919 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.181298 28762 raft_consensus.cc:2410] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:53.181430 28156 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:11:53.164302 28159 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:53.165732 30286 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:53.182793 28081 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:53.174615 30307 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:11:53.185741 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 287578us
I20260709 13:11:53.188442 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.191460 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:53.193427 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1720) took 2578 ms. Trace:
I20260709 13:11:53.139586 30308 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.193825 28159 rpcz_store.cc:276] 0709 13:11:50.606302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.629517 (+ 23215us) service_pool.cc:224] Handling call
0709 13:11:53.143174 (+2513657us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:11:53.164002 (+ 20828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:53.183200 (+ 19198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.184271 (+  1071us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28358912}
I20260709 13:11:53.162086 30253 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.193836 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.197515 28159 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:53.198028 28159 raft_consensus.cc:606] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:53.198469 28159 raft_consensus.cc:614] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:53.201613 30089 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:53.202220 30317 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:11:53.202445 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:53.202986 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.203841 28159 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:53.204417 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:53.204702 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.205065 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.205191 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.205417 28763 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:53.205564 28156 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:53.205789 28157 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
I20260709 13:11:53.208841 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:53.203220 30264 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 2.021s	user 0.007s	sys 0.014s
I20260709 13:11:53.182382 30306 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:53.217160 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 210836us
I20260709 13:11:53.218232 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:53.183615 30286 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:53.219619 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 104036us
I20260709 13:11:53.194335 30206 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.230346 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.243367 30206 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:53.212284 30307 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:53.212937 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:53.207422 30285 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.225308 30286 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:53.249701 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:53.251545 28234 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:53.225618 30274 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:53.255064 28227 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:53.252981 30228 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:11:53.196110 29962 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.263912 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:53.265935 30252 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:11:53.266767 30252 consensus_queue.cc:1243] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:53.269692 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1688) took 2342 ms. Trace:
I20260709 13:11:53.270231 28233 rpcz_store.cc:276] 0709 13:11:50.926846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.927042 (+   196us) service_pool.cc:224] Handling call
0709 13:11:50.927458 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:50.927910 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.927950 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:50.941756 (+ 13806us) log.cc:844] Serialized 26 byte log entry
0709 13:11:51.035720 (+ 93964us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:51.035778 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.035874 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:53.175888 (+2140014us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:53.263308 (+ 87420us) raft_consensus.cc:1685] finished
0709 13:11:53.263339 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:53.269616 (+  6277us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":8632,"spinlock_wait_cycles":6247296,"thread_start_us":50625,"threads_started":1,"wal-append.queue_time_us":143111}
I20260709 13:11:53.271929 28227 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
W20260709 13:11:53.272747 28227 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 1698) took 2060 ms (client timeout 1804 ms). Trace:
I20260709 13:11:53.273195 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:53.273947 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.273329 28227 rpcz_store.cc:269] 0709 13:11:51.212158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.214627 (+  2469us) service_pool.cc:224] Handling call
0709 13:11:53.249489 (+2034862us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b58001c0598. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:53.272624 (+ 23135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9566976}
I20260709 13:11:53.198967 30280 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.278353 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:53.275899 29962 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.286608 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.287567 28154 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:11:53.290211 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:53.292241 28233 raft_consensus.cc:2410] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:53.294160 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:53.297127 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.295089 28227 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:53.303938 28154 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:53.307850 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.310369 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.313650 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.314570 28157 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:53.275256 30274 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:53.324090 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:53.325134 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.326115 28082 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:53.328897 28233 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:53.329734 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:53.330554 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.333460 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:53.334905 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.334632 30307 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.334821 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.338126 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.339555 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.340247 28227 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:53.341260 28082 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:11:53.348322 30234 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:53.352035 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 63975.5us
I20260709 13:11:53.352296 30234 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.353549 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.355278 30234 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.365155 30281 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.295522 30253 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 13:11:53.374759 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:53.370553 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:53.327286 30274 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:53.377583 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:53.379763 28084 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:53.387357 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.388347 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.249568 30305 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.347581 30120 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:53.395835 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:53.401221 30328 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:53.402094 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.403558 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:53.404265 30327 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:53.404718 30109 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.404646 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:53.411147 28198 leader_election.cc:336] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Timed out: RequestConsensusVote RPC to 127.26.83.196:46855 timed out after 2.008s (SENT)
I20260709 13:11:53.390666 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.417215 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.418309 28228 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:53.383741 30253 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:11:53.389829 30093 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.005s
I20260709 13:11:53.425100 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.392535 30147 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:53.449386 28228 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:11:53.423576 29989 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.650s	user 0.013s	sys 0.000s
I20260709 13:11:53.458496 28085 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:53.427515 28156 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:11:53.429531 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.423791 30335 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:53.437347 30319 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.408674 30109 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.439549 30272 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:53.465595 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:53.466012 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:53.473773 28227 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:53.473875 28233 raft_consensus.cc:2376] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7. Re-sending same reply.
I20260709 13:11:53.475780 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.477342 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.478173 28082 raft_consensus.cc:2376] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3. Re-sending same reply.
I20260709 13:11:53.422921 30120 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.483148 30129 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.484259 30120 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:53.485474 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:53.486222 28233 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.460055 30253 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.489812 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:53.492990 28227 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:11:53.463063 30335 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:53.497498 30335 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:53.508080 29911 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:53.453444 30093 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:53.509398 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.487691 30302 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.812s	user 0.001s	sys 0.005s
I20260709 13:11:53.486370 30319 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.460623 29989 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.512339 28233 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:53.492511 30253 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.531092 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.456070 30274 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.538961 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.544088 28155 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:53.546461 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.555281 28077 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:53.558431 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:53.553360 28157 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
W20260709 13:11:53.547411 30219 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.003s
I20260709 13:11:53.567087 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.563540 28084 raft_consensus.cc:2368] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4.
I20260709 13:11:53.561509 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:53.578289 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 183006us
I20260709 13:11:53.566031 30274 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:53.579903 28161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 313) took 4667 ms (client timeout 2241 ms). Trace:
W20260709 13:11:53.583118 28161 rpcz_store.cc:269] 0709 13:11:48.910304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.910466 (+   162us) service_pool.cc:224] Handling call
0709 13:11:53.577685 (+4667219us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54003449d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:53.577890 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025737600}
I20260709 13:11:53.586942 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.569473 30273 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:53.591398 30101 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:53.603689 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.606159 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.601341 30101 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:53.585170 30330 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:11:53.613125 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.579291 30299 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:53.541159 30286 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.618805 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.571277 30219 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.601315 30273 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.637413 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.640460 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.616786 30093 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:53.589998 30272 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:11:53.604072 30109 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:11:53.651697 28159 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:53.641808 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:53.680651 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 2.28831e+06us
I20260709 13:11:53.694993 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:53.701064 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:53.701793 28233 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:53.694352 30104 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.711715 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.712599 28082 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:53.716171 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.717545 28153 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:53.673789 30101 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:53.670192 29989 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:53.675351 30273 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:53.649978 30308 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:53.629309 30299 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.724785 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.695852 30286 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.728135 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.754364 28157 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:11:53.731072 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.716389 28233 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:53.698262 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.756361 28161 raft_consensus.cc:2410] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:53.757364 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:53.758544 30340 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:11:53.742182 30253 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.751720 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:53.759503 28082 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:53.760416 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.762128 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:53.762979 28082 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:53.738651 30101 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:53.758467 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.764245 30101 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:53.764689 30101 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:53.765672 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.766386 28755 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:11:53.720556 30104 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.767935 30284 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:53.768429 30016 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.678249 30335 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.769026 30016 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.770591 30335 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:53.770002 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.745184 28153 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.775892 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:11:53.782778 30306 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.004s
I20260709 13:11:53.782887 28228 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:53.783288 30306 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.770896 28082 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:53.786680 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.788445 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.789477 28161 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
W20260709 13:11:53.752650 30250 consensus_meta.cc:287] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.006s
I20260709 13:11:53.800874 28234 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:11:53.801317 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 106407us
I20260709 13:11:53.802093 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:53.803020 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.803795 28228 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:53.799510 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:53.811221 28153 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:53.810101 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:53.811779 28157 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:53.785492 30016 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.810868 28760 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:53.793237 30219 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:53.814411 28124 connection.cc:256] RPC call timeout handler was delayed by -1.1423677940000516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1869s for a response
I20260709 13:11:53.813756 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.806110 30108 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:53.815615 28082 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:53.816118 28124 connection.cc:256] RPC call timeout handler was delayed by -1.1070453180000186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2042s for a response
W20260709 13:11:53.817234 28124 connection.cc:256] RPC call timeout handler was delayed by -1.0269453180000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
I20260709 13:11:53.819489 28124 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:53.815990 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:53.820924 28124 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:53.820573 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.822233 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.823849 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:53.822232 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:53.816666 30328 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.816766 30343 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:53.831615 28157 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:53.832397 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:53.825466 30120 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:53.828140 30328 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.843230 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:53.841570 28082 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:53.839648 30016 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.829257 30219 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:53.830998 30103 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:53.845486 30104 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:53.886629 30103 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.817164 30338 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:11:53.901031 28082 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:53.807313 30250 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:53.896914 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 53766.8us
I20260709 13:11:53.827417 30233 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:53.843909 30284 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:53.908367 28199 leader_election.cc:400] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4.
I20260709 13:11:53.920529 28079 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:53.914059 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:11:53.893605 30310 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:11:53.920524 28199 leader_election.cc:403] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:11:53.887322 30103 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:53.910470 30343 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:53.907706 30288 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:53.889213 30016 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.932260 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:53.913645 30104 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.935981 30299 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:53.911911 30233 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.944108 28227 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:53.926203 30152 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:53.950073 30152 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:53.944231 30328 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:53.930581 30284 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.957919 30075 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:53.938774 30288 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4.
I20260709 13:11:53.963217 30346 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:11:53.928777 30308 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:11:53.951915 30343 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.936307 30104 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:53.937116 30307 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:11:53.933411 30313 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:11:53.976706 30313 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:53.977277 30313 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:53.979425 30232 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.906082 30349 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:11:53.963238 30101 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.986325 30101 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:53.990087 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 329) took 1357 ms. Trace:
W20260709 13:11:53.984025 30347 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:11:53.960189 30152 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:53.990682 28229 rpcz_store.cc:276] 0709 13:11:52.632573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.632691 (+   118us) service_pool.cc:224] Handling call
0709 13:11:52.633025 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:52.643452 (+ 10427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.989775 (+1346323us) spinlock_profiling.cc:243] Waited 9.64 ms on lock 0x7b54001c7fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:53.990005 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22163200}
I20260709 13:11:53.953774 30233 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:53.961737 30284 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:53.971859 30343 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:53.978580 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:53.983155 30016 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:53.990891 30306 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:53.998834 30349 consensus_queue.cc:1243] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:11:53.978587 30313 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:54.104000 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 276069us
I20260709 13:11:54.005995 30152 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.110867 30313 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.101017 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35896 (request call id 626) took 1245 ms. Trace:
I20260709 13:11:54.110253 30104 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:54.118505 30114 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:54.116062 28757 rpcz_store.cc:276] 0709 13:11:52.775527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.782340 (+  6813us) service_pool.cc:224] Handling call
0709 13:11:53.978436 (+1196096us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552eee 0000000000375939 00007f59d35c61e0 00007f59d368f0c4 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:54.021201 (+ 42765us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25668864}
I20260709 13:11:54.131524 30343 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.118225 30250 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:54.133284 30350 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:11:54.163828 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:54.164876 28759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35896 (request call id 631) took 1363 ms. Trace:
I20260709 13:11:54.165308 28759 rpcz_store.cc:276] 0709 13:11:52.801094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.803707 (+  2613us) service_pool.cc:224] Handling call
0709 13:11:54.163635 (+1359928us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:54.164789 (+  1154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11964160}
I20260709 13:11:54.136229 30114 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:54.174249 30250 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:54.190295 30123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.514s	user 0.015s	sys 0.041s
I20260709 13:11:54.198611 30250 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.205763 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:54.206730 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 274579us
I20260709 13:11:54.208362 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:54.217013 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:54.205487 30123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.540s	user 0.015s	sys 0.041s
I20260709 13:11:54.230782 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:54.231503 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:54.210419 30230 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.233124 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:54.238780 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:54.239084 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1714) took 1423 ms. Trace:
I20260709 13:11:54.239506 28227 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:54.224082 30281 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.210263 30306 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:11:54.247601 28757 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:54.250775 29983 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.253486 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 364) took 2758 ms. Trace:
I20260709 13:11:54.253937 28083 rpcz_store.cc:276] 0709 13:11:51.495280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.368241 (+872961us) service_pool.cc:224] Handling call
0709 13:11:52.368639 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:52.489755 (+121116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.253235 (+1763480us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540017c854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:54.253421 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277007744}
I20260709 13:11:54.261544 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:54.263499 28227 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:54.266873 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.267161 28227 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:54.268095 28161 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:54.267771 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:54.271466 28198 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:54.274153 29983 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:11:54.274178 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 365957us
I20260709 13:11:54.275043 29983 consensus_queue.cc:1243] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:11:54.275590 29983 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:54.275930 29983 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:54.272253 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:54.277729 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:54.248209 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:54.276209 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:54.220980 30114 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.283591 28230 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:54.284263 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1456) took 3247 ms. Trace:
I20260709 13:11:54.284679 28230 rpcz_store.cc:276] 0709 13:11:51.036582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.215086 (+178504us) service_pool.cc:224] Handling call
0709 13:11:54.279621 (+3064535us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b58001c0598. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:11:54.279816 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:54.283977 (+  4161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.284206 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12891904}
I20260709 13:11:54.287351 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:54.288654 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:54.271310 30327 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:54.294138 30123 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:54.295306 30123 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:54.295666 30123 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:54.298009 30284 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.301627 30114 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.245461 30230 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.277923 30291 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:54.258266 30281 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:54.312047 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.281615 29907 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:54.304082 30233 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.289937 29983 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.348232 29983 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:54.350625 30230 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:54.306186 30287 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:54.310038 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:11:54.356467 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46633.1us
I20260709 13:11:54.357288 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:54.290256 30343 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:54.358882 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:54.290305 30086 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:54.362735 30367 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:54.339919 30104 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:54.307421 30327 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.366703 30288 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:54.370276 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:54.374207 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:54.355530 30287 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.349609 30103 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.358333 30291 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:54.366705 30250 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:54.372438 30367 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:54.372061 30274 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.374214 30327 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.379534 30287 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.375222 30346 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:54.347797 30123 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.393115 30281 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:54.371726 30284 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.395319 30233 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:54.383443 30104 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
W20260709 13:11:54.398317 30365 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:11:54.373769 30086 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:54.407050 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 379) took 1566 ms. Trace:
I20260709 13:11:54.377198 30299 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:54.422439 28158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1722) took 3807 ms (client timeout 1716 ms). Trace:
I20260709 13:11:54.433043 30299 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:11:54.407719 30366 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:11:54.433090 28158 rpcz_store.cc:269] 0709 13:11:50.614739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.617755 (+  3016us) service_pool.cc:224] Handling call
0709 13:11:54.422130 (+3804375us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b540010bb54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:54.422339 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3472101120}
I20260709 13:11:54.433677 30299 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:54.429231 30103 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.435240 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1756) took 1020 ms. Trace:
I20260709 13:11:54.430346 30123 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:54.435595 28158 rpcz_store.cc:276] 0709 13:11:53.415069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.434346 (+1019277us) service_pool.cc:224] Handling call
0709 13:11:54.434695 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:54.434903 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:54.434932 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:54.434974 (+    42us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:54.435009 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.435065 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.435187 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:54.438827 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1749) took 1089 ms. Trace:
I20260709 13:11:54.439291 28157 rpcz_store.cc:276] 0709 13:11:53.349724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.833484 (+483760us) service_pool.cc:224] Handling call
0709 13:11:53.833884 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:53.997428 (+163544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:53.997461 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:54.000214 (+  2753us) log.cc:844] Serialized 26 byte log entry
0709 13:11:54.211595 (+211381us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:54.356324 (+144729us) log.cc:844] Serialized 14 byte log entry
0709 13:11:54.357082 (+   758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.357173 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:54.438607 (+ 81434us) raft_consensus.cc:1685] finished
0709 13:11:54.438626 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.438768 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77324,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":4550,"spinlock_wait_cycles":7525120,"thread_start_us":389322,"threads_started":3,"wal-append.queue_time_us":167590}
I20260709 13:11:54.431834 30109 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:54.440884 30109 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:54.441746 30109 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.443087 30109 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:54.443595 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1752) took 1069 ms. Trace:
W20260709 13:11:54.444880 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.444015 28154 rpcz_store.cc:276] 0709 13:11:53.374181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.890004 (+515823us) service_pool.cc:224] Handling call
0709 13:11:53.890457 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:54.209692 (+319235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:54.209719 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:54.211497 (+  1778us) log.cc:844] Serialized 26 byte log entry
0709 13:11:54.375489 (+163992us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:54.375533 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.375597 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:54.443376 (+ 67779us) raft_consensus.cc:1685] finished
0709 13:11:54.443397 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.443536 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21687,"spinlock_wait_cycles":5940480,"thread_start_us":448318,"threads_started":2,"wal-append.queue_time_us":212027}
I20260709 13:11:54.432893 28078 rpcz_store.cc:276] 0709 13:11:52.840778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.840998 (+   220us) service_pool.cc:224] Handling call
0709 13:11:52.841285 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:52.841524 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:52.893769 (+ 52245us) log.cc:844] Serialized 14 byte log entry
0709 13:11:52.898054 (+  4285us) log.cc:844] Serialized 14 byte log entry
0709 13:11:54.399821 (+1501767us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b18003b8c60. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59ce22dd70 00007f59ce221293 00007f59ce216cea 00007f59ce21723a 00007f59ce681a2a 00007f59ce6910bf 00007f59ce691038
0709 13:11:54.406590 (+  6769us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:54.406665 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:54.406726 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.406816 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.406975 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17919,"prepare.queue_time_us":156856,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":90,"raft.queue_time_us":15404,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":21651,"spinlock_wait_cycles":225731584}
I20260709 13:11:54.406312 30367 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:54.427984 29907 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:54.471832 30086 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.473366 29808 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.475046 30299 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.477470 30032 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.488627 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1451) took 3453 ms. Trace:
I20260709 13:11:54.501241 28231 rpcz_store.cc:276] 0709 13:11:51.035388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.208090 (+172702us) service_pool.cc:224] Handling call
0709 13:11:51.208462 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:52.531558 (+1323096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.488289 (+1956731us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54003566d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:54.488535 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564625792}
I20260709 13:11:54.417557 30274 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:54.496485 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 215012us
I20260709 13:11:54.461593 30104 consensus_queue.cc:1243] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:11:54.529503 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:11:54.531587 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 155783us
I20260709 13:11:54.535209 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:54.547011 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:54.550210 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1760) took 1102 ms. Trace:
I20260709 13:11:54.550783 28156 rpcz_store.cc:276] 0709 13:11:53.447664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.440976 (+993312us) service_pool.cc:224] Handling call
0709 13:11:54.441336 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:54.441546 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:54.468290 (+ 26744us) log.cc:844] Serialized 14 byte log entry
0709 13:11:54.549748 (+ 81458us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:54.549821 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:54.549881 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.549971 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.550146 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6933,"prepare.queue_time_us":31517,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"spinlock_wait_cycles":25984,"thread_start_us":79354,"threads_started":1}
I20260709 13:11:54.552688 28154 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: []
I20260709 13:11:54.548353 30133 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:54.552567 30075 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:54.570124 30250 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:54.592437 30327 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.593087 30075 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:54.594249 30075 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.594888 30327 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:54.597174 30299 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:54.599896 28156 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:54.600386 28156 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:54.601123 28156 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:54.607467 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.613493 28156 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.605340 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:54.607398 28081 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:54.607933 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:54.614943 28082 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:54.617388 28083 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:54.620042 28084 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:54.620293 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:54.620718 30228 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:11:54.621693 28152 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 316) took 3808 ms (client timeout 1747 ms). Trace:
W20260709 13:11:54.622097 28152 rpcz_store.cc:269] 0709 13:11:50.813224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.860692 (+ 47468us) service_pool.cc:224] Handling call
0709 13:11:54.621509 (+3760817us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54003444d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:54.621624 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263035264}
I20260709 13:11:54.619894 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 334) took 1416 ms. Trace:
I20260709 13:11:54.623116 28155 rpcz_store.cc:276] 0709 13:11:53.203325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.209908 (+  6583us) service_pool.cc:224] Handling call
0709 13:11:53.210282 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:53.560964 (+350682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.619657 (+1058693us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54000a32d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:54.619817 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767248640}
I20260709 13:11:54.623765 28152 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:11:54.624220 28152 pending_rounds.cc:85] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 2
I20260709 13:11:54.624485 28152 pending_rounds.cc:107] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:11:54.624785 28152 raft_consensus.cc:2889] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:54.624995 28160 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:54.625278 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:43180 (request call id 8) took 1036 ms. Trace:
I20260709 13:11:54.625653 28155 rpcz_store.cc:276] 0709 13:11:53.588812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.624098 (+1035286us) service_pool.cc:224] Handling call
0709 13:11:54.625176 (+  1078us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:54.627813 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 798c63c7bb2e4373af20f55bbe872cf2: Illegal state: Replica 611ec64ec58244fe82058d6130e4e6f3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:11:54.628347 30285 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.629027 30275 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.628949 30285 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.582185 30133 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:54.616175 30281 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:11:54.644143 28156 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:54.604578 30250 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:54.631162 30376 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:11:54.648686 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet b99bb21dbacb4d63839f1403ef9b08b9: Illegal state: Replica 611ec64ec58244fe82058d6130e4e6f3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:11:54.617667 30279 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.569855 29907 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.661872 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1466) took 1277 ms. Trace:
I20260709 13:11:54.662350 28085 rpcz_store.cc:276] 0709 13:11:53.384615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.403281 (+ 18666us) service_pool.cc:224] Handling call
0709 13:11:54.661654 (+1258373us) spinlock_profiling.cc:243] Waited 37.9 ms on lock 0x7b54000c8fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:54.661802 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87083392}
I20260709 13:11:54.673730 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1757) took 1257 ms. Trace:
I20260709 13:11:54.674374 28158 rpcz_store.cc:276] 0709 13:11:53.415946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.436611 (+1020665us) service_pool.cc:224] Handling call
0709 13:11:54.439892 (+  3281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:54.440286 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:54.440316 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:54.440813 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:11:54.566043 (+125230us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:54.566098 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.566238 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:54.673443 (+107205us) raft_consensus.cc:1685] finished
0709 13:11:54.673467 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.673643 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7181,"spinlock_wait_cycles":11594624,"thread_start_us":124779,"threads_started":1,"wal-append.queue_time_us":204531}
I20260709 13:11:54.681850 30285 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:54.687022 28764 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:11:54.686607 30287 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.688979 30250 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.691700 30103 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:54.692185 30103 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:54.649528 30367 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.694607 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:54.695451 28085 raft_consensus.cc:2368] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3.
I20260709 13:11:54.695322 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:54.687844 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 623) took 1913 ms. Trace:
I20260709 13:11:54.670604 30313 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:54.696043 28764 rpcz_store.cc:276] 0709 13:11:52.774368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.778209 (+  3841us) service_pool.cc:224] Handling call
0709 13:11:54.686533 (+1908324us) spinlock_profiling.cc:243] Waited 8.11 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:11:54.686774 (+   241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:54.687522 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.687775 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18661120}
I20260709 13:11:54.661690 30376 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:54.698132 30313 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.700420 30216 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:54.700992 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 165074us
I20260709 13:11:54.701370 30216 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.701742 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:54.702168 30216 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:54.703935 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:54.703697 29962 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:11:54.705004 30379 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:54.705545 29962 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:54.706198 29962 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:54.706568 30379 raft_consensus.cc:2764] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:11:54.707269 30379 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:54.709623 28230 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:54.710517 30274 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:54.712406 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.712214 30272 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:54.713711 30272 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:54.714145 30272 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:11:54.714473 28080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 371) took 3178 ms (client timeout 1610 ms). Trace:
I20260709 13:11:54.715058 30272 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.652488 30133 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.668646 30152 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:54.708329 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 338) took 1503 ms. Trace:
I20260709 13:11:54.710913 30367 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:54.742753 28159 rpcz_store.cc:276] 0709 13:11:53.204688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.205672 (+   984us) service_pool.cc:224] Handling call
0709 13:11:53.205919 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:53.688919 (+483000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.707780 (+1018861us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b5400376ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:54.708262 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1021774592}
W20260709 13:11:54.718039 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.731598 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:54.745379 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47464.8us
W20260709 13:11:54.746433 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:54.714905 28080 rpcz_store.cc:269] 0709 13:11:51.535514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.204589 (+669075us) service_pool.cc:224] Handling call
0709 13:11:54.714093 (+2509504us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54000c80d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:54.714251 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470057216}
I20260709 13:11:54.742172 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:11:54.738322 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.750139 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:54.751257 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:54.716643 30272 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:54.765949 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:54.707636 29962 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.768811 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:54.770792 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:54.771159 28764 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:54.753161 30233 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:11:54.771157 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:54.763147 30367 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.774559 28763 raft_consensus.cc:2410] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:54.761447 30343 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.779114 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.780001 28228 raft_consensus.cc:2410] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:54.744436 30307 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:11:54.775592 28231 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:54.776398 28227 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:54.784229 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.784382 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.775424 28230 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:11:54.784992 28227 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:54.785490 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:54.786126 28231 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:54.778121 30101 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:11:54.786971 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.780936 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.788132 28232 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:54.766131 30032 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:54.701485 30120 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:11:54.769912 29808 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:54.770017 30109 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:54.774304 30299 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:54.770144 30234 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:11:54.790261 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.792675 28227 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:54.795017 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.804031 28234 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:54.804383 28233 raft_consensus.cc:2410] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:11:54.773996 30274 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:54.798575 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.801798 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:54.790102 28228 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:54.764609 30152 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.795199 29962 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.789311 30313 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.808195 28231 raft_consensus.cc:2410] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:54.767493 30103 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.798512 29808 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:54.810786 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.814965 28764 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:54.817795 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:54.818430 28234 raft_consensus.cc:2368] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3.
W20260709 13:11:54.818954 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:54.821458 30391 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:11:54.812403 29808 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:54.858832 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1741) took 1539 ms. Trace:
I20260709 13:11:54.859570 28153 rpcz_store.cc:276] 0709 13:11:53.319195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.824008 (+504813us) service_pool.cc:224] Handling call
0709 13:11:53.824568 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:53.919802 (+ 95234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:53.919853 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:53.919919 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:53.919967 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.858181 (+938214us) spinlock_profiling.cc:243] Waited 94.9 ms on lock 0x7b54000802d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:54.858219 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.858345 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218380544}
I20260709 13:11:54.820152 30103 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:54.829360 30287 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.865093 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.865958 28232 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 6: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:54.803006 30109 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.870044 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:54.866874 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.872079 28231 raft_consensus.cc:2376] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4. Re-sending same reply.
I20260709 13:11:54.873523 28228 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:54.866796 28230 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
W20260709 13:11:54.870242 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.885135 28764 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:11:54.826887 30152 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:54.799381 30376 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.893231 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.894325 28230 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:54.894800 30109 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:54.895591 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.901731 30109 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.902550 30109 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.905023 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:54.905193 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:54.906148 30109 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:54.907524 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.910557 28232 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:54.911664 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:54.912372 28760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 613) took 1798 ms (client timeout 1923 ms). Trace:
I20260709 13:11:54.937804 28230 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:11:54.937320 28760 rpcz_store.cc:269] 0709 13:11:53.114472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.162246 (+ 47774us) service_pool.cc:224] Handling call
0709 13:11:54.912181 (+1749935us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b54006149d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:54.912306 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1389850496}
I20260709 13:11:54.939514 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:54.940788 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.941589 30253 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:54.941890 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.942655 28232 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:11:54.942983 30232 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:11:54.943512 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.944648 28232 raft_consensus.cc:2410] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:54.945698 28228 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:54.946444 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:54.915118 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:54.948994 28762 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:54.949205 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.918314 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.950654 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.951357 28153 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:54.919386 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.928238 29808 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.953073 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:54.953527 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.954182 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.955446 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:54.955317 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:54.956197 28154 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:11:54.914474 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.920079 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.921530 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.957213 28156 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:54.958068 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.919255 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.958895 28156 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:11:54.905891 28233 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:54.933621 28758 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:54.947609 28078 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:54.960480 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:54.915805 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:54.961406 28233 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:54.947002 28080 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:54.963476 30296 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.952421 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:54.964262 30296 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:54.965144 28159 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:11:54.965492 28728 leader_election.cc:336] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.747s (SENT)
I20260709 13:11:54.966236 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:54.947990 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:54.967587 28231 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:54.971738 28728 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:54.974694 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.975658 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.975805 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:54.976411 28763 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:11:54.975464 30109 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:11:54.977627 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:54.978369 28084 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:54.978838 30296 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:54.947986 30287 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.979851 30287 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:54.982048 30287 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:54.989419 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:54.990085 28159 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:54.977370 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:54.978971 30320 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:54.991691 28228 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:11:54.993739 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:54.994580 28232 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:11:54.996949 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:54.997751 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:54.998010 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.002916 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:55.004703 28156 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:11:55.002449 28231 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:54.955472 29808 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:55.010910 28228 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:55.012830 28152 raft_consensus.cc:2410] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:55.022161 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1450) took 1723 ms. Trace:
I20260709 13:11:55.022578 28079 rpcz_store.cc:276] 0709 13:11:53.298373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.298532 (+   159us) service_pool.cc:224] Handling call
0709 13:11:53.340348 (+ 41816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:53.925651 (+585303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.021905 (+1096254us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b540027bfd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:55.022097 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333712640}
I20260709 13:11:54.239538 28226 rpcz_store.cc:276] 0709 13:11:52.815814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.023047 (+207233us) service_pool.cc:224] Handling call
0709 13:11:53.077389 (+ 54342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:53.091287 (+ 13898us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:11:53.277732 (+186445us) log.cc:844] Serialized 14 byte log entry
0709 13:11:54.135052 (+857320us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b18003e4280. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a 00007f59ce681a2a
0709 13:11:54.238697 (+103645us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:54.238750 (+    53us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:54.238817 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.238898 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.239025 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":158981,"prepare.run_wall_time_us":38,"raft.queue_time_us":82,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":96,"spinlock_wait_cycles":37504640,"thread_start_us":199758,"threads_started":2}
I20260709 13:11:55.027945 28154 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:54.976851 30297 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:54.948261 30120 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:11:55.033941 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.036252 28159 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:55.037431 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:55.044284 28127 leader_election.cc:400] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3.
I20260709 13:11:55.044895 28127 leader_election.cc:403] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:11:55.046613 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.049907 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.020666 30343 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:55.053261 28227 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:11:55.053300 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 77037.7us
W20260709 13:11:55.054940 28728 connection.cc:256] RPC call timeout handler was delayed by -1.5924029840000458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
W20260709 13:11:55.057353 28728 connection.cc:256] RPC call timeout handler was delayed by -1.3521185910000213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
I20260709 13:11:55.051905 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:55.059384 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:55.059816 28762 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:55.060305 28763 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:55.060061 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:55.059844 28761 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:55.062999 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:55.063764 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:55.064122 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:55.064589 28154 raft_consensus.cc:2410] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:55.064621 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:55.067701 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:55.039685 30233 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:55.089587 28235 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:11:55.090333 28235 pending_rounds.cc:85] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 5
I20260709 13:11:55.090672 28235 pending_rounds.cc:107] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:11:55.091035 28235 raft_consensus.cc:2889] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:55.071094 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:55.063117 30272 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3.
I20260709 13:11:55.076493 30230 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:55.069262 30320 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:55.001452 30114 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:11:54.997139 30109 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:11:55.085325 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:55.093104 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:55.101378 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:55.108079 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:55.038971 30263 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:11:55.118417 29808 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:55.119871 30329 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.120461 30329 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.121428 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 377) took 2368 ms. Trace:
I20260709 13:11:55.121951 28077 rpcz_store.cc:276] 0709 13:11:52.753210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.764768 (+ 11558us) service_pool.cc:224] Handling call
0709 13:11:52.765263 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:53.562100 (+796837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.121022 (+1558922us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b5400267ad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:55.121297 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816088320}
I20260709 13:11:55.123034 29907 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:55.123600 29907 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:55.124107 29907 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:55.126170 30129 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:55.126937 27981 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 2 to 3, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 3 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:11:55.128173 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:55.130417 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:55.131582 30135 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:55.132198 28084 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:55.065864 30250 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:11:55.134907 30219 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:55.135382 30219 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:55.088557 30343 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:55.137378 30343 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:55.137786 30343 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:55.114849 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:55.116461 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:55.142084 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:55.069949 29974 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:55.143071 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:55.143707 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:55.145381 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:55.153116 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:55.146168 28079 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:11:55.148846 27981 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 2 to 3, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 3 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:55.158028 27981 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 2 to 3, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 3 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:55.155982 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:11:55.159884 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 95826.9us
I20260709 13:11:55.097165 30206 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:55.161587 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.163247 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:55.161820 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:55.166733 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:55.169966 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:55.132426 30135 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:55.102095 30103 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:55.170161 28761 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:11:55.170472 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.165551 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:55.203226 28082 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:55.202775 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.052086 30328 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:11:55.205040 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:55.205677 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:55.202728 27981 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:55.112215 30232 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:55.204764 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:55.210778 28083 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:55.211309 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:55.215981 28081 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:55.161670 30108 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:55.233278 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:55.164471 29808 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:55.164709 30272 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:11:55.175475 30204 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.014s	sys 0.006s
I20260709 13:11:55.167722 30206 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:55.235471 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:55.136310 30219 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:55.241817 28759 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:55.210716 30114 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:55.167773 30296 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:55.175827 30299 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:55.204326 30274 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:55.250072 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:55.210517 30032 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:11:55.233268 30398 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:11:55.213104 30232 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:55.126984 30129 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.239341 30108 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:55.240047 30272 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:55.240957 29974 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:55.167472 30287 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
W20260709 13:11:55.247014 30204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.020s	sys 0.017s
I20260709 13:11:55.315691 30232 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:11:55.306708 30395 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:11:55.325543 30274 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:55.327097 30274 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:55.328893 30032 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:55.330277 30129 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:55.320269 30108 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:55.330382 30032 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.331388 30343 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:55.312109 30219 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:55.275374 30206 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:55.334746 30343 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:55.344770 30404 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:55.339452 30367 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.561s	user 0.000s	sys 0.019s
I20260709 13:11:55.347461 30329 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:55.349704 30367 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:55.207737 27981 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:55.399742 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:55.401846 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:55.402590 28083 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:11:55.403457 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.407179 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.411324 28233 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:55.399299 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:55.276639 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:11:55.424374 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.308642 28764 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:55.428615 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:55.433234 30109 consensus_queue.cc:1243] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:11:55.434211 30216 consensus_meta.cc:287] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.006s
I20260709 13:11:55.337975 30114 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:55.435050 30216 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.435827 30103 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.436667 30103 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:55.451741 30296 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:55.314764 30135 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:55.467056 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.472035 28759 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:11:55.473713 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:55.481971 29907 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.475904 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.459975 30135 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:55.478040 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:55.478942 30216 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:55.482075 30103 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:11:55.456676 30296 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.501121 30296 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:55.503193 30286 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.341687 30272 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.505405 30286 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.451714 30287 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:55.459842 29974 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.423018 27981 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 2 to 4, now has a pending config: VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193), VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194), VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195), NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 4 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:11:55.508345 30272 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:55.517793 29907 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:55.533704 30206 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.534555 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1462) took 2185 ms. Trace:
I20260709 13:11:55.509177 30286 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:55.546845 28086 rpcz_store.cc:276] 0709 13:11:53.349044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.349384 (+   340us) service_pool.cc:224] Handling call
0709 13:11:54.718552 (+1369168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:54.719045 (+   493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:54.719086 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:54.719148 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:54.719205 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:54.719294 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.534026 (+814732us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000c80d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:55.534285 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3147900800}
I20260709 13:11:55.535810 27981 catalog_manager.cc:5697] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:55.313593 30120 consensus_queue.cc:1175] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "11f10c12c03e47dcac88b050d4c1431b" responder_term: 4 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:11:55.549918 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:55.513406 30367 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:55.508500 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 68970.5us
W20260709 13:11:55.508977 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 303265us
I20260709 13:11:55.558161 28230 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:55.559602 30329 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:55.560755 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:55.561736 28228 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:55.561872 28234 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:55.572975 28079 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:55.573976 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1754) took 2186 ms. Trace:
W20260709 13:11:55.577133 30351 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.581s	user 0.002s	sys 0.006s
I20260709 13:11:55.579756 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:55.580886 27981 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 4 to 5, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 5 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
W20260709 13:11:55.587009 29962 consensus_meta.cc:287] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.775s	user 0.007s	sys 0.007s
I20260709 13:11:55.574817 28161 rpcz_store.cc:276] 0709 13:11:53.386827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.284924 (+898097us) service_pool.cc:224] Handling call
0709 13:11:54.285394 (+   470us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:54.306281 (+ 20887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:54.306336 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:54.339050 (+ 32714us) log.cc:844] Serialized 52 byte log entry
0709 13:11:54.911716 (+572666us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b18003d0540. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:11:55.001909 (+ 90193us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:55.572000 (+570091us) log.cc:844] Serialized 14 byte log entry
0709 13:11:55.572767 (+   767us) log.cc:844] Serialized 14 byte log entry
0709 13:11:55.573036 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.573225 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:55.573405 (+   180us) raft_consensus.cc:1685] finished
0709 13:11:55.573418 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.573611 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":574211,"prepare.queue_time_us":576944,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":111,"raft.queue_time_us":7973,"raft.run_cpu_time_us":592,"raft.run_wall_time_us":28799,"spinlock_wait_cycles":35861760,"thread_start_us":89942,"threads_started":1,"wal-append.queue_time_us":670202}
I20260709 13:11:55.587797 30343 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.589008 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:11:55.590023 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 114661us
I20260709 13:11:55.527872 30287 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:55.621057 30367 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:55.632669 30320 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:55.634361 30320 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:55.634121 30343 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.637116 28228 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:11:55.637145 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.638586 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:55.640025 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:55.681501 28234 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:55.705066 28228 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:55.715449 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:55.717123 28077 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:55.717543 28759 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:11:55.717950 28759 pending_rounds.cc:85] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 5
I20260709 13:11:55.718207 28759 pending_rounds.cc:107] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:11:55.718528 28759 raft_consensus.cc:2889] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:55.720161 30234 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:55.708319 30108 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:55.720719 30234 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:55.722988 28228 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:55.724243 30234 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.721906 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 643) took 1130 ms. Trace:
I20260709 13:11:55.725942 28758 rpcz_store.cc:276] 0709 13:11:54.590982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.591149 (+   167us) service_pool.cc:224] Handling call
0709 13:11:54.933378 (+342229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:54.960154 (+ 26776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.721432 (+761278us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54006149d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:55.721686 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":786009344}
I20260709 13:11:55.727130 30232 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.732219 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.733140 30127 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:55.728404 30234 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:55.734016 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:55.741251 30274 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:55.748116 30286 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:55.748879 30297 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:55.751097 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:11:55.753057 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.778339 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.785432 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 209924us
I20260709 13:11:55.785761 30127 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:55.786568 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:55.788337 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.795715 30274 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.800948 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:55.636124 30320 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:55.821643 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.814253 29962 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:55.828658 29974 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:11:55.842078 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 110084us
I20260709 13:11:55.839790 30127 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:55.846204 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 58020.9us
I20260709 13:11:55.848106 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 637) took 1619 ms. Trace:
I20260709 13:11:55.838815 30206 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:55.854995 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 104165us
I20260709 13:11:55.858850 30307 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:55.859321 30279 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:11:55.859658 30320 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:55.859989 30279 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:55.860797 30328 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:55.861382 30279 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:55.862604 30279 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:55.863350 29907 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:11:55.869113 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.870235 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:55.871444 30274 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:55.874003 30129 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:55.874567 30129 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:55.876674 30297 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:55.877305 30297 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:55.877678 30297 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:55.878088 30388 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.111s	user 0.003s	sys 0.011s
W20260709 13:11:55.879034 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.879777 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.880609 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 352) took 1523 ms. Trace:
I20260709 13:11:55.856182 29962 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:55.881439 29962 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.881188 28229 rpcz_store.cc:276] 0709 13:11:54.357181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.357324 (+   143us) service_pool.cc:224] Handling call
0709 13:11:54.357680 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:54.502636 (+144956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:54.502667 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:54.529571 (+ 26904us) log.cc:844] Serialized 52 byte log entry
0709 13:11:54.583845 (+ 54274us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:54.605447 (+ 21602us) log.cc:844] Serialized 14 byte log entry
0709 13:11:54.709458 (+104011us) log.cc:844] Serialized 14 byte log entry
0709 13:11:54.709678 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.021781 (+312103us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b54001c7fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:55.021829 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:55.871842 (+850013us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:55.880109 (+  8267us) raft_consensus.cc:1685] finished
0709 13:11:55.880132 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.880393 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103254,"prepare.queue_time_us":34777,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":107042,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":10898,"spinlock_wait_cycles":47975424,"thread_start_us":158586,"threads_started":2,"wal-append.queue_time_us":125592}
I20260709 13:11:55.869179 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:55.858119 30232 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:55.894632 28229 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:55.895824 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:55.897950 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1759) took 2469 ms. Trace:
I20260709 13:11:55.899984 30135 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:55.898496 28157 rpcz_store.cc:276] 0709 13:11:53.428175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.440946 (+1012771us) service_pool.cc:224] Handling call
0709 13:11:54.441378 (+   432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:54.441733 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:54.441766 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:54.442374 (+   608us) log.cc:844] Serialized 52 byte log entry
0709 13:11:54.549360 (+106986us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:54.571390 (+ 22030us) log.cc:844] Serialized 14 byte log entry
0709 13:11:54.928363 (+356973us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b180041d580. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:11:55.545388 (+617025us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b180041d580. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:11:55.604605 (+ 59217us) log.cc:844] Serialized 14 byte log entry
0709 13:11:55.897329 (+292724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.897473 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:55.897578 (+   105us) raft_consensus.cc:1685] finished
0709 13:11:55.897593 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.897765 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1016924,"prepare.queue_time_us":1027200,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":112,"raft.queue_time_us":12333,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":8209,"spinlock_wait_cycles":8612480,"thread_start_us":358054,"threads_started":2,"wal-append.queue_time_us":154361,"wal-append.run_cpu_time_us":2315,"wal-append.run_wall_time_us":1009393}
I20260709 13:11:55.903153 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.908618 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:55.909813 30286 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:55.910323 30286 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:55.910780 30286 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:55.890720 30103 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:55.913254 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44276.9us
I20260709 13:11:55.913441 30103 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.914641 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:55.915849 30103 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:55.875721 30129 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:55.957393 28230 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:11:55.959283 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50804us
I20260709 13:11:55.940701 30206 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:55.977053 30206 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:55.977311 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 87140.8us
I20260709 13:11:55.977660 30206 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:55.978441 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.980189 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:55.980918 29931 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:55.983918 30204 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:55.984687 30204 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:55.986135 30204 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:55.958966 29962 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:56.040577 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.046427 28231 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:55.961059 30274 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:56.056015 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 75977.7us
I20260709 13:11:56.035053 30234 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:56.061707 30232 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:56.074174 28728 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.082723 28161 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:55.848696 28755 rpcz_store.cc:276] 0709 13:11:54.208574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.225520 (+ 16946us) service_pool.cc:224] Handling call
0709 13:11:54.276899 (+ 51379us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:54.339403 (+ 62504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:54.342954 (+  3551us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:54.343703 (+   749us) log.cc:844] Serialized 52 byte log entry
0709 13:11:55.537796 (+1194093us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b18004361e0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:11:55.704559 (+166763us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:55.705225 (+   666us) log.cc:844] Serialized 14 byte log entry
0709 13:11:55.705684 (+   459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.705871 (+   187us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:55.799627 (+ 93756us) raft_consensus.cc:1685] finished
0709 13:11:55.812087 (+ 12460us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:55.828288 (+ 16201us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3540,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":35023,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":7384,"spinlock_wait_cycles":35843840,"thread_start_us":143224,"threads_started":1,"wal-append.queue_time_us":1453219}
I20260709 13:11:56.071477 30103 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:56.108871 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.082842 29974 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:56.085845 30120 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:56.072826 30274 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.133322 30307 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:11:56.134909 30418 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:11:56.135275 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 224625us
W20260709 13:11:56.136498 30420 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:11:56.136880 30089 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.144630 28231 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:56.147498 30416 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.260s	user 0.003s	sys 0.000s
I20260709 13:11:56.148904 28229 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:56.148371 29974 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.150971 30103 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.165230 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:56.166566 28229 raft_consensus.cc:2410] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:56.168258 28229 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:56.163067 30367 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:56.169725 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1824) took 5122 ms. Trace:
W20260709 13:11:56.174197 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 148152us
I20260709 13:11:56.170310 30089 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.181381 28161 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:56.176846 28160 rpcz_store.cc:276] 0709 13:11:51.046939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.190963 (+144024us) service_pool.cc:224] Handling call
0709 13:11:54.624823 (+3433860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:54.625466 (+   643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:56.162882 (+1537416us) spinlock_profiling.cc:243] Waited 3.43 s on lock 0x7b54003444d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:56.169537 (+  6655us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7896192256}
I20260709 13:11:56.181675 29907 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:56.183451 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:56.184178 28755 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:56.183949 29907 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.184450 30103 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:56.186137 29907 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:56.187022 28229 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:56.189709 30286 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.190488 30274 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:56.192116 30343 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:56.195549 30206 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.197124 30297 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.197667 30089 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:56.198839 28231 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:56.200397 30328 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:56.181420 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:56.197057 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 126620us
I20260709 13:11:56.195971 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 1825) took 1068 ms. Trace:
I20260709 13:11:56.201119 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:56.202561 28077 rpcz_store.cc:276] 0709 13:11:55.127360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.144360 (+ 17000us) service_pool.cc:224] Handling call
0709 13:11:55.144625 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:55.717668 (+573043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:56.195697 (+478029us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b54003aa2d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:56.195891 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1315746560}
I20260709 13:11:56.206463 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.199410 30297 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:56.192229 30219 consensus_meta.cc:287] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.846s	user 0.014s	sys 0.000s
W20260709 13:11:56.215530 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 241833us
I20260709 13:11:56.217123 28086 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:56.221987 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61616us
I20260709 13:11:56.229353 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:56.201053 30328 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.231179 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:56.247844 30299 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:56.248412 30299 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:56.249593 30299 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.250943 30299 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:56.258417 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:56.259861 28231 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
W20260709 13:11:56.263664 30417 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.204s	user 0.002s	sys 0.000s
I20260709 13:11:56.265736 28760 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 1. Current term is 2. Ops: []
I20260709 13:11:56.209477 30367 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:11:56.268049 30287 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:56.274317 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:56.277364 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 58128.1us
I20260709 13:11:56.278699 30367 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:11:56.268563 29931 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:56.282914 28230 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:56.296898 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:56.299041 30307 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:56.299839 30307 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:56.300861 30376 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:11:56.303309 30376 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:11:56.280419 30422 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:56.307300 28086 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:56.284684 30367 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:56.308712 30367 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.310361 30367 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:56.284899 30286 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:56.312572 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:56.315922 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 92010.5us
W20260709 13:11:56.317003 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 83063.7us
I20260709 13:11:56.286387 30219 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:56.318249 30219 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:56.318749 30219 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:56.321250 30129 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:56.322901 30426 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:11:56.323542 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.306135 29931 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:56.338032 29931 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:56.341020 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.336437 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:56.347157 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:56.348963 30329 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:56.349455 30329 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:56.350653 30329 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.351263 30108 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:56.352173 30329 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:56.353034 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1490) took 1205 ms. Trace:
I20260709 13:11:56.353446 28079 rpcz_store.cc:276] 0709 13:11:55.147235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.157143 (+  9908us) service_pool.cc:224] Handling call
0709 13:11:55.157604 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:55.573639 (+416035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:56.343698 (+770059us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400209754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:56.352964 (+  9266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941452032}
I20260709 13:11:56.307184 30422 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:56.306128 30376 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:11:56.358007 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.357182 30422 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:56.361260 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 47457.1us
I20260709 13:11:56.363530 30286 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:11:56.343577 30206 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:56.370059 29974 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:56.370565 29974 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:56.370918 29974 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:56.373467 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:56.381755 30313 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:11:56.384214 30320 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:56.385020 30320 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:56.385071 30313 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:56.385625 30320 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:56.385684 30313 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:56.387107 30313 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.385087 30108 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:56.389850 30108 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.390447 30135 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:56.388597 30313 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:56.393901 30376 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.395373 30376 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.375128 30328 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:56.403494 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 67219.6us
I20260709 13:11:56.404228 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.406934 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.409102 30089 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:56.410704 30089 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:56.411033 30089 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:56.411984 30089 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.413276 28157 raft_consensus.cc:2410] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:11:56.391808 30286 consensus_queue.cc:1175] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "611ec64ec58244fe82058d6130e4e6f3" responder_term: 6 status { last_received { term: 5 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:11:56.428275 30206 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:56.430301 30135 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:56.430995 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.431878 28161 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:56.433897 30234 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:56.434203 28152 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:11:56.434664 28152 pending_rounds.cc:85] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 5
I20260709 13:11:56.434971 28152 pending_rounds.cc:107] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:11:56.435252 28152 raft_consensus.cc:2889] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:56.436376 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.437034 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.437225 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.438084 28160 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
W20260709 13:11:56.438977 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.440034 30089 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:56.440805 30427 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:56.441299 30427 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:56.442832 30427 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.444320 30427 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:56.424031 30120 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:56.446218 30120 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.447579 30135 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:56.449422 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.448711 30429 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:11:56.457269 28156 raft_consensus.cc:2410] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:56.448637 30120 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:56.451073 28124 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:56.459527 28159 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:56.460870 29962 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:56.461220 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:56.461833 29962 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:56.462608 28762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 632) took 1313 ms. Trace:
I20260709 13:11:56.407819 30328 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:11:56.463079 28762 rpcz_store.cc:276] 0709 13:11:55.148837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.149032 (+   195us) service_pool.cc:224] Handling call
0709 13:11:55.149505 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:55.150022 (+   517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:55.150070 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:56.190431 (+1040361us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b48000609d8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:11:56.355197 (+164766us) spinlock_profiling.cc:243] Waited 9.98 ms on lock 0x7b18000e3a40. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a
0709 13:11:56.355808 (+   611us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b2c00047778. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:11:56.356417 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 13:11:56.398604 (+ 42187us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:56.398636 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:56.398736 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:56.462188 (+ 63452us) raft_consensus.cc:1685] finished
0709 13:11:56.462214 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:56.462402 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1165,"raft.run_cpu_time_us":10923,"raft.run_wall_time_us":1028680,"spinlock_wait_cycles":42162944,"thread_start_us":41745,"threads_started":1,"wal-append.queue_time_us":89855}
I20260709 13:11:56.465405 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:56.466936 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.468055 28160 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:56.468544 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.469658 30032 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:56.470461 30032 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:11:56.471213 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.473352 30422 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.473762 28157 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:56.422034 30297 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:11:56.478206 30422 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:56.475237 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.481330 30108 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:56.455343 30320 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.481981 30108 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:11:56.482697 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 78526.9us
I20260709 13:11:56.483742 28157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:56.484031 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:56.473738 30206 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:56.485237 30428 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:56.490253 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.487519 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.488085 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.488308 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.489775 28152 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:11:56.490948 28158 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:56.491536 28154 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:11:56.491536 28155 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:56.507890 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.509389 28159 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:56.510644 28125 leader_election.cc:400] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 2. Current term is 3.
I20260709 13:11:56.511204 28156 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:56.512490 30320 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:56.513672 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.514575 28158 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:56.515556 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.516209 28154 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:56.475229 28755 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:11:56.520960 28161 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:56.521301 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.526952 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:11:56.541885 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.507997 30297 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:56.550540 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.551227 28158 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:56.552778 28154 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
W20260709 13:11:56.550050 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 63689.9us
I20260709 13:11:56.551378 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:56.557430 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:56.557611 28079 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:56.560362 28157 raft_consensus.cc:606] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:11:56.560916 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 99886.9us
I20260709 13:11:56.541318 30232 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:56.562022 28160 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:11:56.562470 28155 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:56.561064 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.549412 30428 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:56.568665 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:56.562325 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:56.571653 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:56.556111 30297 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:56.561481 30120 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.575296 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.574983 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:56.580716 28083 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:11:56.589663 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.589802 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:56.588187 30108 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.597713 30108 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.597946 30219 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.599905 30108 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:56.600031 30219 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:56.600543 29808 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:56.601490 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.602718 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.559737 30320 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:11:56.604908 28080 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:56.606685 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:56.609243 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:56.598034 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:56.619504 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:56.622187 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 49057.9us
I20260709 13:11:56.635195 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.580073 30297 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:56.588207 30234 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.640846 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.571075 29962 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:11:56.641834 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 88974.2us
I20260709 13:11:56.641116 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:56.649700 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:56.571267 29907 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:11:56.645439 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.652915 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.660557 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.661394 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:56.610577 30232 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.650709 28084 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:56.579296 30328 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:11:56.667001 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.669119 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.669373 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.663206 28082 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:56.660398 30219 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:56.675925 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:56.676960 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.678025 28159 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:56.672542 30219 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:56.681202 30032 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:56.689731 30108 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:56.690653 30108 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.660817 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:56.693706 28758 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:56.694607 28085 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:56.696035 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:56.655434 30234 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.684512 28759 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:56.699180 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:56.685532 28153 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:56.707706 28155 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:56.620966 30120 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.702766 28154 raft_consensus.cc:2410] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:11:56.706560 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 125127us
I20260709 13:11:56.709501 28079 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:56.665959 30428 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:56.718199 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:56.730110 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 178087us
I20260709 13:11:56.723476 28198 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:11:56.737936 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.738015 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.740000 28124 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:56.684876 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.741686 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:56.741909 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.682597 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.630827 30206 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.750118 30234 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:56.751263 28230 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:11:56.752372 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.752928 28161 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:11:56.754423 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.754945 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.756220 28760 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:56.746532 30032 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:56.720551 30232 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.759222 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:56.756325 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.773604 28083 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:56.673076 29962 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:56.815299 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.816867 28161 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
W20260709 13:11:56.818346 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 72311.6us
W20260709 13:11:56.821415 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 62270.2us
I20260709 13:11:56.741851 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.757125 30032 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:56.781673 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 146718us
I20260709 13:11:56.803495 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.833853 28160 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:56.835935 28764 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:11:56.747359 30329 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:56.842665 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.842370 30032 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:56.740379 30376 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:56.851418 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:56.684926 30219 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:56.850844 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.541s (SENT)
I20260709 13:11:56.822896 29808 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:56.740144 30424 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 13:11:56.831120 30287 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:56.862702 28086 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:11:56.886093 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 178863us
I20260709 13:11:56.747601 29974 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.841389 30299 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:56.803445 30135 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.055s
W20260709 13:11:56.890986 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57772.6us
I20260709 13:11:56.893456 28159 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:11:56.897908 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72136.7us
I20260709 13:11:56.823423 30443 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:56.894392 28157 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
W20260709 13:11:56.861502 30367 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.548s	user 0.014s	sys 0.000s
I20260709 13:11:56.900699 28157 raft_consensus.cc:606] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:56.860219 30108 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:56.915535 28079 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:56.916716 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.919008 30423 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:56.921326 30120 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:56.921608 30450 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:11:56.855096 30329 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.946389 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.950424 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:56.908439 30299 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.953190 28155 raft_consensus.cc:2410] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:56.958770 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:56.959334 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:56.959805 28157 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.960342 28231 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:56.968616 28156 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:11:56.908643 30376 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:56.980484 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 129691us
W20260709 13:11:56.981355 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 79412.9us
I20260709 13:11:56.976861 28153 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:56.969187 28156 pending_rounds.cc:85] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 5
I20260709 13:11:56.984938 28156 pending_rounds.cc:107] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:11:56.985399 28156 raft_consensus.cc:2889] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:56.986770 30438 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:56.987495 30438 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.990453 28233 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:56.991245 28761 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:11:56.991470 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:56.995574 29974 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:56.996950 28159 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:11:56.997411 28159 pending_rounds.cc:85] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 2
I20260709 13:11:56.997673 28159 pending_rounds.cc:107] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:11:56.997925 28159 raft_consensus.cc:2889] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:56.906550 30428 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:56.967836 30219 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:57.005986 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:57.006569 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:57.007161 28227 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:57.009104 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:57.009384 28230 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:11:57.009894 28152 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
W20260709 13:11:57.010589 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 254359us
I20260709 13:11:57.010952 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:57.011505 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:57.012246 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:57.024175 28231 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:57.013003 30287 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:11:57.013514 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.981873 30129 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:57.014321 30376 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.015640 29962 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.018414 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:57.018144 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:57.033583 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.568s (SENT)
I20260709 13:11:57.038908 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:57.039239 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:56.976109 30232 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.041589 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:57.034843 28152 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:57.025029 30234 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.028803 30449 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:57.046512 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:57.027509 30438 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.049216 28157 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:11:57.059733 28153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:56.975647 30299 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.061363 28157 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:57.070188 30299 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:57.071209 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:11:57.055297 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 154174us
I20260709 13:11:57.061931 28153 raft_consensus.cc:606] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:57.040568 30456 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:57.037791 30129 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.043821 30423 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:57.075434 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:57.054886 30135 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:57.109066 28153 raft_consensus.cc:614] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:57.072335 30438 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.071972 28756 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:57.057600 30367 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:57.060014 30234 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.064649 29808 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:57.054173 30449 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:11:57.128042 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 109959us
I20260709 13:11:57.111378 30232 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:57.116677 29931 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:11:57.140784 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 85354.5us
W20260709 13:11:57.147504 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 113954us
I20260709 13:11:57.144846 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:57.115190 30299 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
W20260709 13:11:57.151726 28728 leader_election.cc:336] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.610s (SENT)
I20260709 13:11:57.154135 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:57.132084 30367 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:57.135156 29808 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:57.158277 28226 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:11:57.125336 30443 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 13:11:57.216259 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.232326 28153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:11:57.233351 28153 raft_consensus.cc:606] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:57.233925 28153 raft_consensus.cc:614] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:57.155158 30423 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.239343 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:57.138993 30438 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.160674 30232 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:57.167764 30367 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:57.167825 30329 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.212656 28226 pending_rounds.cc:85] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 1
I20260709 13:11:57.213430 29808 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:57.257086 28160 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:57.234529 30456 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:11:57.271373 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 200225us
I20260709 13:11:57.272624 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:11:57.258111 30297 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.595s	user 0.006s	sys 0.007s
I20260709 13:11:57.269588 30424 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.282676 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:57.269181 28226 pending_rounds.cc:107] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:11:57.286954 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:57.284092 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:57.264089 29931 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:57.260799 30287 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.257313 30475 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.286490 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:57.273777 30299 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.265817 28756 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
W20260709 13:11:57.288141 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 136334us
I20260709 13:11:57.274253 30329 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:57.298928 28153 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:11:57.300935 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:11:57.303326 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 150390us
I20260709 13:11:57.286324 28226 raft_consensus.cc:2889] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:57.281469 30456 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:57.288414 30320 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:57.313647 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:57.317938 28198 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:57.277319 30423 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.319115 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.323943 28079 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:11:57.320647 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:57.321364 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:57.305701 30108 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:57.322382 28161 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:57.285240 30297 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.327723 28083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8381961442846ab97965dc11b100d75"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:11:57.328351 28083 raft_consensus.cc:606] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:57.287794 30424 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.292135 30474 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:11:57.333091 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:57.333091 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.334477 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.335465 28153 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:57.300438 30287 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.337482 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:57.338996 30287 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:57.339913 30287 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:57.340667 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:57.341205 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:57.341491 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.340775 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.342206 28085 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:57.340440 30287 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.342693 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.340458 28080 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:57.338548 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:57.345731 28153 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:57.302263 30032 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.353170 28760 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:57.306131 30299 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.355777 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:57.308261 29931 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.357385 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:57.354245 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.358490 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.327867 30108 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:57.359441 28161 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:57.355969 28078 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:57.276451 30449 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.371012 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.359972 28077 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:57.344139 30320 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:57.374279 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.350517 30287 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:57.302424 30475 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.377914 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:11:57.379400 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 63522.8us
I20260709 13:11:57.380115 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:11:57.369928 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 61f69a3a4f9a410e9fc9d6e9c38de9f1: Illegal state: Replica 0faaace2b36c4bcebf2fa6835235e92b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:11:57.394014 28161 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:57.391583 30473 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:57.363869 30108 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:57.394991 30456 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.439544 28153 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:57.359242 30329 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 13:11:57.449460 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 116446us
I20260709 13:11:57.363237 30313 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 13:11:57.385641 30482 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:57.453365 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:57.437817 28755 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:57.439649 30120 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:57.475919 28764 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:11:57.449980 30135 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:57.381126 30471 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:11:57.363509 30232 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:57.481971 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 103075us
I20260709 13:11:57.455539 30320 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:57.457211 30482 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.488116 30367 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.476588 30438 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:57.495632 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 115765us
I20260709 13:11:57.489199 30329 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:57.482348 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:57.511752 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:57.494884 30120 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.528965 30120 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:57.529029 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 148965us
I20260709 13:11:57.530509 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:57.531876 28124 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:57.533646 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:57.515897 30299 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:57.508917 30482 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:57.517066 30032 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:57.517976 30479 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:11:57.518913 30483 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:57.519747 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:57.520536 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:57.560001 28760 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:57.521226 30476 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:11:57.521844 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:57.522519 29808 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.523006 30484 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:57.499975 30475 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:57.524082 30489 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:57.537735 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 84441us
W20260709 13:11:57.557593 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.557197 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:11:57.512195 30329 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:57.564682 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:57.566448 28755 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:57.567472 30444 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:57.569312 30461 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:57.570009 30461 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:57.570952 30108 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.571017 30461 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.546017 30089 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:57.548784 28226 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:11:57.554695 30299 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:57.557166 30483 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.553895 30424 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.554872 30135 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:57.595588 30108 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:11:57.616878 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 259547us
I20260709 13:11:57.618799 28764 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:11:57.620618 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 80675.3us
I20260709 13:11:57.624890 30457 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:57.625674 30451 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:11:57.627645 30329 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.621776 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:57.629510 30442 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:11:57.629983 29808 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:57.573968 30032 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:11:57.582268 30297 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.583555 30423 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:57.632035 30234 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:57.613336 30484 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.637173 30484 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:57.637614 30490 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:11:57.638949 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:57.640066 28231 raft_consensus.cc:2410] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:11:57.575560 30489 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.643558 30489 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:57.644138 28231 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:11:57.645493 30467 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.503s	user 0.000s	sys 0.002s
I20260709 13:11:57.645141 30376 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:57.646100 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:57.646425 29974 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:11:57.647745 30466 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.395s	user 0.000s	sys 0.002s
I20260709 13:11:57.648058 29974 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:57.648540 29974 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:57.649115 29974 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:57.650537 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:11:57.647888 30287 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:57.652956 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 71960.1us
I20260709 13:11:57.652951 28124 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:57.651825 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:57.644217 30463 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.585s	user 0.002s	sys 0.001s
I20260709 13:11:57.565059 30328 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:11:57.662875 28760 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:11:57.656431 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:57.620783 30299 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:57.657054 30089 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.669919 30477 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.623065 30483 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:57.672902 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:57.666131 30423 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:11:57.678050 30320 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:57.678659 30376 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:11:57.680872 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:57.661052 30442 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:57.679026 30462 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:11:57.670497 30089 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.689591 30492 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:11:57.678568 30320 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.665277 30471 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:11:57.683902 30485 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.674085 30488 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:57.662168 30451 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:11:57.670898 30299 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:57.697897 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 74575.9us
I20260709 13:11:57.676656 30234 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:57.668562 30438 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:11:57.679997 29962 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:57.690584 30089 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.660863 30444 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 13:11:57.730545 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 147810us
W20260709 13:11:57.733736 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 211887us
I20260709 13:11:57.761524 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:57.761758 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:57.697686 30320 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:57.762260 28229 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:57.762475 30448 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:11:57.765123 30462 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:57.703440 30477 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.766862 30458 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:57.756459 30485 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.771273 30485 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:57.772317 30477 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.771479 28229 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:11:57.757083 30442 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:11:57.776556 28198 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:11:57.777562 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 106819us
I20260709 13:11:57.780545 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:57.789678 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 195794us
I20260709 13:11:57.792109 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:57.796597 28079 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:57.797150 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.798081 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.755424 29962 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:57.807047 29962 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:57.805886 30491 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:57.790293 30108 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:57.816910 30499 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:57.807552 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:57.817652 30499 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:57.817967 30460 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.687s	user 0.008s	sys 0.000s
I20260709 13:11:57.806525 30497 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:57.818616 30497 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:57.799678 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:57.808321 28085 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:57.815846 30320 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.817015 30108 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:11:57.826782 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.818084 30499 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:57.805473 30496 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:57.822248 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:57.796871 30489 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:57.827872 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 165956us
I20260709 13:11:57.833375 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:57.832505 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:57.807924 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:57.840922 30482 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.827785 30471 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:11:57.870689 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48561.5us
I20260709 13:11:57.808189 30120 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.826584 30491 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:57.828648 30496 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:57.833014 30489 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.832407 30462 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:57.774765 30488 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.844630 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:57.844039 30482 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:57.870443 30297 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:57.886215 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 370) took 1152 ms. Trace:
I20260709 13:11:57.759640 30135 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.871942 28755 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:57.880364 30287 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:57.894693 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:57.886132 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:57.898298 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:57.899863 29974 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.901572 29931 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.903138 30320 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.819657 30497 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.894130 30089 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.889694 28226 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:57.981266 28198 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:57.908185 30329 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:57.908416 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:57.909948 30470 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.853s	user 0.001s	sys 0.003s
I20260709 13:11:57.910269 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:57.915369 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:57.917210 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:57.893455 30299 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.885349 30442 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:57.968537 28758 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:57.943421 30423 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.972776 30120 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.921265 30108 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.973711 30491 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:57.975438 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 85136.4us
I20260709 13:11:57.976184 30297 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:57.977321 30488 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:57.922611 30471 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:57.944753 30473 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.978096 29974 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:58.015506 30423 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.014477 30442 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:57.936084 30496 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:58.027983 30442 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.029008 30473 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:57.941435 28159 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:58.030196 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 356) took 1098 ms. Trace:
I20260709 13:11:58.030555 28159 rpcz_store.cc:276] 0709 13:11:56.931339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.999435 (+ 68096us) service_pool.cc:224] Handling call
0709 13:11:57.898147 (+898712us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:11:58.030109 (+131962us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44233088}
I20260709 13:11:58.031916 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:58.032066 30471 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.033423 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.038368 28159 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:57.981213 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 110458us
I20260709 13:11:57.938633 30489 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:57.938978 28235 rpcz_store.cc:276] 0709 13:11:56.673315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.682920 (+  9605us) service_pool.cc:224] Handling call
0709 13:11:56.683326 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:56.746908 (+ 63582us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:56.746937 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:56.854435 (+107498us) log.cc:844] Serialized 26 byte log entry
0709 13:11:56.940628 (+ 86193us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:56.940671 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:56.940759 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:57.817206 (+876447us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:57.819791 (+  2585us) raft_consensus.cc:1685] finished
0709 13:11:57.819811 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:57.825907 (+  6096us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6847,"raft.queue_time_us":7112,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":44604,"spinlock_wait_cycles":339244800,"thread_start_us":79875,"threads_started":1,"wal-append.queue_time_us":198810}
I20260709 13:11:58.009577 28154 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
W20260709 13:11:58.010448 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 198689us
I20260709 13:11:58.012254 28160 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:58.013175 28156 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:58.012647 28155 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:57.938521 30456 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.045841 30479 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:58.045804 30456 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.046398 30479 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:58.047562 30479 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.049188 30488 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:58.049563 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:58.049849 30488 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.050937 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:58.051246 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:57.980600 29808 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.011947 30482 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:58.020391 28198 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:58.020982 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:57.972409 29962 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.008744 30089 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:57.966777 28757 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:57.978936 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.035356 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:57.982632 30328 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:11:58.040498 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:58.043927 30514 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:58.071969 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 5c654c99561449cb823642a1b85879ac: Illegal state: Replica 11f10c12c03e47dcac88b050d4c1431b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:11:58.044575 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:11:58.053778 28124 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Timed out: RequestConsensusVote RPC to 127.26.83.196:46855 timed out after 2.174s (SENT)
I20260709 13:11:58.072636 28157 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:58.073282 30514 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:58.073514 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:58.073807 30514 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:58.074474 30514 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:58.077049 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59724.6us
I20260709 13:11:58.079623 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.069401 30120 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.069955 30297 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.080858 30120 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.081677 30297 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.082279 30297 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.082388 30089 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:58.051586 28235 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:58.056512 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.083467 30120 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:58.083693 28156 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:58.084052 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:58.084263 30485 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:58.084784 28235 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:58.085747 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:58.086889 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:58.087610 30488 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:58.048473 30456 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:58.060680 28234 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:58.061671 30032 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:58.055636 30482 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.089476 30032 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.089717 30482 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.089969 30032 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.090377 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:58.091572 30488 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:58.061976 30479 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.085758 30485 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:58.095343 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:58.095453 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:58.096143 28161 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:58.096668 30488 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:58.097321 30488 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:58.098351 30488 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.100617 28226 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:58.070791 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.051573 30499 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.102244 28226 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:58.103518 30499 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:58.062820 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.106390 29962 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:58.107786 29974 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:58.108620 29974 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:11:58.109037 30512 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:11:58.109112 29974 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:58.107261 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 44556us
I20260709 13:11:58.066565 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:58.062517 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.112176 30423 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.112277 28229 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:58.107758 30428 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:11:58.114104 30299 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.073724 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.088958 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:58.116914 28234 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.117355 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:58.094477 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:58.112283 28155 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:58.122778 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:11:58.125952 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55270.2us
I20260709 13:11:58.139003 28235 raft_consensus.cc:2368] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5.
I20260709 13:11:58.140352 28156 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:11:58.140311 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:58.144047 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.144546 28160 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
W20260709 13:11:58.145077 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 49827.4us
I20260709 13:11:58.150749 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:58.151994 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:58.152674 28160 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:11:58.151994 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.153867 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:58.154188 28161 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.165619 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:58.168156 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:58.170449 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:58.170543 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.171649 28156 raft_consensus.cc:2410] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:11:58.172382 28234 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:58.173818 28229 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:58.150085 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:58.181389 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:58.174182 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.174708 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:58.175187 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:58.181562 28159 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:11:58.117878 30423 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.182911 28156 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:58.069366 29808 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.147204 29962 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:58.143258 30443 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:58.175195 30089 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:11:58.188144 30108 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:58.188848 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:58.197849 28159 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:58.198948 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:58.187400 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:58.147047 30492 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.203197 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.172866 30120 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:58.170034 30313 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:58.200071 28080 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:58.206929 30492 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.207928 30502 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:11:58.209826 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:58.220232 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:58.213891 28234 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:58.202705 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:58.215570 28159 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:11:58.218078 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:58.191777 30438 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:58.226263 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:58.157333 30485 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:11:58.230931 28082 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:58.226554 28159 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:11:58.211301 30483 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.004s
I20260709 13:11:58.235148 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:58.225324 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:11:58.191087 28758 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.227576 28161 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:58.214391 30299 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:58.229954 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:58.244551 28234 raft_consensus.cc:2410] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:58.241766 28761 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:11:58.185187 30320 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:11:58.243415 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:58.242482 30515 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:11:58.252442 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 8d3461aa17604e69b08928a9bc0f8d18: Illegal state: Replica 611ec64ec58244fe82058d6130e4e6f3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:11:58.191660 30423 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.242846 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:58.243503 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.255661 28759 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:58.262630 28233 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:58.263327 30494 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:58.263773 30494 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:58.244241 30483 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.243335 28756 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:58.275146 28230 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:58.264707 30494 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.276579 30494 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.252005 30456 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:58.278028 30504 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:58.278164 30497 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:58.278559 30504 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:58.265130 28233 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:58.279585 30504 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.281098 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:58.282702 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:58.283273 28082 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.283345 30515 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.285871 30515 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:58.289721 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:58.291124 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:58.292248 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:58.297869 29808 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:58.298779 28084 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:11:58.300715 30454 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.174s	user 0.000s	sys 0.008s
I20260709 13:11:58.301546 30032 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.233680 30499 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:11:58.305219 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 375) took 1274 ms. Trace:
I20260709 13:11:58.305383 30499 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.305931 30129 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:58.272277 30483 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.308789 30299 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:58.273906 30472 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:11:58.312103 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 71446.1us
W20260709 13:11:58.310837 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 67441.3us
I20260709 13:11:58.314523 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.321586 28234 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:11:58.329228 29808 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:58.329689 29808 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.330056 29808 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.331357 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:11:58.259274 30461 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.007s	sys 0.010s
W20260709 13:11:58.309974 30495 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.640s	user 0.002s	sys 0.000s
I20260709 13:11:58.322235 28234 pending_rounds.cc:85] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 5
I20260709 13:11:58.338328 28234 pending_rounds.cc:107] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:11:58.338716 28234 raft_consensus.cc:2889] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:58.340807 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:11:58.321401 30427 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.214s	user 0.017s	sys 0.030s
I20260709 13:11:58.305938 30499 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.348119 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.274008 30492 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:58.310149 30472 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:58.361753 30449 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:58.361308 30472 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.363040 30472 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:58.335052 30428 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:11:58.347030 30232 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.366036 30232 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.309938 30299 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.368180 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:58.369087 28229 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:58.371804 28758 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:58.374886 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 359) took 1442 ms. Trace:
I20260709 13:11:58.376415 28730 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:58.328563 30320 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:11:58.353826 30129 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:11:58.336443 30504 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.359704 30423 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.365067 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:58.366349 30428 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:58.399760 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:58.329063 30456 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:58.311019 30032 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:58.378539 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:11:58.337983 30461 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.007s	sys 0.011s
I20260709 13:11:58.305698 28232 rpcz_store.cc:276] 0709 13:11:57.030452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.031003 (+   551us) service_pool.cc:224] Handling call
0709 13:11:57.040943 (+  9940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:57.045915 (+  4972us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:57.914083 (+868168us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b1800450220. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:11:57.954371 (+ 40288us) log.cc:844] Serialized 14 byte log entry
0709 13:11:58.206496 (+252125us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:58.215156 (+  8660us) log.cc:844] Serialized 26 byte log entry
0709 13:11:58.215336 (+   180us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:58.215382 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.215453 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:58.304800 (+ 89347us) raft_consensus.cc:1685] finished
0709 13:11:58.304901 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:58.305040 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39323,"prepare.queue_time_us":219503,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":7048,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":6128,"spinlock_wait_cycles":13501824,"thread_start_us":218027,"threads_started":2,"wal-append.queue_time_us":278996}
I20260709 13:11:58.359755 30514 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:58.377337 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59473.8us
I20260709 13:11:58.421303 28728 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:58.423064 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:58.386010 29808 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.387648 30287 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:58.357710 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:58.388667 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 74222.7us
I20260709 13:11:58.392077 30457 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:11:58.348971 30427 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.247s	user 0.017s	sys 0.031s
I20260709 13:11:58.270933 30489 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.450701 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1768) took 1148 ms. Trace:
I20260709 13:11:58.451320 30489 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.451535 30514 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:58.452399 30367 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:58.431021 30287 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.452865 28227 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:11:58.453073 30367 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.453176 30287 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.453613 30367 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.422690 30423 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:58.454557 30423 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:58.322469 30297 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.344440 30515 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:11:58.397112 30492 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:58.431766 29808 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.437922 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.403234 30429 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.426323 30471 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:11:58.431263 30485 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:58.431296 30313 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:11:58.444897 30468 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.389s	user 0.000s	sys 0.005s
W20260709 13:11:58.446731 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.415730 28756 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.484638 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:11:58.485612 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.404153 30320 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:58.403111 30428 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:58.457744 28079 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:11:58.448513 28082 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:58.451119 28230 rpcz_store.cc:276] 0709 13:11:57.301853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.648906 (+347053us) service_pool.cc:224] Handling call
0709 13:11:57.649222 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.275885 (+626663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.450454 (+174569us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54003bfbd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:58.450640 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438955776}
I20260709 13:11:58.425580 30204 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:58.483120 29808 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:11:58.483224 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45389.8us
I20260709 13:11:58.472263 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:11:58.441700 30457 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:58.495723 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:58.499085 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.467118 30489 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.467628 30492 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.513736 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:58.516621 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.485353 30313 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.536103 30313 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:58.537118 28127 leader_election.cc:400] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5.
I20260709 13:11:58.537606 28127 leader_election.cc:403] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:11:58.456498 30482 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.527976 29808 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:58.540192 28231 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:11:58.506181 30484 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:58.525986 30492 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.514636 30204 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:58.526615 28757 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:11:58.553421 30032 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:58.526768 30500 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:58.482198 30423 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.566200 30232 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:58.584573 30494 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5.
I20260709 13:11:58.531721 30483 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:11:58.606112 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.591316 30367 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.613466 30313 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.552392 30489 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.375470 28152 rpcz_store.cc:276] 0709 13:11:56.932757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.126813 (+194056us) service_pool.cc:224] Handling call
0709 13:11:57.167509 (+ 40696us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:57.278597 (+111088us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:57.310993 (+ 32396us) log.cc:844] Serialized 14 byte log entry
0709 13:11:57.589466 (+278473us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:57.687800 (+ 98334us) log.cc:844] Serialized 26 byte log entry
0709 13:11:57.702157 (+ 14357us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:57.754895 (+ 52738us) log.cc:844] Serialized 14 byte log entry
0709 13:11:57.808081 (+ 53186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.812332 (+  4251us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:58.309268 (+496936us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:58.374601 (+ 65333us) raft_consensus.cc:1685] finished
0709 13:11:58.374636 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:58.374812 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54122,"prepare.queue_time_us":35507,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":1233,"raft.queue_time_us":67291,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":9646,"spinlock_wait_cycles":5035904,"thread_start_us":230611,"threads_started":1,"wal-append.queue_time_us":338338}
I20260709 13:11:58.652019 30473 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:58.649055 30320 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.653026 30489 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.653803 28229 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:58.603541 30484 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.651413 30513 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.651860 30452 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.127s
I20260709 13:11:58.499477 30456 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.607887 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:58.676304 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 68489.1us
W20260709 13:11:58.621611 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 56659.7us
I20260709 13:11:58.611868 30032 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:11:58.672017 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 75415.9us
W20260709 13:11:58.674533 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 163236us
I20260709 13:11:58.680598 28198 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:58.600701 30505 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:11:58.694449 30489 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:58.695245 30489 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:58.696447 30489 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:58.668592 30496 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.000s
I20260709 13:11:58.623685 30515 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:11:58.707583 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:58.704483 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1786) took 1857 ms. Trace:
I20260709 13:11:58.715444 28759 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:11:58.619100 30488 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:11:58.597079 30428 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.715831 28158 rpcz_store.cc:276] 0709 13:11:56.847425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.854841 (+  7416us) service_pool.cc:224] Handling call
0709 13:11:56.860593 (+  5752us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:56.892188 (+ 31595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:56.892224 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:56.916578 (+ 24354us) log.cc:844] Serialized 26 byte log entry
0709 13:11:57.058068 (+141490us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:11:57.058127 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.058226 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:58.397054 (+1338828us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:58.681483 (+284429us) raft_consensus.cc:1685] finished
0709 13:11:58.681508 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:58.704386 (+ 22878us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3009,"raft.queue_time_us":31549,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":12204,"spinlock_wait_cycles":527204096,"thread_start_us":49034,"threads_started":1,"wal-append.queue_time_us":106959}
I20260709 13:11:58.738896 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:11:58.741989 30533 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:11:58.747915 29974 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.001s
I20260709 13:11:58.706110 30494 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:11:58.755092 30534 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:11:58.756676 30313 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.758276 30539 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:58.673038 30484 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.616648 30457 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.712651 30204 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.749043 30297 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:58.776906 30527 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:11:58.786734 28158 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.783703 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.853214 28159 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:58.784858 28155 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:11:58.853873 28155 pending_rounds.cc:85] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 4
I20260709 13:11:58.854218 28155 pending_rounds.cc:107] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:11:58.854562 28155 raft_consensus.cc:2889] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:58.722926 30496 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.862207 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.774473 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:58.863973 28160 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.741823 30032 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.766530 30494 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.683084 30513 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.788959 30232 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:11:58.890208 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.891971 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:58.872473 28154 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
W20260709 13:11:58.878388 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.782403 28756 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:58.943346 30483 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.944025 30483 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.941555 28159 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:58.946452 30483 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.964035 28761 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:58.772650 29974 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:58.875941 30457 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:58.779206 30499 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:58.989130 30535 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260709 13:11:58.990182 28233 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:11:58.991389 30499 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.778278 30492 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.995074 30496 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:58.996697 30484 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:11:58.997664 30538 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:11:58.901204 30313 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.998447 30313 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:11:58.999240 30449 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.000s
I20260709 13:11:59.000828 30514 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.983703 30513 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:58.837532 30539 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.900609 30494 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:58.906018 30534 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:11:59.008679 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:59.012257 28158 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
W20260709 13:11:58.968426 30529 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:11:58.905665 30232 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:11:58.975107 30533 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:11:59.016255 28762 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:59.019259 28158 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:11:59.018236 28159 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:59.024978 28229 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:11:59.002496 30313 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:59.012490 30540 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:59.014804 30479 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.922s	user 0.011s	sys 0.002s
I20260709 13:11:59.035557 28160 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
W20260709 13:11:59.017031 30442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.985s	user 0.012s	sys 0.010s
I20260709 13:11:59.044051 30313 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:59.044756 30313 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:11:59.046066 30313 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.048072 30313 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:59.064596 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 384050us
W20260709 13:11:59.050688 30442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.019s	user 0.013s	sys 0.010s
I20260709 13:11:59.013024 30514 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.073302 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:59.073537 28160 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:11:59.076078 28158 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:11:59.079994 28234 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:59.077548 28158 pending_rounds.cc:85] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 2
I20260709 13:11:59.082396 28158 pending_rounds.cc:107] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:11:59.083137 28158 raft_consensus.cc:2889] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:11:59.076247 30287 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.622s	user 0.008s	sys 0.004s
I20260709 13:11:59.091337 28228 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:59.081319 30429 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.593s	user 0.006s	sys 0.005s
I20260709 13:11:59.094612 28198 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:59.054503 30540 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.098173 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:11:59.090085 30504 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.682s	user 0.012s	sys 0.000s
W20260709 13:11:59.116788 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet ceb515ffdfa04f2f977a4c5aa1aecbee: Illegal state: Replica 611ec64ec58244fe82058d6130e4e6f3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
W20260709 13:11:59.122481 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 116560us
I20260709 13:11:59.104921 30484 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:59.107977 30429 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:59.097631 30287 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.129786 30499 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:59.087875 30232 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.124292 30496 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:11:59.145117 30555 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:59.145686 30555 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.147396 30287 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:59.148243 30555 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:59.150955 30485 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:11:59.151469 30485 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:59.128504 30462 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:11:59.142441 29974 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:59.153999 30492 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:59.156771 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.159960 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1778) took 1003 ms. Trace:
I20260709 13:11:59.160360 28227 rpcz_store.cc:276] 0709 13:11:58.157079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.271644 (+114565us) service_pool.cc:224] Handling call
0709 13:11:58.272021 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:58.453315 (+181294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.159673 (+706358us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54001f4ad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:11:59.159897 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415181056}
I20260709 13:11:59.161512 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 401) took 1023 ms. Trace:
I20260709 13:11:59.162006 28079 rpcz_store.cc:276] 0709 13:11:58.138314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.146726 (+  8412us) service_pool.cc:224] Handling call
0709 13:11:59.161336 (+1014610us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400267ad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:11:59.161452 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296198400}
I20260709 13:11:59.163669 30539 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:59.166078 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.133985 30484 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:59.172142 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.172650 30484 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.179829 30129 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:59.182763 30442 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:59.182948 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:59.183719 30442 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:59.184114 30442 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:59.184659 30457 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:59.185007 30442 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.185762 30482 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:59.185546 30457 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:59.186331 30442 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:59.186919 30561 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:59.190393 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.191644 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.192520 28081 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:59.192950 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.193076 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.194257 28082 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:59.197032 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.196587 30561 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:59.199048 30561 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.200196 30561 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:59.205039 28079 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:59.207171 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.207171 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.208508 28082 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:59.212241 28082 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:59.206606 30313 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:59.223548 30556 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.225265 30556 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.227380 30556 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:59.229804 30540 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:59.231741 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.232509 28235 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:11:59.232853 30313 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:58.903975 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:59.243906 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.251684 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.254247 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:59.255565 28081 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
W20260709 13:11:59.256543 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.258455 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.259135 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:59.250000 30458 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.250031 30456 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.250032 30313 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:59.257608 30549 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.271776 28759 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:11:59.273242 28759 pending_rounds.cc:85] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 5
I20260709 13:11:59.273576 28759 pending_rounds.cc:107] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:11:59.261750 30504 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:59.273867 28759 raft_consensus.cc:2889] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:59.275540 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1808) took 1085 ms. Trace:
I20260709 13:11:59.263062 30320 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:59.283092 28161 rpcz_store.cc:276] 0709 13:11:58.189785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.897958 (+708173us) service_pool.cc:224] Handling call
0709 13:11:58.898337 (+   379us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:58.941695 (+ 43358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:58.941762 (+    67us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:59.040015 (+ 98253us) log.cc:844] Serialized 52 byte log entry
0709 13:11:59.125923 (+ 85908us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:59.126290 (+   367us) log.cc:844] Serialized 14 byte log entry
0709 13:11:59.126746 (+   456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.126826 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:59.275337 (+148511us) raft_consensus.cc:1685] finished
0709 13:11:59.275354 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:59.275479 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":200,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":31,"raft.queue_time_us":17455,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":171,"thread_start_us":43993,"threads_started":1,"wal-append.queue_time_us":136256}
I20260709 13:11:59.292793 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:59.293090 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.293635 28233 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:11:59.294055 30484 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.294968 28233 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:11:59.294801 30484 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.295768 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.296305 30482 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:59.296586 28227 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:59.296892 30482 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.297508 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.298092 30484 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:59.298877 30482 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:59.270717 30458 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.299803 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.300137 30456 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.302299 30456 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:59.301504 30458 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:59.304133 30504 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:11:59.305001 30532 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.597s	user 0.003s	sys 0.000s
W20260709 13:11:59.305789 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.305474 30504 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.307235 30320 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:11:59.303642 30508 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.268s	user 0.002s	sys 0.002s
I20260709 13:11:59.293731 28234 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:59.309675 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:59.309904 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.318533 28233 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:59.310078 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.320642 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1888) took 1257 ms. Trace:
I20260709 13:11:59.321693 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.321125 28157 rpcz_store.cc:276] 0709 13:11:58.063347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.137806 (+ 74459us) service_pool.cc:224] Handling call
0709 13:11:58.138194 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:58.174602 (+ 36408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:58.174634 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:58.185068 (+ 10434us) log.cc:844] Serialized 26 byte log entry
0709 13:11:58.379613 (+194545us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:11:58.379671 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.379775 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:59.307398 (+927623us) spinlock_profiling.cc:243] Waited 9.21 ms on lock 0x7b24001b8c18. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4109c4 00007f59cb40eacb 00007f59cb5c9dc9 00007f59cb5c9a55 00007f59cb5c8fb0 00007f59ce23f488 00007f59ce23fbe8 00007f59ce217398 00007f59ce21276a 00007f59ce211bb6
0709 13:11:59.308051 (+   653us) raft_consensus.cc:1685] finished
0709 13:11:59.308081 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:59.320584 (+ 12503us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27156,"raft.queue_time_us":38273,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":12722,"spinlock_wait_cycles":21184768,"thread_start_us":152951,"threads_started":1,"wal-append.queue_time_us":373072}
I20260709 13:11:59.322464 28233 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:59.322547 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.321182 30089 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:59.325479 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.325650 30089 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:11:59.326237 30531 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.645s	user 0.000s	sys 0.002s
I20260709 13:11:59.327787 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 449) took 1184 ms. Trace:
I20260709 13:11:59.327907 30089 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:59.325482 30320 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.328214 28755 rpcz_store.cc:276] 0709 13:11:58.144338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.146823 (+  2485us) service_pool.cc:224] Handling call
0709 13:11:58.286368 (+139545us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:11:58.286598 (+   230us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:58.287085 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:58.287121 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:58.287679 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 13:11:58.531443 (+243764us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:58.552173 (+ 20730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.577153 (+ 24980us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:59.325645 (+748492us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:59.327564 (+  1919us) raft_consensus.cc:1685] finished
0709 13:11:59.327584 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:59.327721 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5628,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":104,"spinlock_wait_cycles":17162240,"thread_start_us":209618,"threads_started":1,"wal-append.queue_time_us":312756}
I20260709 13:11:59.333197 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 663) took 1560 ms. Trace:
I20260709 13:11:59.333652 28760 rpcz_store.cc:276] 0709 13:11:57.772832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.773126 (+   294us) service_pool.cc:224] Handling call
0709 13:11:57.773526 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:57.773911 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:57.773947 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:57.774467 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:11:57.886936 (+112469us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:57.886990 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.887086 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:59.303046 (+1415960us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:11:59.332871 (+ 29825us) raft_consensus.cc:1685] finished
0709 13:11:59.332892 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:59.333131 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1138,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":12690,"spinlock_wait_cycles":10855808,"thread_start_us":81479,"threads_started":1,"wal-append.queue_time_us":212453}
I20260709 13:11:59.310811 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.337196 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.337261 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:59.338492 28233 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:59.338394 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.264780 30513 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:11:59.381706 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:11:59.348817 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 95665.4us
I20260709 13:11:59.385735 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:59.386052 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:59.382124 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 119875us
I20260709 13:11:59.387735 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:59.387672 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.392042 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.392668 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.353880 28227 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:11:59.394681 28763 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:59.358685 28081 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:59.339857 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:11:59.395942 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 56158.6us
I20260709 13:11:59.397047 28078 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:59.398379 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.401449 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:59.426662 28760 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
W20260709 13:11:59.410761 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.423481 28083 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:11:59.417676 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:59.430675 28157 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:11:59.436128 30458 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:59.436944 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.437707 28085 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:59.438570 30482 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:59.333595 30494 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.439136 30482 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:11:59.439915 28079 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:59.440398 30482 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.440943 30494 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:59.444588 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:59.387116 30489 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.010s	sys 0.015s
I20260709 13:11:59.438508 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.444576 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:59.451058 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:11:59.454003 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:59.455118 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:59.455358 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:59.455917 28161 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:11:59.392164 30488 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:59.459867 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.460122 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:11:59.465080 30561 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:59.465540 30561 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.465936 30561 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:59.467124 30524 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:59.283607 30549 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.468130 30524 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:11:59.450636 30489 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.011s	sys 0.016s
I20260709 13:11:59.470388 28233 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:59.470374 30423 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:59.471158 30494 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:59.471904 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:59.473054 28154 raft_consensus.cc:2410] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:11:59.473101 30488 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:59.474284 30488 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.399989 30127 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:59.475590 30569 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.476068 30127 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:11:59.476153 30569 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.477156 30127 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.479070 28152 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:11:59.460040 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.481889 30489 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:59.482129 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:59.487948 30561 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.488951 30423 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:59.489370 30423 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:59.489761 30423 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:59.492524 30561 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:59.499850 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:59.501669 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:59.507316 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.531970 28155 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:11:59.537118 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:59.539320 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:59.542440 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.530390 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:59.516058 30494 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:11:59.563432 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:59.565936 28155 raft_consensus.cc:2410] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:11:59.570191 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.553347 30120 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:11:59.582684 30485 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.591177 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.592053 28077 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:11:59.592952 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.594507 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.595647 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.573658 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:11:59.530023 30320 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:11:59.598948 30580 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:59.599895 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:11:59.600651 30120 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:59.600386 30580 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.601141 30120 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:11:59.601647 30489 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:59.602085 30489 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.602480 30489 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:59.603127 30580 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:59.603046 30120 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.604502 30120 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:59.604764 28232 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:11:59.600983 30032 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:59.611141 30423 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.612083 28084 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:11:59.613173 30494 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.613780 30494 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.616256 30494 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:11:59.617199 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.617717 30485 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:59.618036 30494 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.618721 30219 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:59.618654 30494 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.619390 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:59.620705 30219 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.620700 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:59.621189 30494 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:11:59.622072 30423 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:59.622822 30423 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:59.622923 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:59.623772 30580 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:59.624516 30580 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:59.625780 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.623566 30423 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:59.627203 30494 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:59.624590 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1788) took 1303 ms. Trace:
I20260709 13:11:59.629014 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:11:59.629148 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:59.630630 30580 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:59.631093 30580 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.631513 30580 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:59.628674 28226 rpcz_store.cc:276] 0709 13:11:58.321185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.514888 (+193703us) service_pool.cc:224] Handling call
0709 13:11:58.600486 (+ 85598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:58.646594 (+ 46108us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:11:58.712935 (+ 66341us) log.cc:844] Serialized 14 byte log entry
0709 13:11:59.338397 (+625462us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b18003e4700. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a 00007f59ce681a2a
0709 13:11:59.576803 (+238406us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:59.576910 (+   107us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:59.599729 (+ 22819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:59.624399 (+ 24670us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:59.624531 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35828,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":103,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":1517,"spinlock_wait_cycles":28804352,"thread_start_us":229945,"threads_started":1,"wal-append.queue_time_us":871457}
I20260709 13:11:59.637112 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:59.640501 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.644167 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:59.650527 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:11:59.663400 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:59.652454 28077 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:11:59.657992 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:11:59.666352 28227 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:59.658651 28229 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:11:59.645725 30488 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:59.674516 30514 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:59.676976 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:59.677887 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:11:59.681180 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.691913 28077 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:11:59.662737 30320 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:59.688606 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:11:59.698537 28227 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:11:59.699872 28161 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:11:59.702920 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:59.703701 28161 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:11:59.704715 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:59.692150 30474 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:59.706600 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.706820 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:59.709177 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:59.712033 28161 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:11:59.714454 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:11:59.694419 28152 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:59.694701 30219 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:11:59.722226 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:59.723803 28160 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:11:59.665402 30489 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.698470 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:11:59.697178 30313 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:11:59.787914 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:59.817626 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:11:59.722118 30456 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:11:59.706691 30472 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:11:59.783459 30474 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.862979 28158 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:11:59.865938 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:11:59.876760 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.843148 30219 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:59.849377 30320 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:59.790386 30313 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.892982 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:11:59.880888 30556 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.850651 30456 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.854967 30474 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:11:59.933586 28156 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:59.943835 30299 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:59.950297 30489 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:11:59.953265 30499 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:11:59.924104 30556 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.950165 30232 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:11:59.917750 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:11:59.902945 30219 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:11:59.906111 30320 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:00.023734 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.970911 30556 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:11:59.973079 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.015538 28077 raft_consensus.cc:2410] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:12:00.015808 30120 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.027635 28156 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:00.028223 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:00.039340 28160 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:59.966545 30499 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:59.996243 30502 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:00.029363 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:00.029490 30590 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.117s	user 0.001s	sys 0.001s
I20260709 13:12:00.044697 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:00.023080 30513 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:59.965435 30299 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.023238 30108 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.051358 28158 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:00.053711 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.042415 30502 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:00.049150 30556 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:00.085553 30474 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.087827 28152 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:00.038730 30580 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.093744 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.095485 30502 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.096590 30502 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:00.041872 30120 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:00.135145 30574 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.855s	user 0.001s	sys 0.006s
I20260709 13:12:00.138972 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.139993 28154 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:12:00.142779 28160 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:00.057056 30543 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:00.081104 30601 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:12:00.151401 28080 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:00.157634 28080 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:00.160256 28080 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.165547 28080 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:00.120559 30438 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.061013 30108 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.158326 30313 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:00.150146 30543 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:00.160691 30320 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.217188 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:00.227501 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 176897us
I20260709 13:12:00.230031 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:00.231011 28764 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:00.217792 30543 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:00.221844 30592 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.196416 30438 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:00.257475 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.237112 30603 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:00.260893 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.259824 30120 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.245970 30219 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.265229 28763 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:00.274075 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:00.275710 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:00.250447 30474 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.278837 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:00.279944 30601 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:00.261978 30089 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:00.261674 30451 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:00.281919 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:00.265810 30320 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:00.278321 30108 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.263700 30606 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.284349 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:00.283994 30605 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.281251 30287 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:00.282302 30602 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:00.282277 30536 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.618s	user 0.002s	sys 0.007s
I20260709 13:12:00.287446 30607 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:00.277956 30206 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:00.282368 30089 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:00.265234 30580 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.294055 28231 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:00.298494 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1805) took 2109 ms. Trace:
I20260709 13:12:00.250952 30592 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.299976 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:00.301273 28153 rpcz_store.cc:276] 0709 13:11:58.188915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.296461 (+107546us) service_pool.cc:224] Handling call
0709 13:11:58.296824 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:58.329339 (+ 32515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:58.329370 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:58.343216 (+ 13846us) log.cc:844] Serialized 26 byte log entry
0709 13:11:58.464911 (+121695us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:58.464967 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:58.465059 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:00.235073 (+1770014us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:00.298103 (+ 63030us) raft_consensus.cc:1685] finished
0709 13:12:00.298128 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:00.298419 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23673,"raft.queue_time_us":39262,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":8833,"spinlock_wait_cycles":31752960,"thread_start_us":96709,"threads_started":1,"wal-append.queue_time_us":255291}
I20260709 13:12:00.305750 30438 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.309067 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:00.283749 30451 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:00.263342 30603 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.283834 30601 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:00.309886 29808 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:00.311440 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.311995 30108 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:00.288326 30299 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:12:00.313381 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.321976 28234 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:00.292071 30313 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.282254 30448 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:12:00.282776 30474 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.307380 30602 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:00.311548 30089 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.292174 30320 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.307209 30429 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:12:00.328457 30313 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.336326 30089 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:00.337565 30313 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.308619 30206 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.305671 30607 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.299846 30606 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.345594 30456 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.346011 30320 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.304064 30605 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.349093 30603 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.350091 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.397207 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:00.398373 28764 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:00.352880 28198 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:00.399605 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 46886us
I20260709 13:12:00.373457 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.386021 30499 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:00.389520 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.391801 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:00.338599 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.402782 28161 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:00.411319 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:00.417485 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:00.351912 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.419206 28235 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:12:00.421136 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.422102 28234 raft_consensus.cc:2410] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:12:00.423043 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.423830 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:00.424453 28234 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:00.425313 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:00.433681 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.450242 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.451923 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:00.456043 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 173963us
I20260709 13:12:00.330758 30471 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:12:00.452776 28157 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:00.476478 28234 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:00.423873 30603 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:12:00.478924 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 175960us
I20260709 13:12:00.479275 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.413772 30602 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:00.480505 28234 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:00.413663 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:00.482822 28234 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:00.423277 30494 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:00.469267 30424 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.483943 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:00.488722 28230 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:00.489187 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:00.489552 28227 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:12:00.431021 30320 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:00.480216 30604 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:12:00.429142 30615 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:00.493415 30615 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.493896 30615 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:00.498610 28152 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:00.478137 30489 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.477993 30609 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.413719 30456 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:12:00.508234 30603 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:00.338651 30451 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.488889 30614 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:00.486325 30610 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.492132 28758 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:00.505587 30494 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:00.506150 30580 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:00.533141 30206 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.535144 28198 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:00.542600 30615 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.545003 30451 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.546561 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 406) took 1238 ms. Trace:
I20260709 13:12:00.546677 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 1873) took 1106 ms. Trace:
I20260709 13:12:00.547435 30474 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.547122 28077 rpcz_store.cc:276] 0709 13:11:59.440400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.061002 (+620602us) service_pool.cc:224] Handling call
0709 13:12:00.061400 (+   398us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:00.061936 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.061977 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:00.239091 (+177114us) log.cc:844] Serialized 52 byte log entry
0709 13:12:00.390106 (+151015us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:12:00.390165 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.390267 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:00.546376 (+156109us) raft_consensus.cc:1685] finished
0709 13:12:00.546397 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:00.546609 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":14368,"thread_start_us":126692,"threads_started":1,"wal-append.queue_time_us":229150}
I20260709 13:12:00.548657 30494 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:00.549629 30605 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.512681 30489 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.550420 30618 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:00.550897 30618 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:00.551282 30618 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:00.502031 30424 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.551684 30489 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:00.549612 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:00.552860 30499 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.525731 30610 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.553005 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.553462 30424 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.553944 30607 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:00.554020 30424 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:00.554185 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:00.535308 30614 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.555034 30610 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.513891 30609 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.551580 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 412) took 1159 ms. Trace:
I20260709 13:12:00.557044 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:00.544448 30599 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.466s	user 0.001s	sys 0.003s
I20260709 13:12:00.546974 28081 rpcz_store.cc:276] 0709 13:11:59.308623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.309143 (+   520us) service_pool.cc:224] Handling call
0709 13:12:00.546326 (+1237183us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b540017c854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:00.546454 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88585088}
I20260709 13:12:00.563081 28084 rpcz_store.cc:276] 0709 13:11:59.391749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.392440 (+   691us) service_pool.cc:224] Handling call
0709 13:12:00.551421 (+1158981us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b540017d254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:00.551525 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314149120}
I20260709 13:12:00.567884 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 420) took 1070 ms. Trace:
I20260709 13:12:00.568259 28081 rpcz_store.cc:276] 0709 13:11:59.497807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.566487 (+1068680us) service_pool.cc:224] Handling call
0709 13:12:00.566788 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.567573 (+   785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.567598 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:00.567640 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:00.567667 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.567720 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:00.567831 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:00.569550 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 80446.2us
I20260709 13:12:00.564254 28078 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:00.570868 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1879) took 1128 ms. Trace:
I20260709 13:12:00.503721 30320 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.574499 28081 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:12:00.571187 28078 rpcz_store.cc:276] 0709 13:11:59.441995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.481188 (+ 39193us) service_pool.cc:224] Handling call
0709 13:12:00.556902 (+1075714us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:00.570806 (+ 13904us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45768576}
I20260709 13:12:00.575946 28084 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:00.576543 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 421) took 1027 ms. Trace:
I20260709 13:12:00.576823 28084 rpcz_store.cc:276] 0709 13:11:59.549329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.573467 (+1024138us) service_pool.cc:224] Handling call
0709 13:12:00.573798 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.576323 (+  2525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.576502 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:00.559320 30206 consensus_queue.cc:1175] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "611ec64ec58244fe82058d6130e4e6f3" responder_term: 4 status { last_received { term: 4 index: 4 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:00.535786 30580 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.581845 28081 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:12:00.582247 28081 pending_rounds.cc:85] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 3
I20260709 13:12:00.582567 28084 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:00.582571 28081 pending_rounds.cc:107] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:12:00.583060 28081 raft_consensus.cc:2889] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:00.557032 30474 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:00.579619 30232 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.000s
I20260709 13:12:00.586563 28078 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:00.556640 30614 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.582247 30580 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:00.549590 30494 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.576196 30605 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:12:00.609028 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:00.602922 30320 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.606746 30602 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.591056 30603 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.607759 30601 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.600042 30607 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.601106 30287 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:00.622875 30607 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.623672 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:00.624943 30614 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:00.625232 30607 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.625828 30614 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:00.626824 30592 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:00.628484 28080 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:00.628159 30614 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:00.632256 30614 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:00.633152 30614 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.630668 30601 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.634478 30614 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:00.635788 30601 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:00.636812 30606 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.638046 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.638840 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 418) took 1200 ms. Trace:
I20260709 13:12:00.607877 30618 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.612259 28234 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:12:00.639165 28083 rpcz_store.cc:276] 0709 13:11:59.438090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.460434 (+ 22344us) service_pool.cc:224] Handling call
0709 13:11:59.608322 (+147888us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552eee 0000000000375939 00007f59d35c61e0 00007f59d368f0c4 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:00.637957 (+1029635us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552f8c 00007f59cb552ef6 0000000000375939 00007f59d35c61e0 00007f59d368f0c4 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:00.638768 (+   811us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85052800}
I20260709 13:12:00.640568 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.641783 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.642596 28234 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:00.642652 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.643914 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.614763 30609 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.590579 30474 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.645748 30442 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:00.646651 30442 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:00.647008 30442 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:00.647051 30474 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.600622 30232 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.600996 30615 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.647796 30543 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.647910 30442 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.646064 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1783) took 2428 ms. Trace:
I20260709 13:12:00.649369 30232 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.638229 30328 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:00.616377 30428 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:00.645267 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1910) took 1002 ms. Trace:
W20260709 13:12:00.618101 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.649199 28232 rpcz_store.cc:276] 0709 13:11:58.217330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.432494 (+215164us) service_pool.cc:224] Handling call
0709 13:11:58.464250 (+ 31756us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:59.605302 (+1141052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.645813 (+1040511us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400288054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:00.645983 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2622150400}
I20260709 13:12:00.648274 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 387) took 1968 ms. Trace:
I20260709 13:12:00.643872 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.662580 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.662776 28231 rpcz_store.cc:276] 0709 13:11:58.680044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.684360 (+  4316us) service_pool.cc:224] Handling call
0709 13:11:58.718115 (+ 33755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.302768 (+1584653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.648046 (+345278us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000e8b54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:00.648216 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3623835904}
I20260709 13:12:00.685909 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:00.631507 30451 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:00.651980 30442 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:00.661299 28155 rpcz_store.cc:276] 0709 13:11:59.643652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.643928 (+   276us) service_pool.cc:224] Handling call
0709 13:11:59.653118 (+  9190us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:59.653599 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:59.653640 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:59.665727 (+ 12087us) log.cc:844] Serialized 52 byte log entry
0709 13:12:00.044366 (+378639us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:00.044427 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.044523 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:00.543170 (+498647us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:00.645007 (+101837us) raft_consensus.cc:1685] finished
0709 13:12:00.645027 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:00.645210 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13895,"raft.queue_time_us":2717,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":6732,"spinlock_wait_cycles":30477696,"thread_start_us":336587,"threads_started":1,"wal-append.queue_time_us":381313}
I20260709 13:12:00.693557 28231 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:00.652070 30328 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:00.649129 30602 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:00.706637 30482 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.241s	user 0.007s	sys 0.030s
I20260709 13:12:00.722443 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 419) took 1244 ms. Trace:
W20260709 13:12:00.744932 30482 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.279s	user 0.008s	sys 0.030s
I20260709 13:12:00.745193 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:00.746178 28764 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:00.745043 28077 rpcz_store.cc:276] 0709 13:11:59.477843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.555285 (+1077442us) service_pool.cc:224] Handling call
0709 13:12:00.555598 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.555931 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.555957 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:00.556449 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:12:00.680950 (+124501us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:00.680991 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.681061 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:00.722198 (+ 41137us) raft_consensus.cc:1685] finished
0709 13:12:00.722223 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:00.722376 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23203,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":99,"spinlock_wait_cycles":38656,"thread_start_us":124204,"threads_started":1,"wal-append.queue_time_us":159777}
I20260709 13:12:00.746510 30618 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.722976 30232 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.722513 30328 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.725080 27981 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 258) took 9690 ms. Trace:
I20260709 13:12:00.732218 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:00.747810 28762 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:00.747876 30232 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.748688 30232 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.749379 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:00.749900 30328 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:00.751784 30232 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.703466 30630 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.738880 28228 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:00.742142 30219 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:00.733489 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:00.755769 28232 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:00.758972 28229 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:00.771733 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:00.773319 30494 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:00.773579 30513 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:00.773890 30494 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.774008 30602 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:00.774385 30494 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:00.748093 27981 rpcz_store.cc:276] 0709 13:11:51.059045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.059413 (+   368us) service_pool.cc:224] Handling call
0709 13:12:00.713955 (+9654542us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:56.171369 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:56.172075 (+   706us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:56.172126 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:11:56.172158 (+    32us) tablet.cc:662] Decoding operations
0709 13:11:56.286648 (+114490us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:56.307433 (+ 20785us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:56.314630 (+  7197us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:11:56.320399 (+  5769us) tablet.cc:701] Row locks acquired
0709 13:11:56.320453 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:11:56.320652 (+   199us) write_op.cc:270] Start()
0709 13:11:56.320868 (+   216us) write_op.cc:276] Timestamp: P: 1783602716320585 usec, L: 0
0709 13:11:56.320911 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:11:56.321702 (+   791us) log.cc:844] Serialized 3489 byte log entry
0709 13:11:56.585883 (+264181us) op_driver.cc:464] REPLICATION: finished
0709 13:11:56.902082 (+316199us) write_op.cc:301] APPLY: starting
0709 13:11:56.902194 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:56.902439 (+   245us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:56.902582 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:56.923149 (+ 20567us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:56.923547 (+   398us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:11:56.923631 (+    84us) write_op.cc:312] APPLY: finished
0709 13:11:56.924026 (+   395us) log.cc:844] Serialized 65 byte log entry
0709 13:11:56.924302 (+   276us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:56.924464 (+   162us) write_op.cc:454] Released schema lock
0709 13:11:56.924541 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:56.954601 (+ 30060us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:56.965963 (+ 11362us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3581,"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":268618,"apply.run_cpu_time_us":4249,"apply.run_wall_time_us":34590,"num_ops":8,"prepare.queue_time_us":328978,"prepare.run_cpu_time_us":41502,"prepare.run_wall_time_us":225515,"raft.queue_time_us":149897,"raft.run_cpu_time_us":68226,"raft.run_wall_time_us":130159,"replication_time_us":264730,"thread_start_us":558359,"threads_started":4,"wal-append.queue_time_us":91630,"wal-append.run_cpu_time_us":58368,"wal-append.run_wall_time_us":1554589}]]}
I20260709 13:12:00.778110 28125 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:00.779841 28082 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:00.769673 30618 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.769366 30482 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:00.779330 30032 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:00.790519 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 268943us
I20260709 13:12:00.795413 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.780596 30616 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.792603 28153 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:00.801064 30618 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:00.803687 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.780553 30605 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:00.811425 28153 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:00.812067 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:00.824659 30428 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:00.786646 30601 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.825769 28050 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:00.820981 28153 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:00.784842 30232 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:12:00.825294 30428 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.846088 28755 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:00.850095 30428 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:00.851506 30631 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:00.849045 30449 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:00.786520 30504 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:12:00.866777 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.866999 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.868010 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:00.869059 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:00.869733 28081 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:00.871042 28160 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:00.864259 30422 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.854086 27982 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 4 to 5, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194), no longer has a pending config: VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193), VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194), VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195), NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 5 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:00.872648 30422 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.872944 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.874161 30618 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:00.873601 27982 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:00.874321 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.874698 30422 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.771600 30630 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.875619 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.875295 27982 catalog_manager.cc:5697] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:00.878156 30630 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.855381 30601 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.880791 30601 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.881465 28152 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:12:00.881893 28152 pending_rounds.cc:85] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 7
I20260709 13:12:00.882135 28152 pending_rounds.cc:107] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:12:00.882551 28152 raft_consensus.cc:2889] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:00.883899 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.885004 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.885370 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.885660 28083 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:00.885847 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:00.886548 28755 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:12:00.886631 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.887279 28152 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:12:00.870056 30543 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:00.790759 30474 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:12:00.888545 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:00.865100 30449 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.889111 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.890595 30580 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.890837 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:00.910424 28086 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:00.864578 30462 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:00.909802 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.875547 30219 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:00.860399 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.815403 30032 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.914383 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:00.894804 27982 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:00.806674 30482 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.915612 28153 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:00.919817 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.911442 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:00.819413 30616 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.911368 30449 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:00.924564 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:00.911549 30601 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.929050 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.922680 27982 catalog_manager.cc:5697] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 4, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 4 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:00.890878 30127 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:12:00.913018 30219 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:00.933967 29808 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:00.917524 30637 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:00.938217 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:00.938589 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:00.932498 27982 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 3, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:00.943408 27982 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 3, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:00.929090 30287 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:00.949048 30287 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.949882 30287 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:00.951339 30622 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:00.952836 30032 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.934645 30451 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:00.921711 30422 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:12:00.955453 27982 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:00.966737 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:00.931417 28152 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:00.968607 27982 catalog_manager.cc:5697] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:00.946174 30628 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:12:00.961516 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.973453 28086 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:00.971267 27982 catalog_manager.cc:5697] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:00.972137 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.974684 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:00.975039 28077 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
W20260709 13:12:00.977349 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.943004 30549 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:00.978400 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.979878 28078 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:00.977600 27982 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 3 to 4, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:00.954002 30614 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:00.982338 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 501335us
I20260709 13:12:00.935910 30601 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.987159 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:00.937924 30456 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:12:00.995128 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:00.968878 30451 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.983320 27982 catalog_manager.cc:5697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:00.912873 30462 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:00.997538 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1532) took 1661 ms. Trace:
W20260709 13:12:00.990199 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.973435 30032 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:00.952001 30622 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:00.983099 30543 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:00.997761 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.004901 28154 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:00.996825 30555 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:01.000373 27982 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:01.002754 28233 rpcz_store.cc:276] 0709 13:11:59.336166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.387074 (+ 50908us) service_pool.cc:224] Handling call
0709 13:11:59.387404 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:59.471095 (+ 83691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:00.997304 (+1526209us) spinlock_profiling.cc:243] Waited 82.7 ms on lock 0x7b5400287154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:00.997479 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190306560}
W20260709 13:12:01.009097 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.008538 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:01.013298 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1811) took 2696 ms. Trace:
I20260709 13:12:01.011238 27982 catalog_manager.cc:5697] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:01.013640 28159 rpcz_store.cc:276] 0709 13:11:58.316714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.945602 (+628888us) service_pool.cc:224] Handling call
0709 13:11:59.017906 (+ 72304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:59.019906 (+  2000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.013105 (+1993199us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b540010ac50. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:01.013243 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165421824}
I20260709 13:12:01.014290 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.014319 27982 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 3, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:01.011703 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:01.002050 30451 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:01.016191 28084 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:01.016350 28728 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.019531 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:01.020049 28159 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:01.007001 30032 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:01.022678 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.022548 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.025142 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:01.026726 28085 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:01.025406 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.243238 28233 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:01.010919 30543 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:01.018539 30641 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:12:01.004760 30646 raft_consensus.cc:3055] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:01.010800 30609 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:01.256024 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:00.984793 30219 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.268886 29808 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:01.330890 29808 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.011519 30555 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:01.300036 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.243551 30032 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:01.307458 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.307723 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.016359 30606 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:01.337563 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:01.253885 30543 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:01.368649 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.306885 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:01.373030 28082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1520) took 2127 ms (client timeout 2170 ms). Trace:
W20260709 13:12:01.373457 28082 rpcz_store.cc:269] 0709 13:11:59.245363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.245513 (+   150us) service_pool.cc:224] Handling call
0709 13:12:01.372813 (+2127300us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54003aa7d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:01.372938 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3497196288}
I20260709 13:12:01.373847 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.374789 28086 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 6: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:01.243604 30592 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:01.384162 30592 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:01.295444 30635 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.463s	user 0.002s	sys 0.000s
I20260709 13:12:01.016986 30618 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.299s
I20260709 13:12:01.254210 30489 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:01.256528 30617 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:01.338747 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.341513 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.344719 30615 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:01.348220 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.334615 30555 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:01.351136 30601 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:01.336226 30580 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.352576 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1525) took 1899 ms. Trace:
I20260709 13:12:01.405604 28233 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:01.357396 30607 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:12:01.358661 30642 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.361s	user 0.001s	sys 0.000s
W20260709 13:12:01.361115 30644 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:12:01.361799 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 671) took 1131 ms. Trace:
I20260709 13:12:01.365687 30606 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:01.437700 30606 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.440121 30606 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.370066 30622 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:01.299328 30569 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:01.250584 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 231099us
I20260709 13:12:01.446920 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:01.378360 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.381721 28085 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:01.321132 30482 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:01.383000 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.383342 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:01.387398 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.377885 28125 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 2.170s (SENT) [suppressed 35 similar messages]
I20260709 13:12:01.335803 30485 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:01.397523 28761 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:01.453609 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.399220 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.455744 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.403194 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.341111 30646 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:01.409685 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.410228 29808 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:01.411602 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.408279 28080 rpcz_store.cc:276] 0709 13:11:59.453265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.264464 (+811199us) service_pool.cc:224] Handling call
0709 13:12:00.264847 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.628883 (+364036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.352337 (+723454us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54001ad454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:01.352515 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835477376}
I20260709 13:12:01.415282 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:01.422819 30328 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.672s	user 0.005s	sys 0.005s
I20260709 13:12:01.424063 30607 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.425005 30462 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.389520 30488 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.426489 30449 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.427428 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:01.496237 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:01.428170 30592 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.389303 30630 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:01.434942 28763 rpcz_store.cc:276] 0709 13:12:00.229936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.230053 (+   117us) service_pool.cc:224] Handling call
0709 13:12:00.230420 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.267078 (+ 36658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.361199 (+1094121us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b540043ab54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:01.361700 (+   501us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79313280}
I20260709 13:12:01.407307 30615 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:12:01.449497 28085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1526) took 1995 ms (client timeout 1830 ms). Trace:
I20260709 13:12:01.440224 30616 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.459743 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.520776 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.434371 30451 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.461668 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.480860 30580 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.420436 30032 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.526074 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:01.488481 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.491796 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.528817 28235 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:01.530084 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:01.531399 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.532089 28235 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:12:01.495234 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:01.499513 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:01.409621 30489 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:01.500667 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.503496 30449 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.504047 28755 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:12:01.509876 29907 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:01.510254 30607 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:01.510427 30458 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:01.511336 30646 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.499207 30206 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:12:01.512396 30488 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:01.516206 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.518733 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:01.516424 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 493909us
I20260709 13:12:01.463017 30513 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.483220 30456 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:01.522657 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 61101.6us
W20260709 13:12:01.519248 28085 rpcz_store.cc:269] 0709 13:11:59.453706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:59.464887 (+ 11181us) service_pool.cc:224] Handling call
0709 13:12:01.024987 (+1560100us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:01.449418 (+424431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96762624}
I20260709 13:12:01.533703 28154 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:01.533092 28158 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
W20260709 13:12:01.519531 30428 consensus_meta.cc:287] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.668s	user 0.010s	sys 0.000s
I20260709 13:12:01.535632 28233 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:01.542073 28080 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:01.569644 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:01.543852 28086 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:01.546789 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.548259 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.548710 28082 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:01.527201 30482 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:01.570497 28760 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:01.577586 28235 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:12:01.568710 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:01.523506 30448 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:12:01.574903 28764 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:01.573163 28160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:12:01.582986 28160 raft_consensus.cc:606] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:01.583439 28160 raft_consensus.cc:614] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:01.576928 29808 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
W20260709 13:12:01.534821 30287 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.001s
I20260709 13:12:01.525792 30630 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.592648 28233 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:01.597208 28233 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:01.573396 30428 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.599862 30472 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:01.601223 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:01.602448 30494 consensus_meta.cc:287] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.828s	user 0.006s	sys 0.005s
I20260709 13:12:01.602933 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:01.603024 30494 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.604008 28763 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:12:01.572832 30449 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:01.606848 30630 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:01.566376 30615 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.609125 30615 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:01.612598 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:01.613790 28760 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 6: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:01.614975 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:01.568617 28758 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:01.586767 30607 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:01.622467 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 7e272db0632a4262a90eaacd6e6d4378: Illegal state: Replica 0faaace2b36c4bcebf2fa6835235e92b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:01.605302 30449 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.586262 30108 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:12:01.616466 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:01.564361 30616 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:01.536984 30580 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.634680 28760 raft_consensus.cc:2410] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 13:12:01.638428 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 260541us
I20260709 13:12:01.567065 30456 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:01.650101 29808 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:01.644979 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.655025 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.661405 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:01.629805 28079 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:01.632205 28756 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:12:01.671432 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:01.657824 30429 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.025s	sys 0.077s
I20260709 13:12:01.678630 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 1804) took 1055 ms. Trace:
I20260709 13:12:01.681399 28231 rpcz_store.cc:276] 0709 13:12:00.613892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.685556 (+ 71664us) service_pool.cc:224] Handling call
0709 13:12:01.663923 (+978367us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b54000de3d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:01.669538 (+  5615us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14693120}
I20260709 13:12:01.566542 30513 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.658648 30456 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:01.639575 30616 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:01.674512 30428 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:01.658104 30618 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:01.652122 30630 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:01.695540 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:01.635291 30472 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.663681 30543 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.683311 28755 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:01.710124 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:01.681149 30429 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.025s	sys 0.078s
I20260709 13:12:01.708717 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:01.658886 30482 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:12:01.720391 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:01.635680 30287 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.726132 28759 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:01.730608 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 75678.5us
I20260709 13:12:01.666108 30489 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.744603 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:01.751528 29808 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:01.751809 28233 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:01.582427 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.706598 30630 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.727114 30449 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.735883 30108 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:12:01.768806 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 186473us
W20260709 13:12:01.762001 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 255638us
I20260709 13:12:01.787294 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.740094 30616 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.791600 30616 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:01.789393 30438 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.792900 30494 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:01.789556 30652 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:12:01.793576 30494 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:01.794121 30494 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.791517 30657 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:01.795163 30657 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.793171 30438 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.797122 30630 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.797644 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1806) took 1183 ms. Trace:
I20260709 13:12:01.797880 30657 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.761963 30601 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:12:01.799064 30438 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.799520 30452 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:01.800005 30452 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:01.800856 30555 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.803275 30555 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:01.806180 28757 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:01.808197 30287 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:01.787810 30428 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:01.743362 30649 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:12:01.809576 30428 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.811200 28161 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:12:01.810077 30494 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.812475 30565 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:01.813001 30565 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:01.814013 30565 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.811936 30428 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:01.815857 30565 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:01.749611 30618 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:01.823668 30607 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:01.824638 30649 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:01.825209 30649 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:01.820520 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.826313 30649 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.827320 28080 pending_rounds.cc:85] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 2
I20260709 13:12:01.760804 30651 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:01.827666 28080 pending_rounds.cc:107] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:12:01.827984 28080 raft_consensus.cc:2889] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:01.821884 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.829185 28086 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:01.830139 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1808) took 1215 ms. Trace:
I20260709 13:12:01.830623 28229 rpcz_store.cc:276] 0709 13:12:00.614875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.778348 (+163473us) service_pool.cc:224] Handling call
0709 13:12:00.778585 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.921410 (+142825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.921450 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:00.921509 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:00.921543 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.829929 (+908386us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400286754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:01.829972 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.830091 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327731072}
I20260709 13:12:01.778184 28764 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:01.798115 28228 rpcz_store.cc:276] 0709 13:12:00.614391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.770383 (+155992us) service_pool.cc:224] Handling call
0709 13:12:00.772885 (+  2502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.926238 (+153353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.926285 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:00.926329 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:00.926366 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.797394 (+871028us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54002dafd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:01.797442 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.797582 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352218368}
I20260709 13:12:01.811223 30543 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:01.831964 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.823156 30576 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:01.842711 30576 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:01.832574 28229 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:01.845666 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.846118 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.763573 30602 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:12:01.798015 30488 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:01.832787 28198 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:01.762370 30451 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:01.758239 30423 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:12:01.834959 30607 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:01.835675 30651 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:01.837206 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1796) took 1230 ms. Trace:
I20260709 13:12:01.835721 30485 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:01.879489 28230 rpcz_store.cc:276] 0709 13:12:00.607246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.608254 (+  1008us) service_pool.cc:224] Handling call
0709 13:12:00.608586 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:00.637031 (+ 28445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:00.637068 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:00.637112 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:00.637144 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.836995 (+1199851us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b5400391cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:01.837038 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:01.837152 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64935296}
I20260709 13:12:01.881444 28228 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:01.838145 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:01.840968 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:01.841455 28155 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:01.847913 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.855047 28052 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.857311 28078 pending_rounds.cc:85] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 4
W20260709 13:12:01.883694 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 51050.1us
I20260709 13:12:01.884260 28078 pending_rounds.cc:107] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:12:01.884757 28078 raft_consensus.cc:2889] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:01.887547 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1832) took 1054 ms. Trace:
I20260709 13:12:01.885330 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:01.888170 28155 rpcz_store.cc:276] 0709 13:12:00.833028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.833251 (+   223us) service_pool.cc:224] Handling call
0709 13:12:01.841044 (+1007793us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:12:01.841266 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.887127 (+ 45861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.887468 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6221184}
W20260709 13:12:01.894984 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56926us
I20260709 13:12:01.897703 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:01.897956 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:01.898634 28229 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:01.901135 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.902676 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 61809.1us
W20260709 13:12:01.903049 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48141.8us
I20260709 13:12:01.903905 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:01.906605 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.921676 29808 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:01.924785 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:01.925525 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.913447 30606 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:01.931617 30482 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:01.880270 30423 consensus_queue.cc:1243] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:01.939000 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.886816 30287 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:12:01.944597 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.876052 30616 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.942551 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:01.913228 30428 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:01.965493 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:01.950106 28229 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:01.893648 30618 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:01.950242 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.976107 28230 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:12:01.900013 30108 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:01.951781 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:01.921514 30429 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:01.968753 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:01.969619 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:01.989859 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.971870 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:01.924407 30659 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:01.948673 30576 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:01.959429 30206 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:01.990255 28230 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:12:01.950039 30474 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:01.985502 30429 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.857901 30452 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.992359 28763 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:02.000965 28230 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:01.988893 30618 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:02.005486 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:02.016489 28082 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:01.970760 30462 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.005801 30474 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.984712 30580 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:02.010226 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:02.020799 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.956286 30456 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.961874 30616 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:01.983991 30592 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:01.995781 30428 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:02.024297 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:01.994210 30659 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.983206 30127 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:12:02.012526 30429 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:02.032400 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:02.022833 30462 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.041054 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:02.042443 30449 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:02.043780 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1554) took 1244 ms. Trace:
I20260709 13:12:02.062493 28085 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:01.948016 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:02.051776 28082 raft_consensus.cc:2368] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5.
I20260709 13:12:02.061192 28231 rpcz_store.cc:276] 0709 13:12:00.799248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.696490 (+897242us) service_pool.cc:224] Handling call
0709 13:12:01.697419 (+   929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:01.698093 (+   674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:01.698138 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:01.698756 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 13:12:01.924515 (+225759us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:01.924575 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:01.924676 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.043567 (+118891us) raft_consensus.cc:1685] finished
0709 13:12:02.043587 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.043724 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1387,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":8083,"thread_start_us":192816,"threads_started":1,"wal-append.queue_time_us":296421}
I20260709 13:12:01.983798 30623 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:12:02.070254 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:02.070608 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 146508us
I20260709 13:12:02.071446 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:02.071732 30623 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:02.072166 30623 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:02.074569 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:02.074316 30623 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.065711 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:02.075560 28155 raft_consensus.cc:2410] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:12:02.076436 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:02.066257 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:01.993585 30607 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:02.081064 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 133134us
I20260709 13:12:02.081478 28155 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:12:02.051226 30127 consensus_queue.cc:1243] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:12:02.090919 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:02.051587 30659 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:02.092873 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:02.093683 28155 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:02.095559 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:02.096066 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:02.096233 28153 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:02.068552 30663 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:02.099583 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:02.101754 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:02.101985 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:02.104383 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.105919 28079 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:02.109882 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:02.110730 28082 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:02.087396 30543 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:02.053866 30592 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.089000 30482 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:02.084839 30666 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:02.125036 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1555) took 1153 ms. Trace:
I20260709 13:12:02.094810 30428 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:02.141060 28227 rpcz_store.cc:276] 0709 13:12:00.971553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.784585 (+813032us) service_pool.cc:224] Handling call
0709 13:12:01.785052 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.785342 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:02.014397 (+229055us) log.cc:844] Serialized 14 byte log entry
0709 13:12:02.124594 (+110197us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:02.124657 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:02.124743 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.124834 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.124970 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":146806,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":363,"thread_start_us":307164,"threads_started":2}
I20260709 13:12:02.133630 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:02.175837 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:02.176715 28082 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:02.093508 30580 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:12:02.124687 30299 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:02.181514 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:02.183670 30669 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:12:02.189750 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:02.190168 30485 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.191097 30474 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:02.122298 30676 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:02.141029 30666 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.201534 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:02.202266 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:02.202555 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:02.201538 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:02.204514 28227 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:02.207924 30299 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.228406 28235 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:02.099072 30618 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.247359 30499 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:02.250499 30499 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:02.234441 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:02.205914 30676 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.123692 30663 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.195302 30675 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:02.205242 30485 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:02.206243 30423 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:02.175782 30428 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.259393 28082 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:02.260025 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:02.261626 30462 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:02.261500 30499 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:02.263093 30670 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:12:02.229393 30474 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:02.264523 30676 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:02.227175 30666 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:02.269299 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1834) took 1434 ms. Trace:
I20260709 13:12:02.268168 28152 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:02.232872 30580 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:02.285003 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.292850 28157 rpcz_store.cc:276] 0709 13:12:00.834768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.834902 (+   134us) service_pool.cc:224] Handling call
0709 13:12:01.773585 (+938683us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:12:01.773863 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:01.944190 (+170327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:01.944218 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:01.950936 (+  6718us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.097655 (+146719us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:02.145802 (+ 48147us) log.cc:844] Serialized 14 byte log entry
0709 13:12:02.146184 (+   382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.146257 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:02.254239 (+107982us) raft_consensus.cc:1685] finished
0709 13:12:02.269022 (+ 14783us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:02.269145 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":91138,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":20928,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":32121,"spinlock_wait_cycles":5805952,"thread_start_us":308356,"threads_started":3,"wal-append.queue_time_us":277855}
I20260709 13:12:02.303716 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:02.272223 30423 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.297353 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:02.297093 30474 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.251255 30299 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.308517 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:02.237426 30489 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.307971 28156 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:02.306140 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:02.313372 28157 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 6: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:02.318303 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.319330 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:02.320075 28157 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:02.273137 30675 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.316562 30494 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:02.323666 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:02.323989 30663 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.326391 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:02.327188 30650 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:02.345502 30650 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.346316 30299 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:02.348034 30650 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:02.348711 30663 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:02.349283 30663 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.352999 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:02.354004 28758 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 6: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:02.329080 30423 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.310842 30668 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:12:02.331604 30474 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.333585 30592 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:02.334802 30428 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.334959 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:02.357229 30663 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.357995 30592 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:02.358783 30592 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.359784 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:02.337213 30615 consensus_meta.cc:287] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.004s
I20260709 13:12:02.357244 28763 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:02.361549 28155 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:02.361887 30485 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.357867 30650 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:02.363116 30650 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.363646 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:02.318529 30489 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.365126 30675 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.365258 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:02.366032 28155 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:02.366520 30592 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.366912 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:02.367838 28155 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:12:02.368907 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:02.369437 28234 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:12:02.371080 30489 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:02.371559 30650 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:02.374475 30666 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.372787 30474 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:02.373294 30663 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:02.374261 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:02.372179 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:02.378746 30676 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.376178 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1799) took 1766 ms. Trace:
I20260709 13:12:02.394382 28232 rpcz_store.cc:276] 0709 13:12:00.609875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.690161 (+ 80286us) service_pool.cc:224] Handling call
0709 13:12:00.701332 (+ 11171us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:00.770720 (+ 69388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.375902 (+1605182us) spinlock_profiling.cc:243] Waited 54 ms on lock 0x7b54003bfbd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:02.376119 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124240128}
W20260709 13:12:02.376418 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.311285 30580 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.317444 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:02.342897 30429 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.385003 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:02.402444 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 85353.8us
I20260709 13:12:02.404908 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:02.364512 30543 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:02.402257 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:02.389436 30485 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:02.415746 29907 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:02.416430 29907 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:02.416617 30676 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:02.418932 28161 raft_consensus.cc:2368] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5.
I20260709 13:12:02.421109 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1891) took 1554 ms. Trace:
I20260709 13:12:02.421705 28084 rpcz_store.cc:276] 0709 13:12:00.866687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:00.866793 (+   106us) service_pool.cc:224] Handling call
0709 13:12:02.420701 (+1553908us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400267fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:02.421021 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341850880}
I20260709 13:12:02.401486 30663 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:02.402369 30489 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:02.448218 28156 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:02.409097 30662 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:02.420403 30615 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:02.449664 28157 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:02.417683 29907 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.448197 30663 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:02.450126 30474 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:02.456205 29907 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:02.481650 30474 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:02.488598 30649 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:02.454027 30489 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:02.532948 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 128194us
I20260709 13:12:02.533892 28125 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:02.518262 30668 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:02.514309 30474 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:02.536569 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:02.534765 30682 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:12:02.535140 30494 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:12:02.582026 30691 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:12:02.589884 30659 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.592667 30659 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:02.565551 30615 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:02.578614 30580 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:02.599244 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 197023us
I20260709 13:12:02.584658 30494 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:02.617274 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:02.618345 28155 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:02.618880 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:02.620299 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 84310us
I20260709 13:12:02.595497 30429 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:02.633414 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:02.642212 30615 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:02.643834 30489 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.632823 30699 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:12:02.621925 30580 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:12:02.645099 30575 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:02.641625 30663 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.645704 30575 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:02.647715 30663 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:02.648001 30575 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.655858 30614 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:02.652644 30474 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.656615 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:02.661064 30614 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:02.661566 28155 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:02.661727 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:02.659293 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:02.664196 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:02.629097 30494 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:02.666199 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:02.666659 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:02.668138 28081 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:12:02.668612 28084 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:02.667114 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:02.670606 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:02.671998 28084 raft_consensus.cc:2410] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:02.676612 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:02.677968 28084 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:02.665776 30618 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:02.665634 30489 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:02.692974 30494 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.686098 30618 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:02.662050 30614 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.706272 30474 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:02.708904 30456 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:12:02.711731 30120 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:02.712827 29808 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:02.743611 29808 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:02.714371 30666 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.715366 28159 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:02.685437 30580 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
W20260709 13:12:02.717128 30693 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
I20260709 13:12:02.747769 30442 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.691095 30489 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:02.746628 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 112783us
I20260709 13:12:02.776253 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:02.777954 28728 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:02.745724 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:02.744948 29808 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.782193 29808 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:02.740970 30618 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:02.789742 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 156582us
I20260709 13:12:02.791572 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:02.696337 30699 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:02.787264 30543 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:02.756465 30120 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:02.699653 30494 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.797665 28728 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:02.789240 30108 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:02.794046 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:02.795845 30614 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:02.802795 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:02.793782 28084 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:02.806455 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:02.798867 28125 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:02.789055 30626 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:02.808460 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:02.797008 30666 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:02.813380 28050 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:02.785669 30442 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.815409 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:02.815583 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:02.789146 30707 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:12:02.801921 30485 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:02.818739 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:02.804924 30108 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:02.820649 28050 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:02.862404 30489 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.804636 30474 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:12:03.038331 30715 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:02.815907 30615 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:02.997903 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:03.040302 30609 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:03.039959 30615 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:02.872054 30120 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.040956 28755 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:03.041478 30120 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:03.043231 30474 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:03.043773 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.039989 30714 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:02.871330 30108 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:03.044804 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:03.043808 30474 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.045615 28763 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:03.046244 30513 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:03.002296 30649 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:03.046960 30474 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.808812 30626 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:03.039845 30666 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.047606 30626 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:03.047994 30666 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.048451 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:03.049036 28159 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:02.818863 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:03.050271 30666 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:02.819094 30606 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:03.051146 30606 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:03.051293 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:03.039842 30659 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:03.052232 30659 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:03.053097 30659 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.007036 30543 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:03.055290 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1812) took 2439 ms. Trace:
I20260709 13:12:03.055035 30543 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.055653 28235 rpcz_store.cc:276] 0709 13:12:00.615911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.587584 (+971673us) service_pool.cc:224] Handling call
0709 13:12:01.587845 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.237167 (+649322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.055030 (+817863us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b5400095bd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:03.055235 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457972480}
I20260709 13:12:03.057404 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:03.058039 28154 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:12:03.058082 30219 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:03.007578 30618 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:02.928941 30712 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 13:12:02.843458 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:03.059926 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.060385 28157 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:03.013339 30032 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:02.893852 30616 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:03.014114 30660 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.023s	user 0.004s	sys 0.000s
I20260709 13:12:03.015911 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 424) took 1107 ms. Trace:
I20260709 13:12:03.014151 30494 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:03.004521 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:03.076583 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:03.077771 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.079604 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:03.028949 30458 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.028916 30614 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.029798 30631 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.051576 30606 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:03.030421 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.032001 30442 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.091465 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:02.821224 29907 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:02.821538 30485 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.818701 30429 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:03.034268 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 216298us
W20260709 13:12:03.034758 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 214170us
I20260709 13:12:03.036351 28761 pending_rounds.cc:85] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 4
I20260709 13:12:03.036423 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.125934 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:03.127417 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:03.128084 28760 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:12:03.036438 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:03.129017 28234 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:02.800257 30607 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:12:03.058962 28730 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:03.059758 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:03.130668 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 71781.3us
W20260709 13:12:03.040023 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 214119us
I20260709 13:12:02.819634 30717 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:03.063005 28233 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:03.063655 28159 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:03.076114 28085 rpcz_store.cc:276] 0709 13:12:01.908639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.040633 (+131994us) service_pool.cc:224] Handling call
0709 13:12:03.015571 (+974938us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b54000c99d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:03.015734 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13504384}
W20260709 13:12:03.023561 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.094419 29907 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:03.086113 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.061172 30618 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:03.038367 30580 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.100713 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:03.090495 30328 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:03.102762 28762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 676) took 1649 ms. Trace:
I20260709 13:12:03.119995 30429 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:03.068202 30543 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.065728 30714 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.124570 28761 pending_rounds.cc:107] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:12:03.137131 28728 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:03.120510 30485 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:03.155359 28234 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:03.160491 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.137233 30424 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.165640 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.093937 30614 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.195065 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.194777 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.195888 28156 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:03.161062 29907 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:03.091720 30458 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.179050 28761 raft_consensus.cc:2889] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:03.093470 30631 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.166886 28762 rpcz_store.cc:276] 0709 13:12:01.453589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.453733 (+   144us) service_pool.cc:224] Handling call
0709 13:12:01.454126 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:01.513330 (+ 59204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:01.513365 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:01.514012 (+   647us) log.cc:844] Serialized 52 byte log entry
0709 13:12:01.763849 (+249837us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:01.763900 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.311703 (+547803us) spinlock_profiling.cc:243] Waited 42.4 ms on lock 0x7b5400563954. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:02.311767 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:03.013382 (+701615us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:03.102467 (+ 89085us) raft_consensus.cc:1685] finished
0709 13:12:03.102497 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.102689 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2644,"raft.queue_time_us":16630,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":136033024,"thread_start_us":249230,"threads_started":1,"wal-append.queue_time_us":427818}
I20260709 13:12:03.141441 30609 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:03.165490 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:03.187530 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50487us
I20260709 13:12:03.186898 28159 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:03.197314 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:03.082628 28154 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:03.180313 30429 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:03.211853 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.161078 30442 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:12:03.208320 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.208534 28728 leader_election.cc:400] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5.
I20260709 13:12:03.209124 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:03.199779 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 68529.3us
I20260709 13:12:03.212033 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.232893 28160 raft_consensus.cc:2368] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6.
I20260709 13:12:03.232275 28728 leader_election.cc:403] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:03.234092 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:03.234125 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:03.179164 30580 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.237155 28760 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:03.212847 30609 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:03.249790 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:03.254799 28234 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:12:03.255667 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:03.256376 28762 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:03.257470 28049 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:03.257608 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:03.256431 28234 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:03.194736 30424 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:03.262910 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:03.263546 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 118298us
I20260709 13:12:03.213903 30699 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:03.264391 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 122025us
I20260709 13:12:03.243969 30299 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.264606 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:03.246703 30659 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:03.269402 28234 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:03.258616 30543 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:03.254186 30618 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.258932 30609 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:03.270193 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:03.245882 30108 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.285435 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:03.285615 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:03.288890 28228 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:03.281946 30607 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5.
W20260709 13:12:03.296598 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 62246.3us
I20260709 13:12:03.274791 30659 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:03.272997 30648 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.266535 30616 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:12:03.276664 30543 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:03.278993 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.287496 30729 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.273895 30299 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:03.341341 30724 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:12:03.318138 30543 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:03.370263 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.285173 30618 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.342082 30424 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:03.300139 28226 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:03.304164 30659 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:03.380977 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 111230us
I20260709 13:12:03.337178 30607 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:03.388712 28234 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
W20260709 13:12:03.373555 30719 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
W20260709 13:12:03.391366 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 157334us
I20260709 13:12:03.395166 28233 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:03.318275 30648 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.303738 30723 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:03.342799 28756 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:03.398803 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:03.405742 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 134475us
I20260709 13:12:03.407469 28232 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: [5.3-6.4]
I20260709 13:12:03.409099 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:03.394414 30458 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.348896 30729 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.396158 30714 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:03.425992 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 116880us
I20260709 13:12:03.402704 30727 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.405576 30732 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:03.429548 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 158540us
I20260709 13:12:03.413611 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:03.407804 30614 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:03.439256 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:03.446156 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.418226 30733 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.451712 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:03.427867 30458 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:03.445767 30616 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:03.450798 30736 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:12:03.449247 30580 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.445909 30732 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.445788 30108 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:03.452153 30631 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.475418 30733 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.479568 30485 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.482573 30452 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:03.436803 30727 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.494877 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1861) took 1172 ms. Trace:
I20260709 13:12:03.495404 28155 rpcz_store.cc:276] 0709 13:12:02.322199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.642633 (+320434us) service_pool.cc:224] Handling call
0709 13:12:02.647155 (+  4522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.662003 (+ 14848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.494590 (+832587us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b5400237154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:03.494807 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32711552}
I20260709 13:12:03.452368 30623 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:03.498462 28758 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:03.502351 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1933) took 1272 ms. Trace:
I20260709 13:12:03.477648 30733 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:03.511577 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 698) took 2109 ms. Trace:
I20260709 13:12:03.512061 28757 rpcz_store.cc:276] 0709 13:12:01.401961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.404277 (+  2316us) service_pool.cc:224] Handling call
0709 13:12:01.427127 (+ 22850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:01.806760 (+379633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.511308 (+1704548us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b5400436554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:03.511492 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871145344}
W20260709 13:12:03.452492 30721 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.184s	user 0.003s	sys 0.000s
I20260709 13:12:03.513870 30740 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:03.514564 30740 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.465655 30618 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:03.465863 30725 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:12:03.518357 30646 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:03.521637 30606 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.523905 30606 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.524685 30668 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:03.525185 30668 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:03.502827 28158 rpcz_store.cc:276] 0709 13:12:02.229768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.229912 (+   144us) service_pool.cc:224] Handling call
0709 13:12:02.257475 (+ 27563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.285140 (+ 27665us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:02.590149 (+305009us) log.cc:844] Serialized 14 byte log entry
0709 13:12:02.692590 (+102441us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:02.692663 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:02.692756 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.502038 (+809282us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b540033e354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:03.502101 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.502275 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32692,"prepare.queue_time_us":12682,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":32881,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":113,"spinlock_wait_cycles":48445184,"thread_start_us":365027,"threads_started":2,"wal-append.queue_time_us":119649}
I20260709 13:12:03.526706 30429 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.529296 30649 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:03.529763 30649 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:03.530679 30649 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.531996 30649 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:03.465906 30626 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.534165 28159 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:03.471609 28228 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:03.471948 30299 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:03.536064 30626 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.536679 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1557) took 2550 ms. Trace:
I20260709 13:12:03.537052 28229 rpcz_store.cc:276] 0709 13:12:00.986107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.989110 (+1003003us) service_pool.cc:224] Handling call
0709 13:12:01.989509 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.182213 (+192704us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:02.192904 (+ 10691us) log.cc:844] Serialized 14 byte log entry
0709 13:12:02.208720 (+ 15816us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:02.208801 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:02.208891 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.161292 (+952401us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b180039d540. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419
0709 13:12:03.536469 (+375177us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54001a0254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:03.536510 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.536627 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13913,"prepare.queue_time_us":10662,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":106,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":490696320,"thread_start_us":14907,"threads_started":1,"wal-append.queue_time_us":69242,"wal-append.run_cpu_time_us":1340,"wal-append.run_wall_time_us":1001362}
I20260709 13:12:03.540762 30442 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.541314 30442 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.538626 29907 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.539368 30488 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:03.539990 28764 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:12:03.563037 29907 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.467834 28153 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:03.466084 28226 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:03.565439 30704 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.564401 30727 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:03.566089 30704 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.572772 30429 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:03.574267 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 87833.4us
I20260709 13:12:03.576574 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:03.577323 28084 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:03.577366 28085 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:12:03.577781 28085 pending_rounds.cc:85] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 2
I20260709 13:12:03.578944 28082 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:03.580116 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.580854 28077 raft_consensus.cc:2368] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:03.456816 30458 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.585108 30458 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.578979 30472 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.586484 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.587880 28081 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:12:03.589097 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1935) took 1336 ms. Trace:
W20260709 13:12:03.590372 30718 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.493s	user 0.000s	sys 0.003s
I20260709 13:12:03.589624 28152 rpcz_store.cc:276] 0709 13:12:02.252236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.319714 (+ 67478us) service_pool.cc:224] Handling call
0709 13:12:02.321144 (+  1430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:02.333379 (+ 12235us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:02.333862 (+   483us) log.cc:844] Serialized 14 byte log entry
0709 13:12:02.524157 (+190295us) log.cc:844] Serialized 14 byte log entry
0709 13:12:02.532893 (+  8736us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:02.534626 (+  1733us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:02.538784 (+  4158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.554890 (+1016106us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b180028a4a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419
0709 13:12:03.555549 (+   659us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b54003444d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:03.588858 (+ 33309us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.589023 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3584,"prepare.queue_time_us":5332,"prepare.run_cpu_time_us":387,"prepare.run_wall_time_us":968,"raft.queue_time_us":12084,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":19255,"spinlock_wait_cycles":54430208,"thread_start_us":156206,"threads_started":1,"wal-append.queue_time_us":209676}
W20260709 13:12:03.469360 30726 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:12:03.471274 30648 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:03.605891 30549 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:03.577988 28085 pending_rounds.cc:107] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:12:03.462596 30728 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:03.607182 28085 raft_consensus.cc:2889] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:03.608470 30728 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:03.609102 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.612516 28077 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:12:03.458609 30729 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:03.474761 30732 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.552147 30424 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:03.472467 30736 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:03.623500 30736 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:03.624944 28153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 390) took 1492 ms (client timeout 1570 ms). Trace:
I20260709 13:12:03.624715 30736 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:03.625481 28153 rpcz_store.cc:269] 0709 13:12:02.132936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.133293 (+   357us) service_pool.cc:224] Handling call
0709 13:12:03.624683 (+1491390us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540033e854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:03.624864 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3041301376}
I20260709 13:12:03.626690 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.627493 28085 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:03.614861 30728 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.629030 30728 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:03.555130 30442 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:03.558490 30485 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.558481 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 385) took 1475 ms. Trace:
I20260709 13:12:03.646426 28161 rpcz_store.cc:276] 0709 13:12:02.082640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.095822 (+ 13182us) service_pool.cc:224] Handling call
0709 13:12:03.558187 (+1462365us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b540010a754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:03.558366 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740748032}
I20260709 13:12:03.561285 30488 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:03.647140 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:03.647428 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:03.647989 30488 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.608656 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.648986 30488 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:03.649431 30704 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:03.551916 30735 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:03.618412 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.652277 28077 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
W20260709 13:12:03.607882 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 155871us
I20260709 13:12:03.653390 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:03.654413 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:03.621714 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:03.655149 28084 raft_consensus.cc:2376] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5. Re-sending same reply.
I20260709 13:12:03.621868 30424 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:03.622965 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.631438 28086 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:03.637279 30543 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:03.640388 30722 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 13:12:03.664105 27981 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:03.648985 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:03.673583 27982 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58554 (request call id 281) took 8597 ms. Trace:
I20260709 13:12:03.716814 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.717536 28154 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:12:03.656679 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.682049 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.718441 28077 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:03.718704 28161 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
W20260709 13:12:03.674487 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.702376 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.695032 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.682466 28159 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:03.719938 28157 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:03.693593 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.666383 28085 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:03.716138 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:03.721060 28153 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:03.724078 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1911) took 1060 ms. Trace:
I20260709 13:12:03.717270 27981 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 6, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 6 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:03.724480 28086 rpcz_store.cc:276] 0709 13:12:02.663781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.663925 (+   144us) service_pool.cc:224] Handling call
0709 13:12:03.723923 (+1059998us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b5400207e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:03.724026 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075009920}
I20260709 13:12:03.725787 27981 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 5, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 5 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:03.728112 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.728044 27981 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 3 to 4, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 4 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:03.728811 28086 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:03.728971 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:03.729285 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.729808 28156 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:03.729720 27981 catalog_manager.cc:5697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:03.733630 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.733898 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.734459 28158 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:03.734566 28160 raft_consensus.cc:2410] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:12:03.732530 27981 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 5, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 5 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:03.737442 27981 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:12:03.738945 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 331922us
I20260709 13:12:03.656960 30622 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:03.740813 28156 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:03.716660 27982 rpcz_store.cc:276] 0709 13:11:55.076484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.082092 (+  5608us) service_pool.cc:224] Handling call
0709 13:12:03.673486 (+8591394us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:01.734865 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:01.811681 (+ 76816us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:01.811735 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:12:01.811757 (+    22us) tablet.cc:662] Decoding operations
0709 13:12:01.813318 (+  1561us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:01.813409 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:01.813438 (+    29us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:12:01.814711 (+  1273us) tablet.cc:701] Row locks acquired
0709 13:12:01.814735 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:12:01.814907 (+   172us) write_op.cc:270] Start()
0709 13:12:01.815028 (+   121us) write_op.cc:276] Timestamp: P: 1783602721814851 usec, L: 0
0709 13:12:01.815053 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:12:01.977183 (+162130us) log.cc:844] Serialized 5940 byte log entry
0709 13:12:02.401910 (+424727us) op_driver.cc:464] REPLICATION: finished
0709 13:12:02.613033 (+211123us) write_op.cc:301] APPLY: starting
0709 13:12:02.613126 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:02.613430 (+   304us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:02.613554 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:02.637514 (+ 23960us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:02.637861 (+   347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:12:02.637924 (+    63us) write_op.cc:312] APPLY: finished
0709 13:12:02.638345 (+   421us) log.cc:844] Serialized 107 byte log entry
0709 13:12:02.638555 (+   210us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:02.638752 (+   197us) write_op.cc:454] Released schema lock
0709 13:12:02.638818 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:02.639293 (+   475us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:02.639327 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":203367,"apply.run_cpu_time_us":4731,"apply.run_wall_time_us":26220,"num_ops":15,"prepare.queue_time_us":278284,"prepare.run_cpu_time_us":14334,"prepare.run_wall_time_us":376588,"raft.queue_time_us":118307,"raft.run_cpu_time_us":37286,"raft.run_wall_time_us":274920,"replication_time_us":503220,"thread_start_us":453886,"threads_started":4,"wal-append.queue_time_us":199685,"wal-append.run_cpu_time_us":58273,"wal-append.run_wall_time_us":1453322}]]}
I20260709 13:12:03.725934 30646 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:03.749743 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.750998 28083 raft_consensus.cc:2410] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:03.742465 27981 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 5, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 5 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:03.723827 30614 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.754361 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:03.754828 27981 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:03.640429 30609 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.732357 30735 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:12:03.756556 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:03.746412 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.763623 27981 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 5, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 5 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:03.768939 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:03.773566 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:03.772598 27981 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:03.776233 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.753194 30622 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:03.774793 30646 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:03.782444 30299 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:03.626259 30736 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:03.789618 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 394713us
W20260709 13:12:03.796326 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.007s	sys 0.006s
W20260709 13:12:03.797917 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.007s	sys 0.006s
W20260709 13:12:03.798151 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 518562us
I20260709 13:12:03.800037 28077 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:12:03.805441 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:03.806680 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.829614 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:03.830440 28728 leader_election.cc:400] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5.
I20260709 13:12:03.831286 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:03.775168 30499 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:03.830502 28083 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:03.796849 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.865216 28198 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:03.864370 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:03.833289 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.763119 30614 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.867318 28760 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 6: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:03.864041 28085 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:03.868398 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:03.835422 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.869352 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:03.870419 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:03.866679 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:03.818886 30451 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.012s	sys 0.007s
I20260709 13:12:03.876650 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.878505 28155 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:12:03.872833 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.882354 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.882719 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:03.883616 28083 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:03.844359 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.884440 28077 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:03.889254 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:03.898963 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:03.796615 30299 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:03.902578 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 129129us
I20260709 13:12:03.913658 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:03.905586 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.831838 30740 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:03.911120 29808 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:12:03.876031 30451 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.012s	sys 0.008s
I20260709 13:12:03.877262 30727 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:03.928187 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.937743 28125 leader_election.cc:400] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:03.938299 28125 leader_election.cc:403] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:03.877743 30735 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:03.872234 30749 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:03.939891 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:03.959981 30749 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:03.939994 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:03.940678 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:03.960830 30704 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.961377 28229 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:03.942704 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:03.962199 28235 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:03.962855 30747 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:03.962932 28755 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:03.963486 30747 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:03.875414 30734 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:12:03.965682 30747 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:03.966477 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:03.967096 28158 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:03.939319 28125 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:03.968402 30743 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:03.970180 28083 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:03.971397 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.971689 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:03.971935 28728 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:03.972455 28229 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:03.972702 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.902863 30032 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:03.951448 30504 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:03.952173 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.953457 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:03.947372 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:03.957156 30543 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:03.909044 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.947901 30735 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:03.959333 28756 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:03.960453 30749 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:03.973526 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 68367.5us
I20260709 13:12:03.948985 30646 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.899677 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.950558 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:03.974467 28235 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:04.098639 28085 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:03.981611 29808 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:04.099493 29808 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:04.100683 29808 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.102367 29808 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:03.980198 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:03.995211 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:04.013726 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 100184us
I20260709 13:12:04.107141 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:03.924341 30733 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:04.014813 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:03.923178 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:03.925223 30299 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:03.964430 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:03.817054 30424 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:04.029814 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:03.902287 30668 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:03.968847 30743 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:03.930140 30601 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 13:12:04.013257 30458 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:04.022586 30219 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:04.067955 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:04.070060 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:04.018436 30287 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:03.941629 30748 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:12:03.973732 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 103381us
W20260709 13:12:04.171465 28082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.029s	sys 0.057s
W20260709 13:12:04.172155 28082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.029s	sys 0.057s
I20260709 13:12:04.050649 30735 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:03.980110 28125 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:04.106184 28764 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:04.022784 30576 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:12:03.937464 30622 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.094043 28758 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:12:04.129168 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 114468us
I20260709 13:12:04.225404 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:04.112448 28199 leader_election.cc:400] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6.
I20260709 13:12:04.054157 30623 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:04.152302 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.152891 28085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:12:04.234676 28085 raft_consensus.cc:606] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:04.134792 28077 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:12:04.092383 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:04.172919 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.157410 28763 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:04.171965 28050 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:04.157686 28760 raft_consensus.cc:2410] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:12:04.224143 28049 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b99bb21dbacb4d63839f1403ef9b08b9. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:04.238902 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 292071us
I20260709 13:12:04.228626 28728 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:04.239046 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:04.239542 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:04.237974 28763 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:04.243108 29808 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:04.174409 30504 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.153990 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:04.128974 30727 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:04.111979 30733 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:04.194991 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 214979us
I20260709 13:12:04.242718 30485 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:04.108543 30206 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.130s
I20260709 13:12:04.198273 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:04.112174 30746 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:04.182657 28159 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
W20260709 13:12:04.133718 30660 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:12:04.154366 30543 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:04.233561 30618 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:12:04.230708 28199 leader_election.cc:403] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:12:04.253310 30543 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.255211 30668 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:04.156432 30219 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:04.253075 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:04.256075 30219 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:04.222291 30704 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:04.157618 30458 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.255318 30543 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:04.256774 30704 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:04.257284 30704 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:04.257371 30651 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:04.221963 30609 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:04.237948 30731 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:04.256141 30668 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:04.240525 30623 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.261031 30668 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.262527 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:04.262913 30623 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:04.263192 28085 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:04.263558 30668 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:04.251204 30504 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.264225 30668 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.264905 30219 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.265925 30504 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:04.266573 30504 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:04.269101 30458 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:04.271507 30219 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:04.279763 30488 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.630s	user 0.006s	sys 0.002s
I20260709 13:12:04.174638 30748 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:04.243810 30287 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:04.281509 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 189210us
W20260709 13:12:04.282289 28227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.058s	sys 0.083s
W20260709 13:12:04.282368 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 110498us
I20260709 13:12:04.283574 28085 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:04.284304 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:04.284103 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:04.284960 28227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.058s	sys 0.083s
W20260709 13:12:04.109014 30659 consensus_meta.cc:287] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.710s	user 0.007s	sys 0.007s
I20260709 13:12:04.249317 30733 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:04.282826 28226 raft_consensus.cc:2410] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:12:04.304086 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:04.297412 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 188800us
W20260709 13:12:04.304770 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 65254.3us
I20260709 13:12:04.292480 30576 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:04.285856 30748 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:04.285898 30287 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.285584 30614 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:04.319347 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:04.324378 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:04.333307 28228 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:04.335045 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:04.298386 30206 consensus_queue.cc:1243] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:12:04.327085 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:04.290361 30727 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:04.328634 28086 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.035s	sys 0.046s
I20260709 13:12:04.328321 28232 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:04.322409 30623 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:04.329969 30287 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:04.290578 30753 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:04.320959 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:04.281538 30601 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:12:04.338896 28086 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.035s	sys 0.046s
I20260709 13:12:04.317659 30648 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:04.302248 30746 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:04.321352 30549 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:04.364339 30745 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.365115 30745 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.368013 30745 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:04.369692 30755 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 13:12:04.381103 28124 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.030s	sys 0.044s
W20260709 13:12:04.381523 28124 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.030s	sys 0.044s
W20260709 13:12:04.381870 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 660942us
I20260709 13:12:04.289300 30668 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.317966 30752 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6.
I20260709 13:12:04.336022 30736 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:04.359853 30032 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:04.391068 30032 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.392149 28124 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:04.393450 30032 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:04.349715 30737 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:04.394402 30737 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:04.394910 30737 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:04.350207 30631 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:04.397130 30631 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.397584 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:04.397951 30488 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:04.398619 28758 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:04.399775 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:04.357431 30648 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.387856 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:04.350440 30451 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.360191 30746 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:04.352339 30753 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.356320 30614 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.397610 30631 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:04.353413 30601 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:04.348048 30442 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.049s
W20260709 13:12:04.407631 28156 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.010s	sys 0.012s
W20260709 13:12:04.437160 28156 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.010s	sys 0.012s
I20260709 13:12:04.413513 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:04.439910 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:04.406092 30727 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.414907 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:04.387658 30576 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.303548 30659 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.361117 30749 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.408303 30648 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:04.414566 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:04.477126 28234 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:04.408182 30734 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:04.443230 28124 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:04.480993 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:04.450663 28079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.011s	sys 0.021s
W20260709 13:12:04.482059 28079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.012s	sys 0.021s
I20260709 13:12:04.443656 28229 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:04.441461 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:04.484815 28228 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:04.425956 30753 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:04.488137 28231 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:04.433195 30488 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:04.431499 30754 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:04.500593 28080 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:12:04.503706 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.511425 28761 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:04.463574 30219 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:04.425189 30451 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.575428 30458 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.579468 30489 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:04.583586 30488 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:04.435181 30614 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.483518 30734 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:04.440230 30704 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.461530 30735 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.753720 30032 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:04.779074 30032 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.781872 30032 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:04.482142 30576 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.485661 30543 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.754287 30745 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:04.761348 30488 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:04.809569 28083 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:04.773810 30734 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:04.606858 30601 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.769623 30489 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.609856 30746 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:04.580090 30733 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:04.861706 28125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.028s	sys 0.041s
W20260709 13:12:04.873402 28125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.028s	sys 0.041s
W20260709 13:12:04.872009 28126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.035s	sys 0.042s
I20260709 13:12:04.816463 30488 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:04.805374 30754 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:04.862385 28127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.017s	sys 0.057s
W20260709 13:12:04.879489 28127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.017s	sys 0.057s
W20260709 13:12:04.878827 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 630671us
W20260709 13:12:04.880916 28126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.035s	sys 0.042s
W20260709 13:12:04.867785 30622 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.019s	sys 0.056s
W20260709 13:12:04.883538 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:04.879158 28757 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:04.891901 28759 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:04.820035 30745 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.893522 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:04.894675 28086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.004s	sys 0.019s
I20260709 13:12:04.893484 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:04.917829 28086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.005s	sys 0.019s
I20260709 13:12:04.918619 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1572) took 1170 ms. Trace:
I20260709 13:12:04.919035 28086 rpcz_store.cc:276] 0709 13:12:03.749145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.749292 (+   147us) service_pool.cc:224] Handling call
0709 13:12:03.749616 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:03.749815 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:03.759735 (+  9920us) log.cc:844] Serialized 14 byte log entry
0709 13:12:04.918230 (+1158495us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:04.918289 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:04.918337 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:04.918418 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:04.918557 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1831,"prepare.queue_time_us":9796,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":1156194,"threads_started":2,"wal-append.queue_time_us":1084849}
W20260709 13:12:04.921458 28050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.007s	sys 0.024s
I20260709 13:12:04.879956 30746 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:04.922587 28050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.007s	sys 0.024s
W20260709 13:12:04.922972 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 602070us
W20260709 13:12:04.881299 30456 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.031s	sys 0.046s
W20260709 13:12:04.925808 28049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.006s	sys 0.015s
W20260709 13:12:04.998586 28049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.010s	sys 0.025s
I20260709 13:12:04.923763 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:04.922886 30728 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.293s	user 0.014s	sys 0.000s
W20260709 13:12:04.892791 30622 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.020s	sys 0.056s
W20260709 13:12:04.927214 30760 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.462s	user 0.001s	sys 0.001s
I20260709 13:12:04.865422 30743 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:04.920680 30569 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:04.993525 30424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.052s	sys 0.061s
W20260709 13:12:04.925210 30755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.006s	sys 0.016s
W20260709 13:12:04.998929 30757 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.533s	user 0.000s	sys 0.002s
I20260709 13:12:04.954185 30668 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:04.978788 30659 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:05.007329 30456 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.031s	sys 0.047s
I20260709 13:12:04.998347 30752 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:05.016269 30424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.052s	sys 0.061s
I20260709 13:12:04.998692 30032 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:05.022948 30755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.007s	sys 0.016s
I20260709 13:12:05.023562 30749 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:05.041764 30458 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:05.042240 30451 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:05.026993 30766 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:05.083541 30743 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:05.065059 30763 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:12:05.047402 30668 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:05.090153 30631 consensus_meta.cc:287] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.650s	user 0.011s	sys 0.000s
I20260709 13:12:05.092396 30766 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:05.048411 30752 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:05.065133 30759 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:12:05.066872 30762 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:12:05.084928 30771 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:05.051627 30704 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:05.121563 30767 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:05.104748 30631 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:05.595203 30631 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:05.070551 30737 consensus_meta.cc:287] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 13:12:05.599316 30737 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.060526 30735 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:05.294879 27981 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.430s	user 0.000s	sys 0.029s
W20260709 13:12:05.900305 27981 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.035s	user 0.001s	sys 0.030s
W20260709 13:12:05.563915 28049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.000s	sys 0.011s
W20260709 13:12:05.901726 28049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.002s	sys 0.013s
I20260709 13:12:05.533453 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 425) took 1669 ms. Trace:
I20260709 13:12:05.905344 28232 rpcz_store.cc:276] 0709 13:12:03.863770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.866463 (+  2693us) service_pool.cc:224] Handling call
0709 13:12:05.533164 (+1666701us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54003917d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:05.533334 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1050626816}
W20260709 13:12:05.630333 28235 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.246s	user 0.040s	sys 0.053s
W20260709 13:12:05.534343 28727 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.032s	sys 0.059s
W20260709 13:12:05.522562 28728 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.027s	sys 0.024s
I20260709 13:12:05.485186 28755 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:12:05.330526 30728 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:05.923075 30728 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:05.541477 30328 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:12:05.539225 30488 consensus_meta.cc:287] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.643s	user 0.012s	sys 0.003s
I20260709 13:12:05.380461 30543 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:05.506902 30576 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:05.929275 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:05.511514 30614 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:05.105944 30668 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.930701 28763 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:05.930452 30488 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:05.933645 28728 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.120s	user 0.028s	sys 0.025s
I20260709 13:12:05.105993 30733 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:05.603128 30737 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:05.092511 30745 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:05.550895 30748 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.228s	user 0.010s	sys 0.005s
I20260709 13:12:05.940042 30748 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.940953 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:05.942364 28763 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:05.942240 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:05.474282 30753 consensus_meta.cc:287] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.957s	user 0.016s	sys 0.001s
I20260709 13:12:05.947348 30753 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:05.948208 30737 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:05.949199 30614 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.949203 30737 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.950688 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1550) took 2375 ms. Trace:
W20260709 13:12:05.395316 30648 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.903s	user 0.013s	sys 0.000s
I20260709 13:12:05.951030 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:05.038661 30754 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:05.955513 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:05.956772 28079 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:05.173295 30767 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:05.957926 30753 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:05.958420 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:05.959594 30543 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.959383 30767 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:05.049890 30032 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.961262 30733 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:05.962203 30733 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.963640 30032 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:05.573726 30732 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.964970 30659 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:05.960446 30543 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.965294 30732 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.965969 30659 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.968230 30732 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:05.968415 30733 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:05.573241 30456 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.028s	sys 0.052s
I20260709 13:12:05.969296 30733 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:05.969668 30456 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.028s	sys 0.053s
I20260709 13:12:05.970178 30733 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.121953 30766 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:05.693332 28757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 714) took 1764 ms (client timeout 2081 ms). Trace:
W20260709 13:12:05.719570 28084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1920) took 1965 ms (client timeout 2075 ms). Trace:
I20260709 13:12:05.728044 30780 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:05.732059 30781 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:05.733124 30504 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.733899 30779 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:05.732498 28125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.001s	sys 0.011s
W20260709 13:12:05.734890 28126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.023s	sys 0.018s
W20260709 13:12:05.737140 28127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.005s	sys 0.008s
W20260709 13:12:05.782820 28050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.021s	sys 0.019s
W20260709 13:12:05.785594 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.431s	user 0.022s	sys 0.040s
I20260709 13:12:05.840129 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 440) took 2013 ms. Trace:
I20260709 13:12:05.870810 30731 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:05.871085 30764 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.033s	sys 0.027s
I20260709 13:12:05.878250 30601 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:05.879081 30606 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:05.879258 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:05.886487 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 93949.3us
W20260709 13:12:05.915820 28761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 485) took 2081 ms (client timeout 1530 ms). Trace:
W20260709 13:12:05.929432 28235 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.545s	user 0.042s	sys 0.054s
I20260709 13:12:05.926167 30728 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.926108 30328 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:05.936342 30488 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:05.934633 30668 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:05.938171 28727 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.170s	user 0.033s	sys 0.060s
I20260709 13:12:05.943746 30748 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:05.950296 30614 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.952382 28082 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:05.121011 30768 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:05.952172 28086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:12:05.960291 30737 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:05.951885 30648 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:05.951174 28080 rpcz_store.cc:276] 0709 13:12:03.575100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.575836 (+   736us) service_pool.cc:224] Handling call
0709 13:12:03.576134 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:04.501951 (+925817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:05.950127 (+1448176us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b5400209c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:05.950479 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2125509888}
W20260709 13:12:06.126360 28126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.024s	sys 0.030s
W20260709 13:12:06.127664 28127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.206s	user 0.014s	sys 0.010s
W20260709 13:12:06.128468 28050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.128s	user 0.021s	sys 0.030s
W20260709 13:12:05.938233 30623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.543s	user 0.009s	sys 0.012s
I20260709 13:12:06.082476 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:06.129926 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.776s	user 0.026s	sys 0.047s
W20260709 13:12:06.124147 28084 rpcz_store.cc:269] 0709 13:12:03.753976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.754105 (+   129us) service_pool.cc:224] Handling call
0709 13:12:05.718848 (+1964743us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400267fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:05.719204 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2886803840}
W20260709 13:12:06.127391 28125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.209s	user 0.010s	sys 0.016s
I20260709 13:12:06.123183 28083 rpcz_store.cc:276] 0709 13:12:03.826737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.972487 (+145750us) service_pool.cc:224] Handling call
0709 13:12:05.796701 (+1824214us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54000c85d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:05.839896 (+ 43195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922345728}
W20260709 13:12:06.026113 28226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 427) took 2161 ms (client timeout 2157 ms). Trace:
I20260709 13:12:05.973453 30733 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:06.124286 28757 rpcz_store.cc:269] 0709 13:12:03.928776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.133326 (+204550us) service_pool.cc:224] Handling call
0709 13:12:05.647501 (+1514175us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54004d1554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:05.692628 (+ 45127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1358887680}
W20260709 13:12:06.010632 30219 consensus_meta.cc:287] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.426s	user 0.014s	sys 0.011s
I20260709 13:12:06.139564 30219 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:06.130558 30623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.749s	user 0.011s	sys 0.013s
I20260709 13:12:06.140515 30623 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:06.141484 30614 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.143056 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1847) took 1820 ms. Trace:
I20260709 13:12:06.143978 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.145210 28757 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:06.146502 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.148176 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:06.123314 28761 rpcz_store.cc:269] 0709 13:12:03.834195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.835089 (+   894us) service_pool.cc:224] Handling call
0709 13:12:05.915227 (+2080138us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54006b3d54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:05.915538 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476873600}
I20260709 13:12:06.149477 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1885) took 2392 ms. Trace:
I20260709 13:12:06.149760 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.149957 28156 rpcz_store.cc:276] 0709 13:12:03.756778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.756935 (+   157us) service_pool.cc:224] Handling call
0709 13:12:03.761202 (+  4267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:03.761618 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:03.761656 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:03.770613 (+  8957us) log.cc:844] Serialized 506 byte log entry
0709 13:12:04.463945 (+693332us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:04.464003 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:04.464100 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:06.148972 (+1684872us) raft_consensus.cc:1685] finished
0709 13:12:06.149010 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.149215 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":633416,"threads_started":1,"wal-append.queue_time_us":1096912}
I20260709 13:12:06.150607 28761 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 6: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:12:06.152547 28015 debug-util.cc:398] Leaking SignalData structure 0x7b080003eca0 after lost signal to thread 26962
I20260709 13:12:06.153023 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.153774 28761 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:06.153918 30728 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:06.154807 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:06.155498 28761 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.157595 30607 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.158457 30732 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.159458 30732 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.161497 30648 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.127059 28086 raft_consensus.cc:606] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:06.022809 28728 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:06.127058 30764 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.035s	sys 0.033s
I20260709 13:12:06.017004 30659 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:05.968258 30543 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:06.133677 28226 rpcz_store.cc:269] 0709 13:12:03.864307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.871381 (+  7074us) service_pool.cc:224] Handling call
0709 13:12:06.025510 (+2154129us) spinlock_profiling.cc:243] Waited 56.7 ms on lock 0x7b5400287154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:06.025815 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130511232}
I20260709 13:12:06.143515 28235 rpcz_store.cc:276] 0709 13:12:04.322847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.324126 (+  1279us) service_pool.cc:224] Handling call
0709 13:12:04.324597 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:04.324993 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:04.325031 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:04.338060 (+ 13029us) log.cc:844] Serialized 506 byte log entry
0709 13:12:06.142456 (+1804396us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:06.142515 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.142672 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:06.142710 (+    38us) raft_consensus.cc:1685] finished
0709 13:12:06.142724 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.142945 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1742722,"threads_started":1,"wal-append.queue_time_us":835724}
W20260709 13:12:06.137739 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 1.24424e+06us
I20260709 13:12:06.170032 30731 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.170722 30732 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.163915 30751 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:06.168808 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 1.81455e+06us
I20260709 13:12:06.177793 28764 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:06.158419 30607 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.180948 28758 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:06.141973 30623 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.190886 30748 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.185883 30606 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.186780 30751 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.189343 30607 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:05.077332 30489 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.182793 30766 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.193909 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:06.194361 30779 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.195518 28763 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:06.179030 30504 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.201164 30607 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:06.200810 30748 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.202880 30504 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.203017 30751 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.203140 30748 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.203752 30607 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.204181 30607 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.204280 30732 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:06.205000 30732 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:06.205986 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.206797 28758 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:06.203577 30768 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.207453 30606 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.207751 30768 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.208596 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:06.209622 30731 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.200204 30779 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.210597 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.170900 30328 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.211660 28764 raft_consensus.cc:2410] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:06.212954 30328 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.214564 30489 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:06.215420 30489 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:06.214113 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.217926 30489 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.218072 30793 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.219071 30456 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:06.218904 30793 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.219612 30456 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:06.214056 30794 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.221161 30456 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.221261 30794 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.223651 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:06.195047 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 1.30164e+06us
I20260709 13:12:06.224601 28232 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:06.206439 30732 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.221855 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:06.225610 30794 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.226481 30781 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.240657 28235 raft_consensus.cc:2410] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:12:06.242414 30781 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.225617 28762 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.233223 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.244607 28232 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:06.245102 30748 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.245651 30748 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:06.235571 30299 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 2.081s	user 0.016s	sys 0.002s
I20260709 13:12:06.247051 30299 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.248642 30752 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.249362 30299 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.249930 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 701) took 1880 ms. Trace:
I20260709 13:12:06.250367 28230 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:06.250293 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.251225 28233 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:12:06.251339 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1568) took 2593 ms. Trace:
I20260709 13:12:06.252293 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 424) took 2388 ms. Trace:
I20260709 13:12:06.253106 30789 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.253916 30789 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.254758 28198 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.236704 28125 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:06.227499 30780 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.222824 30456 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.252663 28230 rpcz_store.cc:276] 0709 13:12:03.863465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.865971 (+  2506us) service_pool.cc:224] Handling call
0709 13:12:03.866290 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:06.250931 (+2384641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.252026 (+  1095us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54001f5ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:06.252202 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5482018688}
I20260709 13:12:06.265882 30299 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:06.268702 30299 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.269182 30299 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.269374 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.270725 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:06.271227 30606 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:06.271739 30606 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.272114 30606 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:06.272995 30751 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:06.251735 28081 rpcz_store.cc:276] 0709 13:12:03.657465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.657604 (+   139us) service_pool.cc:224] Handling call
0709 13:12:03.735356 (+ 77752us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:03.751510 (+ 16154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:03.751559 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:03.756298 (+  4739us) log.cc:844] Serialized 52 byte log entry
0709 13:12:03.982458 (+226160us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:03.982503 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:03.982582 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:06.250868 (+2268286us) raft_consensus.cc:1685] finished
0709 13:12:06.250905 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.251135 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8905,"raft.queue_time_us":9075,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":15674,"spinlock_wait_cycles":868608,"thread_start_us":217338,"threads_started":1,"wal-append.queue_time_us":636276,"wal-append.run_cpu_time_us":6436,"wal-append.run_wall_time_us":1672816}
I20260709 13:12:06.274526 30780 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:06.275758 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 253238us
I20260709 13:12:06.279537 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:06.281184 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:06.281740 28124 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.283509 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:06.283869 30789 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.284518 28086 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
W20260709 13:12:06.279834 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.285020 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.285806 28233 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
W20260709 13:12:06.285888 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49435.4us
I20260709 13:12:06.250375 28755 rpcz_store.cc:276] 0709 13:12:04.369455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.369633 (+   178us) service_pool.cc:224] Handling call
0709 13:12:04.370052 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:05.922975 (+1552923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.249424 (+326449us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54004d21d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:06.249718 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2563863808}
I20260709 13:12:06.286119 30752 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.286870 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:06.286972 30752 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.289870 30793 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:06.285912 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.291486 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.292156 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.288941 30767 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:06.293143 30794 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.284569 30789 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.294037 30794 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:06.295779 28759 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 711) took 2367 ms (client timeout 1830 ms). Trace:
I20260709 13:12:06.297003 30767 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:06.297209 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:06.297523 30767 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.297982 30767 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:06.298409 28081 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:12:06.299635 30794 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.300372 30794 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:06.301342 30287 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.920s	user 0.020s	sys 0.005s
I20260709 13:12:06.301019 30794 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.301811 30287 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:06.303188 28078 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1563) took 2656 ms (client timeout 1835 ms). Trace:
I20260709 13:12:06.303403 30780 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:06.303838 30780 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:06.303643 28078 rpcz_store.cc:269] 0709 13:12:03.646732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.646863 (+   131us) service_pool.cc:224] Handling call
0709 13:12:06.302829 (+2655966us) spinlock_profiling.cc:243] Waited 2.65 s on lock 0x7b54000c80d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:06.302977 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6103828736}
I20260709 13:12:06.304134 30287 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.304345 30780 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:06.296265 28759 rpcz_store.cc:269] 0709 13:12:03.927757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.927923 (+   166us) service_pool.cc:224] Handling call
0709 13:12:06.295271 (+2367348us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b54007693d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:06.295550 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2192297728}
I20260709 13:12:06.308525 30752 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.310904 28760 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:06.313477 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 705) took 1732 ms. Trace:
I20260709 13:12:06.313594 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:06.313887 28760 rpcz_store.cc:276] 0709 13:12:04.580834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:04.581349 (+   515us) service_pool.cc:224] Handling call
0709 13:12:04.581666 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:06.311446 (+1729780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:06.312679 (+  1233us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b5400686fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:06.313403 (+   724us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3975689472}
I20260709 13:12:06.314563 30779 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:06.317622 30748 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:06.282833 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.319165 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2730425290001222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
I20260709 13:12:06.319936 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.321348 30489 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:06.321542 28015 debug-util.cc:398] Leaking SignalData structure 0x7b080017c280 after lost signal to thread 28018
W20260709 13:12:06.324347 28015 debug-util.cc:398] Leaking SignalData structure 0x7b0800196700 after lost signal to thread 28032
I20260709 13:12:06.321993 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.325493 28759 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:06.272274 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.239698 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:06.326772 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 54599.7us
I20260709 13:12:06.329180 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.331754 28761 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:06.335076 28229 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:06.330436 30733 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:06.345088 30794 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.335971 30793 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.345914 30780 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.346454 30793 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.349406 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.351500 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.351999 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:06.352239 30609 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:06.352485 28233 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:06.340931 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.352685 30609 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.351468 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.353404 30609 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:06.353936 28235 raft_consensus.cc:2410] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:06.352762 28232 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.329901 30752 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.356029 30752 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:06.356863 29808 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.357436 28760 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:06.357497 29808 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.358345 30752 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.358897 30752 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.359730 29808 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.359479 30752 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.360268 29808 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:06.360903 30756 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.356s	user 0.004s	sys 0.002s
I20260709 13:12:06.364495 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:06.364705 28015 debug-util.cc:398] Leaking SignalData structure 0x7b08002fd460 after lost signal to thread 28110
W20260709 13:12:06.364962 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 45156.6us
W20260709 13:12:06.365867 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 52367.6us
W20260709 13:12:06.367928 28015 debug-util.cc:398] Leaking SignalData structure 0x7b08002e2c20 after lost signal to thread 28111
I20260709 13:12:06.348513 30780 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:06.351378 30746 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.397s	user 0.008s	sys 0.007s
I20260709 13:12:06.373018 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:06.374257 28015 debug-util.cc:398] Leaking SignalData structure 0x7b08002fbf40 after lost signal to thread 28185
I20260709 13:12:06.374734 29808 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.377148 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:06.377832 28081 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:06.362672 30733 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.379268 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:06.348543 30793 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.374248 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:06.382076 28230 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.381464 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:06.381623 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:06.385516 30206 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:06.385421 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:06.386029 30206 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.386474 28234 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:06.387596 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.388226 28234 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:06.388857 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1574) took 3565 ms. Trace:
I20260709 13:12:06.386438 30206 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:06.389199 30485 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:06.390568 30485 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:06.389348 28227 rpcz_store.cc:276] 0709 13:12:02.824833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.825013 (+   180us) service_pool.cc:224] Handling call
0709 13:12:02.825504 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:03.064113 (+238609us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:12:03.064142 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:03.541915 (+477773us) spinlock_profiling.cc:243] Waited 4.99 ms on lock 0x7b2c0004c898. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:03.542321 (+   406us) log.cc:844] Serialized 26 byte log entry
0709 13:12:04.285627 (+743306us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:04.285683 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:04.285774 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:06.361997 (+2076223us) raft_consensus.cc:1685] finished
0709 13:12:06.362039 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:06.388647 (+ 26608us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6791,"raft.queue_time_us":237601,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":12239,"spinlock_wait_cycles":11472128,"thread_start_us":742876,"threads_started":1,"wal-append.queue_time_us":1321407}
I20260709 13:12:06.390982 30485 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:06.391665 28226 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:12:06.382009 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:06.383793 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.393703 30752 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.405884 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:06.406952 28229 raft_consensus.cc:2410] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:12:06.407498 28077 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:06.406955 30606 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.410451 30606 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.410777 30789 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:06.396051 28232 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:06.411648 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.411893 28229 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:06.362782 30489 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.413026 28228 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
W20260709 13:12:06.414053 30734 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.577s	user 0.007s	sys 0.012s
I20260709 13:12:06.414312 30489 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.414525 30607 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.414690 30734 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.416000 28085 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:06.416276 30607 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.412999 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:06.417371 30789 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:06.417608 30779 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:06.417814 30789 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.392062 28226 pending_rounds.cc:85] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 6
I20260709 13:12:06.418370 30789 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:06.418640 28125 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:06.419060 28762 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:12:06.420369 28125 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:06.420902 30779 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:06.422019 30299 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.451630 30299 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.453056 28231 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:06.453595 30734 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.453590 30779 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:06.455582 30779 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.456775 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.457172 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.457878 28757 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
W20260709 13:12:06.459493 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 79984.3us
I20260709 13:12:06.461262 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:06.379477 28728 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.771s (SENT) [suppressed 7 similar messages]
I20260709 13:12:06.427248 30789 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.428782 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.429313 30485 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.431274 30728 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:06.432006 30623 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.468406 30728 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.432238 30442 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:06.432631 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.435012 29808 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:06.434341 30659 raft_consensus.cc:2955] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:12:06.437525 28233 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:06.437742 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:06.418782 28226 pending_rounds.cc:107] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:12:06.441387 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.441581 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:06.444563 28230 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:06.384287 30733 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:06.394681 28015 debug-util.cc:398] Leaking SignalData structure 0x7b0800247300 after lost signal to thread 28259
I20260709 13:12:06.459299 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:06.464697 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:06.463476 28757 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:06.467283 30485 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.468492 30623 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.435223 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:06.475919 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.456207 30779 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.471413 28226 raft_consensus.cc:2889] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:06.473587 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 94143.9us
W20260709 13:12:06.476931 28015 debug-util.cc:398] Leaking SignalData structure 0x7b08003ce7a0 after lost signal to thread 28789
I20260709 13:12:06.466131 30206 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:06.479995 28015 debug-util.cc:398] Leaking SignalData structure 0x7b08003ce860 after lost signal to thread 28790
I20260709 13:12:06.480811 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.483134 28761 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:06.483549 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.484556 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.484823 30734 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:06.485352 30206 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.485472 30734 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:06.486683 30734 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.488511 28228 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:06.489183 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 151069us
I20260709 13:12:06.491748 30328 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:06.492367 30328 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:06.492543 28759 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:06.492650 30748 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.493486 30748 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.496779 28763 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:06.497668 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:06.498000 30728 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.498587 30728 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:06.470607 30442 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.499599 30648 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:06.478121 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.501008 30442 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.501305 28758 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:06.503037 28756 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:06.493506 30328 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.492439 28764 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
W20260709 13:12:06.506203 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47105.6us
W20260709 13:12:06.481410 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 43776.9us
I20260709 13:12:06.508617 28124 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.466176 30768 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.463653 30609 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.503062 30623 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.523072 28760 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:06.507051 30748 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.527458 28228 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:06.529140 30781 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:06.529683 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.532122 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.546414 30735 raft_consensus.cc:993] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:12:06.547228 30735 raft_consensus.cc:1081] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:12:06.543960 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:06.541347 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:06.518834 30504 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:12:06.571358 28015 debug-util.cc:398] Leaking SignalData structure 0x7b0800340940 after lost signal to thread 30691
I20260709 13:12:06.514552 30780 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:06.573016 28198 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:06.574105 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 109748us
W20260709 13:12:06.566116 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 53352.2us
I20260709 13:12:06.581130 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.548854 30607 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:06.581826 29808 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:06.529726 30781 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.582163 29808 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.583624 29808 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:06.585320 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 1.66151e+06us
I20260709 13:12:06.582963 30648 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:12:06.589324 30442 raft_consensus.cc:993] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:12:06.590087 30442 raft_consensus.cc:1081] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:12:06.591207 30299 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:06.591538 30424 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:06.591698 30299 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.592262 30299 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.592199 30424 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.593008 30219 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:06.581506 30504 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:06.594677 30504 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.532991 28755 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:06.572625 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:06.596679 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.609072 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.611415 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.520813 30732 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:12:06.574155 30606 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
W20260709 13:12:06.613793 28015 debug-util.cc:398] Leaking SignalData structure 0x7b080037e9c0 after lost signal to thread 30718
W20260709 13:12:06.616614 28015 debug-util.cc:398] Leaking SignalData structure 0x7b08003cc780 after lost signal to thread 30725
I20260709 13:12:06.582922 30607 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:06.588094 30733 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.574079 30767 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.533134 30206 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:12:06.616039 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:06.633070 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 55757us
I20260709 13:12:06.633478 28764 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:06.583272 30623 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:06.636225 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 58327.9us
I20260709 13:12:06.641369 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:06.640017 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:06.641849 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 59893.3us
I20260709 13:12:06.644237 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:06.598047 30781 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.589349 28757 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:06.630331 28756 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:06.617241 30731 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:06.654129 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.657248 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:06.635116 30768 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.663069 30767 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.663848 28233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39712
I20260709 13:12:06.664314 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.664528 28233 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.642541 30488 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:06.665020 28233 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.665119 28230 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:06.636276 30780 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:06.635937 30607 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:06.662693 30609 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.663960 30735 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:06.632287 30733 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.621085 30485 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:06.663434 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:06.704805 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 66261.6us
I20260709 13:12:06.705487 28228 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:06.634100 30648 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:12:06.655514 30748 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:06.715415 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:06.716166 28230 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:12:06.663993 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:06.689703 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.651341 30751 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:06.714668 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:06.675273 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:06.703941 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 92614.6us
I20260709 13:12:06.706210 28232 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39712
I20260709 13:12:06.722020 28232 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.724800 28232 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.725389 28231 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:06.665189 30488 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:06.708964 30485 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.701766 30789 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.642807 30456 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:12:06.712133 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.656900 30731 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.695724 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:06.722450 28226 raft_consensus.cc:2955] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:12:06.661528 30623 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:06.749413 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:06.749766 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.723636 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:06.728363 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:06.711373 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 52682us
I20260709 13:12:06.751192 28234 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.731921 28085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:12:06.751895 28085 raft_consensus.cc:606] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:06.752282 28085 raft_consensus.cc:614] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:06.713820 30328 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:12:06.741581 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.714007 30748 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.745464 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:06.745888 30424 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:06.746093 30723 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:06.746434 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:06.736794 30485 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.759501 30723 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.759980 30723 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:06.748164 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:06.720376 30751 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.718264 30735 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:06.712776 30614 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:06.713994 30768 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:06.754010 28232 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.747439 30456 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:06.748919 30731 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:06.730844 30609 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:12:06.773540 28230 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:06.762028 30748 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.768802 28081 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:06.764691 30751 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:06.768292 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 126966us
W20260709 13:12:06.769315 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 94179.9us
W20260709 13:12:06.771262 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 102820us
I20260709 13:12:06.787766 28230 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:06.788566 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.797623 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:06.801393 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.801568 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.802155 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.769766 30456 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:06.771239 30328 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:06.762092 30789 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:12:06.775372 30614 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:06.821720 30659 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:36752 (request call id 14) took 5194 ms. Trace:
I20260709 13:12:06.822221 30659 rpcz_store.cc:276] 0709 13:12:01.626947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.627164 (+   217us) service_pool.cc:224] Handling call
0709 13:12:01.784298 (+157134us) log.cc:844] Serialized 506 byte log entry
0709 13:12:06.821597 (+5037299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56298,"raft.queue_time_us":34280,"raft.run_cpu_time_us":2707,"raft.run_wall_time_us":62774,"thread_start_us":214361,"threads_started":1,"wal-append.queue_time_us":284108,"wal-append.run_cpu_time_us":3021,"wal-append.run_wall_time_us":1206425}
I20260709 13:12:06.825124 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:06.825977 28759 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.829844 30614 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.833002 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:06.833916 28758 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:06.833477 30299 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.834892 30623 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:06.835470 30623 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.837416 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:06.811968 30780 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.838485 30766 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:06.842767 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:06.837651 30623 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:06.848381 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:06.841382 28756 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:06.851220 28234 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
W20260709 13:12:06.853461 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 193438us
I20260709 13:12:06.854316 29808 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.849606 30659 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:06.858973 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.782241 30791 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:06.809271 30648 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:12:06.818996 30752 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:06.856897 28156 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:12:06.857614 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:06.851176 28077 raft_consensus.cc:2376] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 6. Re-sending same reply.
I20260709 13:12:06.812536 30606 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:06.836548 30299 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.865976 29808 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.859980 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:06.867975 30735 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.868577 30767 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:12:06.877169 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 107612us
I20260709 13:12:06.901607 30488 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.903369 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:06.904487 28232 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:06.905148 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:06.928123 28235 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:12:06.893853 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 56550us
W20260709 13:12:06.886510 30811 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:12:06.905933 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:06.970569 28764 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:06.904479 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:06.971952 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 67558.1us
I20260709 13:12:06.926280 28759 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:06.961386 30648 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:06.885068 28156 raft_consensus.cc:606] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:06.973073 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:06.916702 28233 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:06.975303 28228 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:06.877359 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:07.003400 28757 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:06.975965 28198 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:06.892985 30808 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:12:07.039813 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 64892.8us
I20260709 13:12:06.966161 30485 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:06.890105 30752 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:06.918620 30606 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:07.012259 28124 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:07.050217 28235 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:06.906610 30733 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:07.025593 30809 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:07.028427 30751 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.033102 28086 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:12:07.036357 30723 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:07.036944 28758 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:06.885452 30456 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.018123 30812 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:12:06.877358 30328 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.050707 30299 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:07.009891 30732 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:07.039906 30808 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:07.045603 28233 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:07.049665 30543 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:12:07.055245 30767 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:07.052927 30733 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:07.082018 30607 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.065886 30609 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:12:07.064603 30781 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.108371 30781 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:07.091457 30299 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:07.102463 30818 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.105640 30606 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:07.111531 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.088598 28764 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:07.101712 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:07.090226 30767 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:07.091198 30814 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:07.111341 30659 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:07.180625 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 138671us
W20260709 13:12:07.174386 30779 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.661s	user 0.009s	sys 0.010s
I20260709 13:12:07.173982 30767 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:07.186839 30820 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:12:07.188982 30299 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:07.229142 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 47109.6us
I20260709 13:12:07.234464 28198 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:07.239718 29808 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:07.240430 29808 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:07.244199 30823 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:12:07.227105 30748 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.228214 30804 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:07.245925 30804 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.247293 30748 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:07.248082 30804 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:07.248294 30723 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:07.248889 30814 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:07.250103 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.249861 30733 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:07.250660 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.251226 30733 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:07.251354 30814 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:07.174180 30818 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.254638 30818 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:07.254874 30485 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:07.255187 30818 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:07.175439 30731 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.255336 30424 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.255842 30818 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.257266 30731 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:07.258180 30780 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.259889 28232 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:07.211972 30779 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:07.273075 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:07.274343 28233 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:07.261113 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:07.277006 28226 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:07.278159 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:07.269840 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.292826 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:07.293797 28764 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:07.287482 29808 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:07.298817 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 59449.5us
W20260709 13:12:07.300163 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 118638us
I20260709 13:12:07.177606 30752 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.305861 28156 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:07.273115 30668 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:07.281561 30814 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:07.226501 30659 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:07.272015 30328 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.258757 30780 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.332218 28226 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:07.333312 28158 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:07.342798 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.346820 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:07.347393 28758 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:07.342361 30781 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:12:07.319103 30751 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:07.314175 30814 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:07.349025 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:07.259385 30808 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.354120 28764 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
W20260709 13:12:07.349332 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 302919us
I20260709 13:12:07.335884 30804 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:07.360607 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:07.361720 30489 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:07.362140 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:07.362304 30489 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:07.363576 30543 raft_consensus.cc:2764] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:12:07.353677 30668 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.363412 30489 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.364923 30543 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:07.365777 30543 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:07.366138 30543 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:07.367421 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:07.367360 30543 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.368176 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:07.370437 30543 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:07.370867 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:07.371614 28085 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:07.373265 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:07.374073 28084 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:07.377312 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:07.378067 28078 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:12:07.379218 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.381121 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:07.381887 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:07.382551 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:07.382684 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:07.383352 28083 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:07.383189 28086 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:07.383153 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:07.367470 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:07.386583 28764 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:07.390266 28156 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:07.354142 30299 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:07.340915 30328 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:07.400838 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:07.401403 30827 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:12:07.401654 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:07.401940 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:07.402352 30826 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:12:07.402686 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:07.418256 28082 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.406822 28079 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:07.403086 30822 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:12:07.410763 28153 raft_consensus.cc:2410] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:12:07.360018 30804 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:07.414063 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.389986 30442 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:07.415083 28160 raft_consensus.cc:2410] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:12:07.403211 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:07.428913 28155 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:07.430642 30659 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:07.431133 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:07.432816 28159 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:07.433653 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:07.434087 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:07.434409 28160 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:07.400838 30810 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:07.439810 28083 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:07.434691 28159 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:07.442130 28084 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:07.442982 28764 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:07.444434 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:07.411973 30299 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:07.446820 30668 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:07.448582 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:07.428153 30804 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.428097 30442 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:07.474087 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:07.486197 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:07.489778 30607 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:07.525354 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:07.536615 30614 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:07.537032 30834 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:07.537458 30614 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:07.538034 30834 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:07.539252 30614 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.542021 30614 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:07.542943 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:07.545842 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:07.547174 30731 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:07.539547 30834 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.454095 30485 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.566293 28078 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:12:07.565954 28153 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:12:07.578068 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:07.569675 28152 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:07.575577 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:07.474589 30818 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:07.475867 30779 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:07.428088 30814 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:07.588184 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:07.599315 28158 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 3.
I20260709 13:12:07.566349 30780 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:07.605432 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:07.606920 28153 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:07.608717 30607 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:07.609884 30607 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:07.580641 30206 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:07.660310 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.661283 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:07.663206 28079 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:07.661962 28155 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:07.573400 30485 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.470438 30810 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:07.720964 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.725405 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:07.726130 28158 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:07.659574 30746 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.611454 30607 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.775134 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:07.758813 30840 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.341s	user 0.002s	sys 0.000s
I20260709 13:12:07.776873 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:07.673789 30442 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.758821 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:07.778580 28079 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:07.776038 28229 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:12:07.785964 28229 pending_rounds.cc:85] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 3
I20260709 13:12:07.786321 28229 pending_rounds.cc:107] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:12:07.786681 28229 raft_consensus.cc:2889] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:07.791678 30829 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 13:12:07.791985 30830 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:07.792656 30830 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:07.796200 30789 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:07.797008 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:07.797636 30846 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:12:07.805117 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:07.796044 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:07.809336 28764 raft_consensus.cc:2410] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:12:07.773591 30837 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:07.816246 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:07.818533 28080 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:07.816797 28226 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:07.821022 28082 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:12:07.821722 28083 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:07.659574 30299 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.782950 30623 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:07.759368 30206 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:07.886351 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.855093 28079 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:12:07.888204 28226 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:07.845849 30767 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.000s
I20260709 13:12:07.796656 30789 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:07.894716 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:07.816013 30734 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:07.891697 28079 pending_rounds.cc:85] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 3
W20260709 13:12:07.899380 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:07.815753 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:07.813822 30328 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:07.773489 30746 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:07.902190 28079 pending_rounds.cc:107] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:12:07.913328 28079 raft_consensus.cc:2889] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:07.914383 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:07.817700 30424 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:12:07.923385 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:07.929278 30848 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:12:07.929663 28079 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:07.930447 30850 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:12:07.931025 28079 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:07.941354 28086 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.000s
I20260709 13:12:07.915791 30623 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:07.943104 30746 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:07.943724 28086 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:07.938649 28155 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:07.946750 28080 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
W20260709 13:12:07.933089 30841 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:12:07.915145 30734 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:07.949676 28082 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:07.950635 28079 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:07.900698 30767 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:07.957808 30821 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:12:07.960399 30843 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.004s
W20260709 13:12:07.961515 30836 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.468s	user 0.001s	sys 0.000s
W20260709 13:12:07.962240 30733 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.710s	user 0.013s	sys 0.000s
I20260709 13:12:07.984205 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:07.985911 28155 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:07.986132 28085 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.003s
I20260709 13:12:07.995417 28085 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:08.008886 28157 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:08.018743 28160 consensus_meta.cc:287] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
I20260709 13:12:08.019580 28160 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
W20260709 13:12:08.021088 28159 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.579s	user 0.012s	sys 0.000s
I20260709 13:12:08.021685 28159 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:08.026711 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.027607 28160 raft_consensus.cc:2368] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:08.030002 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.030747 28159 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:08.031368 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.042891 28159 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:08.047222 28155 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:08.051270 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.051926 28160 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:12:08.053035 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.052798 30734 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:08.055022 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.055765 28079 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:08.061316 28159 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
W20260709 13:12:08.061483 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:08.065259 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.070827 30810 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:08.073134 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:08.074033 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.074424 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:08.075094 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.075402 30659 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.075999 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:08.076350 30857 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:12:08.077186 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.077822 28079 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
W20260709 13:12:08.077973 30859 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:12:08.079960 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.073870 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.081632 28082 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:08.014645 30789 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:08.074458 30856 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:12:08.097699 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:08.067629 30830 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:08.136616 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.140702 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.141541 28229 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:08.142263 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.142805 30789 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:08.143110 28159 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:12:08.147637 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.149171 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.149957 28229 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 2.
I20260709 13:12:08.150652 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:08.151455 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:08.152374 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.155892 28160 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:08.155977 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.157243 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.159524 28229 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:08.159621 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.160930 28085 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:08.177265 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.187732 28080 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:08.187871 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.181173 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.189875 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.186022 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.190332 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.191339 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.192093 28078 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:08.192902 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.193933 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.194932 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.196132 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.188717 28079 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:08.197453 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.076036 30659 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:08.194818 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.198408 28078 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:08.202392 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49368.7us
I20260709 13:12:08.203222 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.197705 30767 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:08.203864 28079 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:08.204622 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:08.205595 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.206444 28085 raft_consensus.cc:2368] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:08.206696 28082 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:08.207360 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.207783 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.219493 28078 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:08.208616 28159 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:12:08.220103 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:08.220813 28764 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
W20260709 13:12:08.212400 30804 consensus_meta.cc:287] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.646s	user 0.007s	sys 0.001s
I20260709 13:12:08.222366 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:08.366086 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.387792 28159 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:08.367950 28226 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:08.271018 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.223755 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.390985 28080 raft_consensus.cc:2410] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:08.285372 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.391978 28085 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:08.355975 28079 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:08.292117 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:08.366019 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.366540 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:08.367642 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.402616 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:08.334821 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 105526us
I20260709 13:12:08.232880 28764 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:08.411650 28078 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:08.417325 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.418299 28161 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:08.412043 28229 raft_consensus.cc:2368] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8.
W20260709 13:12:08.409720 28127 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43184 dropped due to backpressure: service queue is full with 50 items
I20260709 13:12:08.419814 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.420431 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.421509 28159 raft_consensus.cc:2410] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:12:08.408809 28157 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
W20260709 13:12:08.229075 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.420847 28228 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:08.424618 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.424868 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:08.425573 28159 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:08.425371 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.427040 28084 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:08.428015 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.430542 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.432828 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.313880 30766 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:08.333577 30810 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.434877 28759 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:08.435182 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.288520 30768 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.304500 28755 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:08.434387 28763 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:08.464939 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.304590 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 494) took 1401 ms. Trace:
I20260709 13:12:08.267788 30837 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:08.438704 28761 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:12:08.452450 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.394353 30659 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:08.409986 30809 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.462940 28086 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:08.475765 28085 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:08.393486 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:08.465764 28157 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:08.470418 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:08.471072 30206 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:08.477262 28759 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:08.223259 30804 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:08.478879 28762 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:08.493821 28229 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:12:08.481159 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.497689 28086 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:08.464609 30543 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:08.469082 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:08.500993 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.502682 28159 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:08.504123 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.494488 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.506012 28160 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:08.420166 28127 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43184 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1909)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1893)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 422)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1985)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1978)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1990)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1989)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1991)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1977)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1984)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1976)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1986)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1974)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1992)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1999)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1979)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1895)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1891)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1890)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1996)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1997)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1898)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1987)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1892)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1983)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1970)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1971)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 420)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1972)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1981)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1994)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1888)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1896)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1900)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1901)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1908)
Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:43180 (request call id 12)
I20260709 13:12:08.497541 28757 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:08.497315 30809 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:08.513351 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.514510 28159 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:08.515215 30809 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:08.515362 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.517500 28159 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:08.517171 30837 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:08.516275 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.522464 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.438879 30766 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:08.523725 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.524341 28079 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:08.524698 28079 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:08.525440 30766 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:08.525578 28079 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:08.526818 28079 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:08.538545 28159 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:12:08.538743 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:08.539516 28226 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:08.539621 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.540832 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:08.541236 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.541463 28159 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:12:08.541668 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.542253 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:08.542948 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.543350 28231 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:08.544757 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 405) took 1138 ms. Trace:
I20260709 13:12:08.545298 28156 rpcz_store.cc:276] 0709 13:12:07.406390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.414184 (+  7794us) service_pool.cc:224] Handling call
0709 13:12:07.414573 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.791172 (+376599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:07.791195 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:07.791226 (+    31us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:07.791245 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:08.544392 (+753147us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54003769d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:08.544452 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:08.544632 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848295296}
I20260709 13:12:08.547228 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.542349 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.548517 28079 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:08.565804 28230 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:12:08.552103 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.567550 28231 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:08.567863 28082 raft_consensus.cc:2376] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5. Re-sending same reply.
W20260709 13:12:08.566807 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.569805 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.570453 30814 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:08.570863 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.571936 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.572317 30804 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:08.550474 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.573314 28082 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:08.573840 28226 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:08.477814 30830 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:12:08.569099 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.559466 28157 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:08.568609 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:08.575235 30861 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.435s	user 0.004s	sys 0.000s
I20260709 13:12:08.575404 30830 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:08.575599 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.576373 28157 raft_consensus.cc:2368] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:08.576287 30830 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:08.568348 30485 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:08.567813 30623 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:08.578092 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.577792 30830 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:08.578681 28153 raft_consensus.cc:2376] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3. Re-sending same reply.
I20260709 13:12:08.579619 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:08.578507 30866 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.438s	user 0.002s	sys 0.002s
I20260709 13:12:08.595140 28761 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:08.593416 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.597693 28233 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:08.580613 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.596135 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.598955 28153 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:12:08.584926 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.599839 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:08.600841 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.601037 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.595620 30442 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:08.602013 30735 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:08.603644 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.600798 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.574635 28082 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:08.604473 30488 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:08.615434 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.606699 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.607782 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.606644 28125 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:50458 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 423)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1895)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1891)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1890)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1996)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1997)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1898)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1987)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 1892)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1983)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1970)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1971)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 420)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1972)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1981)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 1994)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1888)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1896)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1900)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1901)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1908)
Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:43180 (request call id 12)
I20260709 13:12:08.608886 28234 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:08.616556 28228 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:08.611526 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:08.611816 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.595376 28231 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:08.620203 30623 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:08.620579 28153 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:08.548606 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.569185 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:08.603803 30828 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:08.613875 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.613853 28158 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 2.
I20260709 13:12:08.606858 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:08.621845 28085 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:08.587334 28081 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 3.
I20260709 13:12:08.621851 28156 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:08.621084 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:08.628696 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.629593 28160 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:08.630776 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.637271 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.638542 28158 raft_consensus.cc:2410] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:12:08.639268 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.639442 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:08.640348 28156 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 7: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:08.633074 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.643245 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:08.652246 28153 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:08.615594 30735 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:08.647137 28198 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:08.650254 28160 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:08.612385 30830 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:08.468420 28756 rpcz_store.cc:276] 0709 13:12:06.902826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.903018 (+   192us) service_pool.cc:224] Handling call
0709 13:12:08.304364 (+1401346us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b54006c02d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:08.304523 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2145540864}
I20260709 13:12:08.641849 30838 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:08.643944 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 175127us
I20260709 13:12:08.495805 30810 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:08.620605 30623 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:08.714886 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.717082 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 1921) took 1119 ms. Trace:
I20260709 13:12:08.717621 28083 rpcz_store.cc:276] 0709 13:12:07.576516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.576715 (+   199us) service_pool.cc:224] Handling call
0709 13:12:07.821549 (+244834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:07.822077 (+   528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:08.695658 (+873581us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54000c8fd0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:08.695902 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532667136}
I20260709 13:12:08.658177 28161 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:08.689536 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.725790 28160 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:08.673755 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.625250 30543 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:08.690310 28763 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:08.691460 28761 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:08.697666 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.468703 30768 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:08.713397 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:08.738318 28153 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 8: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:08.652891 30808 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:08.734968 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.669591 30614 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:08.670105 30812 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:08.714892 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:08.628558 30819 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:12:08.746554 28159 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:08.747745 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.741611 28160 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:08.742172 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.739885 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:08.744890 28085 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:08.752527 28161 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:08.757095 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 106101us
I20260709 13:12:08.757947 28125 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:08.765666 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.769094 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.739663 30623 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:08.734799 30456 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:08.774631 28125 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:08.769779 30485 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:08.776809 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:08.767427 30870 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:08.780421 28157 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:08.743500 30614 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:08.786405 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:08.743537 30733 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:08.789481 28153 raft_consensus.cc:2410] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:12:08.775856 28160 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:08.794622 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 136073us
I20260709 13:12:08.672559 30814 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:12:08.798803 28161 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:08.820979 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:08.779477 30837 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:08.830797 28153 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:12:08.822660 28160 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:08.785477 30873 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:08.745913 30808 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:08.841737 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48357.1us
I20260709 13:12:08.842481 28161 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
W20260709 13:12:08.844878 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 70351.5us
I20260709 13:12:08.817914 30804 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:08.797104 30733 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:08.872910 28160 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:08.895236 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:08.833567 28153 pending_rounds.cc:85] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 3
I20260709 13:12:08.797600 30614 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:08.782035 30735 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:08.832249 30766 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:08.969259 28153 pending_rounds.cc:107] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:12:08.937664 28157 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 6. Ops: []
I20260709 13:12:08.927748 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:08.971117 28161 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:08.973554 28153 raft_consensus.cc:2889] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:08.975404 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 81511.7us
I20260709 13:12:08.992138 28078 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:12:08.932191 30874 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:12:08.996023 28160 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:12:08.997267 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 270196us
I20260709 13:12:08.893764 30780 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:12:08.998490 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:08.896703 30733 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:08.999881 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 327669us
I20260709 13:12:08.844367 30873 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:08.895865 30804 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:08.931844 30768 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.014295 28160 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:08.819362 30870 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.063133 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:08.932646 30746 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:12:08.896212 30779 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 13:12:08.845714 30299 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.191s
I20260709 13:12:08.998378 30735 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:08.977166 30878 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 13:12:09.001065 30780 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:09.039326 30873 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:09.084493 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:09.086923 28764 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:09.037185 30804 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:09.097616 28226 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:09.065080 30768 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:08.893146 30543 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:09.108450 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.083009 30780 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:09.033668 30812 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:09.099821 30876 ts_tablet_manager.cc:933] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:12:09.097982 30890 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:09.120261 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.123093 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:09.076987 30882 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:12:09.113555 30485 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:09.136994 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 100932us
W20260709 13:12:09.140270 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 289168us
I20260709 13:12:09.122392 30768 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:12:09.138774 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 205141us
I20260709 13:12:09.143415 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:09.135468 28755 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:09.135128 30810 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.142967 30768 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.157102 30768 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:09.151103 30889 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:09.153388 30870 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:09.162631 30881 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
W20260709 13:12:09.163619 30442 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.003s
I20260709 13:12:09.164255 28077 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:09.175143 30768 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:09.179761 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:09.182309 30880 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:12:09.183306 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:09.187141 30809 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.187820 30809 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.188513 30837 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:09.189884 30837 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:09.191543 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:09.192101 30837 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.196950 30731 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.197734 30731 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.199139 30804 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.200052 30808 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.172952 30780 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:09.205191 28127 leader_election.cc:400] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8.
I20260709 13:12:09.205628 28127 leader_election.cc:403] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 13:12:09.208866 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 64335.4us
I20260709 13:12:09.209911 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:09.213930 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.214798 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:09.129478 30890 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.216151 28124 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:09.203364 30766 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:09.223624 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:09.223752 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:09.242735 28156 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:09.243835 30891 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:09.244844 30891 consensus_queue.cc:1243] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:12:09.245891 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.247799 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:09.250267 28764 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:09.249818 30766 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:09.252813 28153 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:09.253134 28124 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:09.255192 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.256331 28153 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:09.259080 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.260457 28155 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 7: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:09.247948 30891 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:09.262696 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.265499 28156 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:09.267488 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.268309 28156 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:09.270373 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.278820 30894 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:09.201243 30878 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:12:09.214311 30810 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8.
I20260709 13:12:09.199141 30623 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:09.289273 28153 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:09.290957 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:09.292119 28764 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:09.288592 30804 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:09.221475 30837 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:09.241665 30812 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:09.296010 28081 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1584) took 1868 ms (client timeout 1534 ms). Trace:
W20260709 13:12:09.300652 28081 rpcz_store.cc:269] 0709 13:12:07.427236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.427501 (+   265us) service_pool.cc:224] Handling call
0709 13:12:09.295719 (+1868218us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002972d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:09.295905 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2657464704}
I20260709 13:12:09.221261 30897 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:09.295738 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 111369us
W20260709 13:12:09.305063 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 377410us
I20260709 13:12:09.310103 28125 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:09.310353 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 1989) took 1233 ms. Trace:
W20260709 13:12:09.302075 30488 consensus_meta.cc:287] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.681s	user 0.011s	sys 0.001s
I20260709 13:12:09.312583 28227 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:09.317448 28157 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:09.314859 28159 rpcz_store.cc:276] 0709 13:12:08.077193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.568860 (+491667us) service_pool.cc:224] Handling call
0709 13:12:09.310163 (+741303us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400255154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:09.310292 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394235136}
I20260709 13:12:09.295248 30893 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:09.322506 28764 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:09.303171 30837 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:09.325431 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 182099us
I20260709 13:12:09.322139 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 731) took 1025 ms. Trace:
I20260709 13:12:09.328866 28760 rpcz_store.cc:276] 0709 13:12:08.290682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.290823 (+   141us) service_pool.cc:224] Handling call
0709 13:12:09.313037 (+1022214us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:12:09.313257 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:09.315513 (+  2256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.315549 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:09.315596 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:09.315634 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.315700 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.315842 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50454272}
I20260709 13:12:09.309487 30804 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:09.332094 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:09.335525 28230 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:09.311398 30898 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:09.336918 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.339562 28160 raft_consensus.cc:2376] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 4. Re-sending same reply.
I20260709 13:12:09.340955 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.333945 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:09.341523 30905 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:09.319393 30808 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.301918 30485 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:09.351539 30901 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.103s	user 0.004s	sys 0.000s
I20260709 13:12:09.352986 30909 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.353562 30909 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:09.303582 30810 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
W20260709 13:12:09.335922 30830 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.002s
I20260709 13:12:09.322175 30809 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.331830 30837 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:09.369515 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:09.305225 30894 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:12:09.371659 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.380540 28160 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:09.332494 28152 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:12:09.375793 28155 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:09.377123 30804 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:09.315518 30902 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:12:09.346645 30898 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:12:09.373955 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 45827.9us
I20260709 13:12:09.395475 28728 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:09.384827 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1908) took 1068 ms. Trace:
W20260709 13:12:09.388912 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.400287 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.357075 30488 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.394912 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:09.396579 28161 rpcz_store.cc:276] 0709 13:12:08.311272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.974976 (+663704us) service_pool.cc:224] Handling call
0709 13:12:08.975327 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.116702 (+141375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.116789 (+    87us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:09.125583 (+  8794us) log.cc:844] Serialized 26 byte log entry
0709 13:12:09.319824 (+194241us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:09.319869 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.319937 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:09.379493 (+ 59556us) raft_consensus.cc:1685] finished
0709 13:12:09.379517 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:09.379641 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10823,"raft.queue_time_us":200877,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":94,"spinlock_wait_cycles":38144,"thread_start_us":292807,"threads_started":2,"wal-append.queue_time_us":241156}
I20260709 13:12:09.378932 30900 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:12:09.384979 30809 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:09.405220 30907 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:09.383075 30910 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 13:12:09.382639 30838 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.666s	user 0.008s	sys 0.004s
I20260709 13:12:09.326072 30746 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:12:09.409093 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.389840 30668 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.385922 30766 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:09.372749 30808 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.394498 30768 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.406430 30915 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:09.412253 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 102643us
I20260709 13:12:09.399089 30898 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:09.433382 28199 leader_election.cc:336] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:50458 dropped due to backpressure: service queue is full with 50 items
I20260709 13:12:09.435253 28227 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:09.413513 30838 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:09.382293 30908 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:12:09.409392 30809 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.407236 30804 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:09.476329 30873 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.531181 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:09.529341 28085 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:12:09.448318 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.545929 28084 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:09.482255 30909 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.546705 28159 raft_consensus.cc:2410] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:12:09.429589 30731 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:09.436193 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:09.520653 30614 consensus_meta.cc:287] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.003s
I20260709 13:12:09.488108 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:09.430985 30488 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:09.549289 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 61291.8us
I20260709 13:12:09.435415 30780 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.472149 30892 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.140s
I20260709 13:12:09.549094 30614 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:09.564707 30488 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.483450 28158 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:12:09.487962 28154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 407) took 2056 ms (client timeout 2221 ms). Trace:
I20260709 13:12:09.531468 30804 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:09.551362 30780 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:12:09.586191 28152 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 416) took 1924 ms (client timeout 1891 ms). Trace:
W20260709 13:12:09.586841 28152 rpcz_store.cc:269] 0709 13:12:07.661903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.731519 (+ 69616us) service_pool.cc:224] Handling call
0709 13:12:09.585948 (+1854429us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54001978d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:09.586112 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3246136192}
I20260709 13:12:09.553107 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:09.475212 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:09.548249 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 112156us
I20260709 13:12:09.531530 28230 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:12:09.588967 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 157565us
I20260709 13:12:09.588488 28081 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:09.551512 30731 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:09.565611 30893 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:09.590833 28199 leader_election.cc:400] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:09.591866 30925 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:09.591740 30893 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:09.528914 30623 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.567871 30873 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:09.569267 28228 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:09.602985 30733 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.539s	user 0.008s	sys 0.003s
I20260709 13:12:09.603472 30733 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.563885 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:09.606587 30752 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:09.607092 30752 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:09.607395 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:09.462541 30915 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:09.610018 30879 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.510s	user 0.001s	sys 0.001s
W20260709 13:12:09.573764 28154 rpcz_store.cc:269] 0709 13:12:07.431108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.475105 (+ 43997us) service_pool.cc:224] Handling call
0709 13:12:09.487713 (+2012608us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b540033e854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:09.487860 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183636736}
I20260709 13:12:09.610572 30485 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:12:09.480154 30456 consensus_meta.cc:287] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.691s	user 0.009s	sys 0.000s
I20260709 13:12:09.591325 28199 leader_election.cc:403] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:09.598850 28125 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:09.619388 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:09.620450 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:09.620999 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.600152 28762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 740) took 1196 ms. Trace:
I20260709 13:12:09.572552 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:09.609899 30731 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:09.614003 30892 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:09.563122 30809 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:09.547894 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:09.647377 28159 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:09.649142 28125 leader_election.cc:400] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:09.649441 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:09.649778 28125 leader_election.cc:403] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:12:09.621418 28762 rpcz_store.cc:276] 0709 13:12:08.403805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.403945 (+   140us) service_pool.cc:224] Handling call
0709 13:12:08.478639 (+ 74694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:08.492626 (+ 13987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.589500 (+1096874us) spinlock_profiling.cc:243] Waited 47 ms on lock 0x7b5400531e50. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:09.600069 (+ 10569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108204928}
I20260709 13:12:09.650192 28081 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:09.642357 28084 raft_consensus.cc:2410] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:12:09.433997 30668 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.641638 30731 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.659590 30746 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:09.655769 30810 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:09.662329 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:09.585506 30808 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.608080 30752 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:09.589818 30780 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:12:09.671082 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 80406.5us
I20260709 13:12:09.646893 30892 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.684832 28156 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:12:09.641690 30768 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:09.447672 30766 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.677141 30808 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.677963 30817 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:09.694195 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1902) took 1388 ms. Trace:
I20260709 13:12:09.670585 30909 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:09.682051 30752 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:09.686651 30892 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:09.671144 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 262448us
I20260709 13:12:09.669859 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:09.681406 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:09.711663 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:09.694798 28157 rpcz_store.cc:276] 0709 13:12:08.305845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.974864 (+669019us) service_pool.cc:224] Handling call
0709 13:12:09.310405 (+335541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:09.317916 (+  7511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.693900 (+375984us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b5400255150. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:09.694109 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770655616}
I20260709 13:12:09.712225 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:09.671172 30731 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:09.714432 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:09.697742 30817 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:09.725031 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:09.737416 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.704327 28232 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:09.718592 30810 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:09.753858 30810 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:09.754678 30810 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:09.721375 30817 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:09.753762 30888 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.701013 30808 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.742725 30897 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:09.751519 28125 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:09.716346 30819 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:12:09.757287 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.757755 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:09.757485 30748 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.757352 30837 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:09.771561 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.753886 30896 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.773628 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:09.742393 30908 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:09.781425 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:09.794407 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:09.761564 30733 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:09.796118 28230 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:09.800944 30915 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:09.809197 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:09.810284 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:09.811635 28764 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:09.821318 28077 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1939) took 1747 ms (client timeout 1675 ms). Trace:
W20260709 13:12:09.821957 28077 rpcz_store.cc:269] 0709 13:12:08.073649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.074163 (+   514us) service_pool.cc:224] Handling call
0709 13:12:09.821070 (+1746907us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000c8ad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:09.821239 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2504853888}
W20260709 13:12:09.825102 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.826849 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:09.827664 28227 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:09.835263 30812 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.835891 30812 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.837677 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:09.838454 28153 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:09.859154 30886 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.709s	user 0.003s	sys 0.001s
I20260709 13:12:09.861943 30905 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:09.862560 30905 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:09.864912 30543 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.007s	sys 0.002s
W20260709 13:12:09.865332 30543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.008s	sys 0.002s
I20260709 13:12:09.866508 30890 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.869592 30812 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.063180 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:09.878439 30891 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.000s	sys 0.005s
W20260709 13:12:09.878978 30891 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.000s	sys 0.005s
I20260709 13:12:09.879307 30891 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:09.796223 30748 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:09.795665 30888 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:09.793356 30897 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:09.803463 30873 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:09.787977 30488 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:12:09.896134 30927 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
W20260709 13:12:09.897027 30926 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:12:10.036980 30543 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:09.897550 30938 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:09.903836 30837 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:09.795420 30779 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.008s	sys 0.001s
I20260709 13:12:09.741993 30614 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:09.805401 30896 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:09.971493 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 104425us
I20260709 13:12:09.904973 30804 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:12:10.070564 30746 raft_consensus.cc:2764] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:12:10.073622 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 264466us
I20260709 13:12:10.075398 28235 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:09.897353 30838 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:12:10.081965 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 226631us
W20260709 13:12:10.217667 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:10.082530 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 231743us
I20260709 13:12:10.084046 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:10.083051 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 304358us
I20260709 13:12:10.224474 28050 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:10.225948 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:10.111913 28232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1594) took 1963 ms (client timeout 2199 ms). Trace:
I20260709 13:12:10.083542 30882 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:10.086828 30897 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.084600 30828 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:10.078464 30885 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:12:10.122313 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:09.897944 30734 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:10.278784 30885 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:09.781958 30780 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.279815 30885 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.280907 30885 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:09.895153 30902 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:09.794610 30819 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.283295 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:10.250720 28755 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:10.285221 30819 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:10.286008 30896 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:10.287194 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:10.287456 30940 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.001s
I20260709 13:12:10.189716 30942 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:10.289065 30942 consensus_queue.cc:1243] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:10.156041 30889 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.289228 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:10.195379 28230 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:10.289824 30889 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.195465 28084 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:12:10.184803 30779 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.090s	user 0.009s	sys 0.001s
I20260709 13:12:10.292399 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1608) took 1828 ms. Trace:
I20260709 13:12:10.304133 28235 rpcz_store.cc:276] 0709 13:12:08.463677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.463969 (+   292us) service_pool.cc:224] Handling call
0709 13:12:10.074939 (+1610970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:10.216993 (+142054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.292222 (+ 75229us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400306950. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:10.292346 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3703975296}
I20260709 13:12:10.222923 28153 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
W20260709 13:12:10.226917 28232 rpcz_store.cc:269] 0709 13:12:08.113767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.145232 (+ 31465us) service_pool.cc:224] Handling call
0709 13:12:09.986735 (+1841503us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400306954. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:10.075870 (+ 89135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3440888704}
W20260709 13:12:10.229329 28199 connection.cc:256] RPC call timeout handler was delayed by -1.6127176959998906s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
I20260709 13:12:10.253288 28077 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:10.233157 30882 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:10.254237 30911 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:10.254051 27982 catalog_manager.cc:5697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 4 to 6, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 6 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:10.255272 30932 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:12:10.257918 30817 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:10.259708 30930 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.096s	user 0.003s	sys 0.000s
I20260709 13:12:10.272509 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 475) took 1071 ms. Trace:
I20260709 13:12:10.273891 30830 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:10.276849 30905 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.278344 30808 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:10.278491 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:10.283207 30485 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:10.185575 30543 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.076447 30733 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:10.293056 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.294943 28227 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
W20260709 13:12:10.216038 30876 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 1.063s	user 0.024s	sys 0.032s
I20260709 13:12:10.298242 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:10.299039 30888 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:10.316431 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 90582.9us
W20260709 13:12:10.343434 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.385620 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:10.387107 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.233147 30668 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:10.395540 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 99577.9us
I20260709 13:12:10.405560 27982 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 3 to 4, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 4 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:10.399389 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 549294us
I20260709 13:12:10.439437 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:10.399586 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 436) took 1021 ms. Trace:
I20260709 13:12:10.394186 28086 rpcz_store.cc:276] 0709 13:12:09.201268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.201462 (+   194us) service_pool.cc:224] Handling call
0709 13:12:09.201908 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.754664 (+552756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.754697 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:09.755234 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:12:10.061130 (+305896us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:10.061197 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.061315 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:10.272211 (+210896us) raft_consensus.cc:1685] finished
0709 13:12:10.272235 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.272435 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":585630,"raft.queue_time_us":556662,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":305414,"threads_started":1,"wal-append.queue_time_us":477607}
I20260709 13:12:10.233705 30766 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:10.413053 28152 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.029s	sys 0.061s
W20260709 13:12:10.440887 28152 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.029s	sys 0.062s
I20260709 13:12:10.320081 30882 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:10.315390 30884 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:10.428478 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1610) took 1918 ms. Trace:
I20260709 13:12:10.442019 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:10.318604 30911 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:10.393455 30734 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:12:10.431159 30876 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 1.279s	user 0.024s	sys 0.032s
I20260709 13:12:10.436901 28233 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:10.443497 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:10.444957 28764 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:10.445526 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 437) took 1066 ms. Trace:
I20260709 13:12:10.446046 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:10.442324 28229 rpcz_store.cc:276] 0709 13:12:08.509931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.510448 (+   517us) service_pool.cc:224] Handling call
0709 13:12:08.510938 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:08.511229 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:08.614853 (+103624us) log.cc:844] Serialized 14 byte log entry
0709 13:12:09.686709 (+1071856us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b18003d4680. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a 00007f59ce681a2a
0709 13:12:10.397043 (+710334us) log.cc:844] Serialized 14 byte log entry
0709 13:12:10.427991 (+ 30948us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:10.428069 (+    78us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:10.428142 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.428238 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.428401 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166205,"prepare.queue_time_us":103316,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":32001,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":17975,"spinlock_wait_cycles":23132032,"thread_start_us":378807,"threads_started":2,"wal-append.queue_time_us":1468305}
I20260709 13:12:10.448164 30734 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:10.454506 30734 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.455720 30734 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.444790 30884 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:10.461720 30884 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:10.463614 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:10.464570 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:10.465102 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:10.465497 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:10.466593 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:10.466785 30914 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.154s
I20260709 13:12:10.467664 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:10.468858 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:10.465184 28081 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:10.451318 30668 raft_consensus.cc:2764] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:12:10.435297 30733 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.440506 28158 rpcz_store.cc:276] 0709 13:12:09.378468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.574578 (+196110us) service_pool.cc:224] Handling call
0709 13:12:09.851195 (+276617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:10.291106 (+439911us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b480024c258. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:12:10.305651 (+ 14545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:10.305698 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:10.305748 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:10.305791 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.305881 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.399346 (+ 93465us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54000a32d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:10.399510 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":309392,"raft.queue_time_us":342103,"raft.run_cpu_time_us":2241,"raft.run_wall_time_us":88041,"spinlock_wait_cycles":674230016,"thread_start_us":144591,"threads_started":1}
I20260709 13:12:10.474515 30733 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:10.458814 30735 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:10.475111 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:10.475335 30735 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:10.475754 28158 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:10.448819 30911 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:10.477234 30876 tablet_copy_client.cc:323] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
W20260709 13:12:10.479094 30809 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.809s	user 0.001s	sys 0.021s
I20260709 13:12:10.479564 30809 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.459594 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:10.450237 27982 catalog_manager.cc:5697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:10.482201 30543 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:10.483193 27982 catalog_manager.cc:5697] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 4, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 4 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:10.485596 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 439) took 1106 ms. Trace:
I20260709 13:12:10.486086 28154 rpcz_store.cc:276] 0709 13:12:09.379327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.612572 (+233245us) service_pool.cc:224] Handling call
0709 13:12:09.612931 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.613326 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.613359 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:09.613878 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:12:09.805128 (+191250us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:09.805168 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.805245 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:10.485363 (+680118us) raft_consensus.cc:1685] finished
0709 13:12:10.485387 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.485535 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2720,"raft.queue_time_us":1196,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":30531,"thread_start_us":190865,"threads_started":1,"wal-append.queue_time_us":276449}
I20260709 13:12:10.487413 28081 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:10.488186 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:10.488857 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:10.489348 28078 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:10.461279 30888 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:10.465178 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:10.492688 28154 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:10.489203 30905 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:10.458261 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:10.458210 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:10.466423 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:10.502954 28161 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
W20260709 13:12:10.456984 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 9e12858e91dd456a9f422bca6d2d6ac3: Illegal state: Replica 611ec64ec58244fe82058d6130e4e6f3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:10.466518 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:10.468928 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:10.420070 30485 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.497552 28157 raft_consensus.cc:2376] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8. Re-sending same reply.
I20260709 13:12:10.466993 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:10.506850 28153 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 4: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:10.500067 28755 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:10.451275 30814 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:10.509773 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:10.215454 30891 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.501000 30766 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:12:10.513149 27982 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 6, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 6 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:10.514768 28084 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:12:10.537453 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:10.515516 30888 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:10.522375 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 64239.7us
I20260709 13:12:10.540006 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:10.523954 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1905) took 1107 ms. Trace:
I20260709 13:12:10.525498 30817 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:10.501300 30668 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:10.476351 30735 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.530954 30485 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:10.550547 27981 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 189) took 8503 ms. Trace:
I20260709 13:12:10.553686 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:10.555930 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:10.558341 27982 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:10.561337 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.541102 28231 rpcz_store.cc:276] 0709 13:12:09.416391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.433106 (+ 16715us) service_pool.cc:224] Handling call
0709 13:12:09.467104 (+ 33998us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:09.522119 (+ 55015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.522162 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:09.526910 (+  4748us) log.cc:844] Serialized 52 byte log entry
0709 13:12:09.687578 (+160668us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:09.687619 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.687685 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:10.258797 (+571112us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:10.523620 (+264823us) raft_consensus.cc:1685] finished
0709 13:12:10.523645 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.523888 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29000,"raft.queue_time_us":54606,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":14463872,"thread_start_us":110779,"threads_started":1,"wal-append.queue_time_us":241356}
I20260709 13:12:10.530313 30882 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:10.561121 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 254377us
I20260709 13:12:10.538885 30791 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:10.486100 30734 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:10.549629 30828 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:12:10.590282 28199 connection.cc:256] RPC call timeout handler was delayed by -1.4966176959999302s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
I20260709 13:12:10.540751 30814 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:10.599051 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:10.544310 30947 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:10.558871 30659 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:10.621466 30889 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:10.622884 30884 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.623131 30882 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:10.595897 30898 consensus_meta.cc:287] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.077s	user 0.012s	sys 0.008s
I20260709 13:12:10.581221 30668 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:10.570719 30837 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
W20260709 13:12:10.580685 30731 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.841s	user 0.011s	sys 0.006s
I20260709 13:12:10.583302 27982 catalog_manager.cc:5697] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 4 to 5, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 5 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:10.596655 30768 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.906s	user 0.031s	sys 0.021s
I20260709 13:12:10.579066 30206 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:10.634526 30892 consensus_meta.cc:287] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.914s	user 0.004s	sys 0.029s
I20260709 13:12:10.634784 30828 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:12:10.628331 30752 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.924s	user 0.012s	sys 0.008s
W20260709 13:12:10.625047 28199 leader_election.cc:336] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.879s (SENT)
W20260709 13:12:10.626338 30810 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.871s	user 0.010s	sys 0.001s
I20260709 13:12:10.614825 30819 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.580570 30838 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:10.643055 30837 consensus_queue.cc:1243] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:10.643778 30768 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:10.670229 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 357054us
I20260709 13:12:10.638949 30898 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:10.688482 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.551119 27981 rpcz_store.cc:276] 0709 13:12:02.041121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:02.042103 (+   982us) service_pool.cc:224] Handling call
0709 13:12:10.536834 (+8494731us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:05.005875 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:05.073295 (+ 67420us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:05.073333 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:12:05.073362 (+    29us) tablet.cc:662] Decoding operations
0709 13:12:05.074538 (+  1176us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:05.074603 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:05.074631 (+    28us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:12:05.075667 (+  1036us) tablet.cc:701] Row locks acquired
0709 13:12:05.075686 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:12:05.075864 (+   178us) write_op.cc:270] Start()
0709 13:12:05.075972 (+   108us) write_op.cc:276] Timestamp: P: 1783602725075808 usec, L: 0
0709 13:12:05.075994 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:12:05.089420 (+ 13426us) log.cc:844] Serialized 4367 byte log entry
0709 13:12:06.236360 (+1146940us) op_driver.cc:464] REPLICATION: finished
0709 13:12:06.470619 (+234259us) write_op.cc:301] APPLY: starting
0709 13:12:06.470739 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:06.502705 (+ 31966us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67432a 00007f59ce5f9f14 00007f59ce5f9ec3 00007f59ce5f9e38 00007f59ce5f9df3 00007f59ce5f90dc 00007f59ce622e16 00007f59ce60acbb 00007f59ce5f539d 00007f59ce5f8d75
0709 13:12:06.504896 (+  2191us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:06.504942 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:06.633390 (+128448us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:06.665896 (+ 32506us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:12:06.665985 (+    89us) write_op.cc:312] APPLY: finished
0709 13:12:06.666586 (+   601us) log.cc:844] Serialized 83 byte log entry
0709 13:12:06.844564 (+177978us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:06.959541 (+114977us) write_op.cc:454] Released schema lock
0709 13:12:06.959611 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:06.960096 (+   485us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:06.960139 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":290610,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":224810,"apply.run_cpu_time_us":26393,"apply.run_wall_time_us":489503,"num_ops":11,"prepare.queue_time_us":1053417,"prepare.run_cpu_time_us":74928,"prepare.run_wall_time_us":1256185,"raft.queue_time_us":24856,"raft.run_cpu_time_us":33819,"raft.run_wall_time_us":1217077,"replication_time_us":1160101,"spinlock_wait_cycles":2185067904,"thread_start_us":3364530,"threads_started":5,"wal-append.queue_time_us":407946,"wal-append.run_cpu_time_us":14109,"wal-append.run_wall_time_us":2037163}]]}
I20260709 13:12:10.705859 28154 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:10.708441 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 91987.8us
W20260709 13:12:10.708961 28199 connection.cc:256] RPC call timeout handler was delayed by -1.4827243479999197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2801s for a response
I20260709 13:12:10.642445 30731 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:10.713516 28199 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 2.075s (SENT)
I20260709 13:12:10.714634 27982 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 5, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 5 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:10.718618 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.648918 30892 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.645081 30206 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.712113 28198 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:10.727841 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:10.728520 28229 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
W20260709 13:12:10.738818 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 139826us
I20260709 13:12:10.643849 30888 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.666627 30810 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.708202 30809 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:10.740796 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:10.741863 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:10.670198 30819 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:10.743489 28199 connection.cc:256] RPC call timeout handler was delayed by -1.3139176959998622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 13:12:10.667097 30814 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.751971 27982 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:10.736354 30882 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:10.765189 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:10.765909 28227 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:10.765911 28158 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:10.781376 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:10.782536 27982 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 5, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 5 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:10.750171 30838 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:10.711469 30668 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.766881 28756 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:12:10.806571 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:10.802642 28127 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.003s	sys 0.009s
W20260709 13:12:10.813203 28127 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.003s	sys 0.009s
W20260709 13:12:10.813946 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.803104 27982 catalog_manager.cc:5697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 3, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 3 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:10.814810 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:10.815629 28234 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.815584 27982 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 3 to 4, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 4 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:10.817552 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:10.814013 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 688025us
I20260709 13:12:10.819988 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:10.861093 30953 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.002s
I20260709 13:12:10.861691 27982 catalog_manager.cc:5697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 2, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 2 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:10.811048 30882 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:10.805047 30962 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:10.818065 30838 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:12:10.815665 30884 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:10.898646 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:10.866539 30206 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:10.867872 30819 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:10.868152 30949 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 13:12:10.868762 30960 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:12:10.870345 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 441) took 1261 ms. Trace:
I20260709 13:12:10.804231 30885 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:10.881145 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.889519 30817 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.869144 28757 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:10.865736 30810 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:10.868975 30902 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:10.890115 30959 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:12:10.905123 28159 rpcz_store.cc:276] 0709 13:12:09.609415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.648377 (+ 38962us) service_pool.cc:224] Handling call
0709 13:12:09.654894 (+  6517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.670047 (+ 15153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.670080 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:09.678182 (+  8102us) log.cc:844] Serialized 26 byte log entry
0709 13:12:09.967633 (+289451us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:09.967707 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:09.967859 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:10.867840 (+899981us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b24001b8c18. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4109c4 00007f59cb40eacb 00007f59cb5c9dc9 00007f59cb5c9a55 00007f59cb5c8fb0 00007f59ce23f488 00007f59ce23fbe8 00007f59ce217398 00007f59ce21276a 00007f59ce211bb6
0709 13:12:10.870079 (+  2239us) raft_consensus.cc:1685] finished
0709 13:12:10.870098 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.870294 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14657,"raft.queue_time_us":14858,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":4112,"spinlock_wait_cycles":68420864,"thread_start_us":288911,"threads_started":1,"wal-append.queue_time_us":493125}
W20260709 13:12:10.909857 30950 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:12:10.885740 30909 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:12:10.895341 30958 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 13:12:10.910602 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.895462 27982 catalog_manager.cc:5697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:10.898386 30914 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:12:10.911613 30748 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:10.915328 30819 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:10.889892 30837 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:10.917723 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 484) took 1301 ms. Trace:
I20260709 13:12:10.911677 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:10.901710 30911 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:10.900368 30485 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:10.906960 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:10.963362 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:10.962903 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:10.951979 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:10.895998 30791 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:12:10.966521 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:10.967293 28235 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:10.954221 28232 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:10.967595 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:10.957377 27982 catalog_manager.cc:5697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 3 to 4, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 4 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:10.959510 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:10.955092 28730 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:10.969458 28084 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:10.962852 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:10.952317 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:10.963826 28161 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:10.970713 28080 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:10.970809 28159 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
I20260709 13:12:10.964346 28231 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:10.967679 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:10.956719 28085 rpcz_store.cc:276] 0709 13:12:09.616507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.616639 (+   132us) service_pool.cc:224] Handling call
0709 13:12:09.617099 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:09.770723 (+153624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.770773 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:09.783997 (+ 13224us) log.cc:844] Serialized 52 byte log entry
0709 13:12:10.537149 (+753152us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:10.537193 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.537266 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:10.917479 (+380213us) raft_consensus.cc:1685] finished
0709 13:12:10.917502 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.917648 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153067,"raft.queue_time_us":154493,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":377082,"threads_started":1,"wal-append.queue_time_us":699924}
I20260709 13:12:10.935585 28234 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:10.972344 28079 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:10.972909 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:10.947122 30898 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:10.948812 30909 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:10.976693 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:10.956800 30819 consensus_queue.cc:1243] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:12:10.975725 27982 catalog_manager.cc:5697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 5 to 6, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 6 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:10.979454 28081 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:12:10.922106 30885 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.980254 28228 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:10.981086 28082 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:10.989068 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:10.991571 28230 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: [5.3-7.5]
I20260709 13:12:10.978855 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:11.001571 28079 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:10.966503 30947 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:10.943444 30817 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.946877 30902 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:10.976923 30812 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:10.974057 30892 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:11.023736 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.959888 30915 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:12:11.022207 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:11.070181 28077 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
W20260709 13:12:11.034104 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:10.969506 30485 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.019742 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:10.960055 30746 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:12:11.035562 28227 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:11.029050 27982 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 5, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 5 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:11.038015 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet b99bb21dbacb4d63839f1403ef9b08b9 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:12:11.038872 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:10.965860 30882 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.021916 30947 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.045730 30911 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:10.985898 30837 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.053823 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:11.057756 30731 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:10.449784 28152 rpcz_store.cc:276] 0709 13:12:09.378725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.593391 (+214666us) service_pool.cc:224] Handling call
0709 13:12:09.593836 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:09.596529 (+  2693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:09.596565 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:09.597189 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 13:12:09.780843 (+183654us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:10.441293 (+660450us) log.cc:844] Serialized 14 byte log entry
0709 13:12:10.445078 (+  3785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:10.445192 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:10.445250 (+    58us) raft_consensus.cc:1685] finished
0709 13:12:10.445271 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:10.445447 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3497,"raft.queue_time_us":9653,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":204,"thread_start_us":816233,"threads_started":2,"wal-append.queue_time_us":297694}
I20260709 13:12:11.103420 28152 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:11.059257 30885 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:11.064774 30908 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:11.066741 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:11.105274 30908 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:11.105553 28086 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:11.066371 30909 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.108294 28153 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:11.065977 28231 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:11.067471 28079 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:11.067911 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:11.109941 28085 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:11.110352 28155 raft_consensus.cc:2410] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:12:11.074608 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.076613 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:11.034451 28084 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:11.112339 27982 catalog_manager.cc:5697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:11.090907 28234 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 3: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:11.113441 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 137261us
W20260709 13:12:11.038918 30964 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:12:11.074856 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 202989us
I20260709 13:12:11.113763 27982 catalog_manager.cc:5697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:11.119691 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:11.101043 30897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.019s	sys 0.013s
I20260709 13:12:11.115123 28230 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:11.120841 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:11.130203 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:11.114248 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:11.131906 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:11.137792 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:11.137372 27982 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:11.138396 28234 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:11.138989 28086 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:11.140323 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:11.099088 30973 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.143241 28050 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:11.129287 30897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.020s	sys 0.013s
I20260709 13:12:11.147763 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:11.129786 30888 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:11.149667 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:11.152266 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:11.129765 30946 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:11.153638 28228 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:11.153954 28228 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:11.141683 30838 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:11.154639 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:11.155654 28234 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:11.129050 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet b99bb21dbacb4d63839f1403ef9b08b9 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-b99bb21dbacb4d63839f1403ef9b08b9
I20260709 13:12:11.154443 28228 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.171401 28228 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:11.149374 30976 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:11.137732 30914 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:12:11.148978 30543 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:11.153417 30888 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:11.189955 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:11.198978 28762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 749) took 1388 ms. Trace:
I20260709 13:12:11.199431 28762 rpcz_store.cc:276] 0709 13:12:09.811031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:09.812697 (+  1666us) service_pool.cc:224] Handling call
0709 13:12:10.121750 (+309053us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:10.480869 (+359119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:10.480896 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:10.480935 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:10.480963 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:11.198759 (+717796us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400654354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:11.198802 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:11.198924 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825355136}
I20260709 13:12:11.170262 30889 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:11.214001 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:11.171491 30735 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:11.215581 28231 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:11.156052 30791 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:12:11.183478 30976 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.203408 30888 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:11.216281 28228 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:11.218994 30971 raft_consensus.cc:2764] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:12:11.221796 30837 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:11.226539 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:11.231362 28096 tablet_copy_source_session.cc:215] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:11.214267 30974 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:11.249297 30485 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:11.250718 30734 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:11.251714 30828 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:11.251506 30735 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:12:11.251273 30975 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:12:11.220094 30976 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:11.261361 30978 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:12:11.227591 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:11.165122 30946 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.148275 30973 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.287386 30734 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:11.287886 30734 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:11.293923 28152 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:11.297905 30902 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:11.298738 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.280201 30817 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:11.288766 30911 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.310596 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:11.106530 30908 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.318436 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:11.321897 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:11.322924 28235 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
W20260709 13:12:11.324965 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:11.325883 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.324824 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:11.330487 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:11.327769 28157 raft_consensus.cc:2955] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:12:11.334512 28154 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:11.309197 30734 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.336860 30734 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:11.344669 30206 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:11.345232 30206 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:11.346433 30206 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.349360 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:11.350165 28153 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:11.359761 28158 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:11.423040 28229 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:11.431854 28084 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:12:11.432439 28084 pending_rounds.cc:85] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 6
I20260709 13:12:11.432721 28084 pending_rounds.cc:107] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:12:11.433032 28084 raft_consensus.cc:2889] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:11.434783 28079 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:11.432430 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:11.436988 28154 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:12:11.319511 30889 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:11.318292 30911 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.332340 30973 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:11.333847 30946 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:11.447603 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:11.450752 28081 raft_consensus.cc:2376] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7. Re-sending same reply.
I20260709 13:12:11.448717 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:11.448403 30828 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.426366 30543 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:11.448115 30884 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.460640 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:11.461659 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:11.463027 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:11.461692 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:11.465032 28081 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:11.462327 28228 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:11.469946 30885 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:12:11.489183 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:11.490526 30915 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:11.491946 30748 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:11.492340 30748 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:11.492431 30888 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.493495 30748 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:11.496848 30972 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.326s	user 0.004s	sys 0.001s
I20260709 13:12:11.495625 28228 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:11.465343 30884 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.504194 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:11.505710 30884 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:11.507409 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:11.508677 28233 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:11.509816 30888 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:11.515046 30976 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:11.516358 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.517024 30911 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:11.458684 30828 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.519937 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:11.520602 28233 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:11.521677 30828 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:11.522868 30930 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:12:11.523861 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:11.525213 30976 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:11.525911 30976 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:11.526496 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:11.528247 28081 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:12:11.529628 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:11.537984 28233 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:11.539248 28728 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:11.540409 28233 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:11.471818 30889 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:11.555079 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.546072 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:11.541661 30990 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:11.543569 30986 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:11.565925 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:11.577283 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:11.585459 30895 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:11.585917 30895 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:11.588311 28233 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:11.556236 30987 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:12:11.589042 30804 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.589587 30804 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.594537 30804 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:11.596963 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:11.586833 30895 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:11.562990 30892 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:11.598805 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1609) took 1138 ms. Trace:
W20260709 13:12:11.601341 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 132714us
I20260709 13:12:11.601716 28077 rpcz_store.cc:276] 0709 13:12:10.460905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.463317 (+  2412us) service_pool.cc:224] Handling call
0709 13:12:11.598630 (+1135313us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54001ae0d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:11.598749 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393307520}
I20260709 13:12:11.614559 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:11.650265 28153 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:11.651741 28153 pending_rounds.cc:85] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 3
I20260709 13:12:11.652125 28153 pending_rounds.cc:107] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:12:11.652612 28153 raft_consensus.cc:2889] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:11.654786 30892 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:11.655989 30892 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.656430 30485 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:11.681578 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:11.682015 28158 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:11.684844 28157 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:11.684566 28158 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:11.686424 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:11.686776 28158 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.689671 28158 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:11.692524 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:11.693821 30955 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:11.696662 31002 raft_consensus.cc:2764] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:12:11.698971 28158 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:12:11.699438 28158 pending_rounds.cc:85] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 4
I20260709 13:12:11.699563 30876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99bb21dbacb4d63839f1403ef9b08b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:11.699692 28158 pending_rounds.cc:107] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:12:11.700055 28158 raft_consensus.cc:2889] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:11.659101 30984 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:12:11.575373 30543 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.715025 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:11.648566 30735 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:11.701027 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:11.702775 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:11.704663 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:11.706326 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:11.707870 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.719914 28158 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:11.720860 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:11.721236 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:11.721841 28157 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:11.721788 28079 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:11.737391 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:11.739018 28157 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:11.688612 30809 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:11.747380 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46408.8us
I20260709 13:12:11.748190 28158 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:11.743577 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:11.744421 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:11.715744 30986 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:11.717610 30766 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:11.766582 31008 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:12:11.769075 30968 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:12:11.751065 30987 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:11.746642 30809 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:11.755836 30986 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.756366 31002 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:11.790323 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:11.792351 28050 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:11.809247 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:11.824882 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:11.740200 30735 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.828141 28227 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:11.758499 30955 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:12:11.831292 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:11.791512 30809 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:11.835753 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:11.826634 30614 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:11.842646 30614 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.843053 30614 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:11.782414 30766 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.785599 30987 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:11.841372 28081 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:11.826231 30986 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:11.830300 30731 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:11.822713 30885 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:12:11.791760 30911 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:12:11.834070 30896 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:11.849689 30976 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:12:11.865227 30731 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:11.870992 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:11.825243 30828 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:12:11.863610 30987 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:11.861089 30810 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:11.891836 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:11.872257 30896 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:11.895840 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:11.871905 30734 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:12:11.884112 30731 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:11.900264 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 71428.1us
I20260709 13:12:11.870465 30974 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:12:11.907143 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:11.873557 30982 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 13:12:11.914948 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:11.915663 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:11.901017 30896 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:11.918277 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:11.896894 31012 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:11.920460 28081 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:11.930231 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:11.931459 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:11.934949 28199 leader_election.cc:336] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.675s (SENT)
I20260709 13:12:11.894876 30810 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.912658 30990 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:11.904971 31002 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:11.937345 28199 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.569s (SENT)
I20260709 13:12:11.949779 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:11.906827 31017 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:11.913151 30485 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:11.902863 30968 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:12:11.921306 30914 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:11.940989 30902 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:11.911512 30942 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:12:11.922374 31012 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:12:11.956872 28199 leader_election.cc:336] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.661s (SENT)
I20260709 13:12:11.961264 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:11.959041 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:11.939013 30768 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:12:11.967000 30768 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:11.967417 30768 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:11.968580 31018 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:11.969167 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:11.971647 30485 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:11.972647 30485 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.962666 30809 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:11.942121 30808 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:11.973235 30748 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:11.986754 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:11.995503 28199 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 4.023s (SENT) [suppressed 54 similar messages]
I20260709 13:12:11.958053 30990 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:11.961913 30837 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:12.019631 30766 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:12.020308 30735 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:12.022145 31002 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:11.968417 30768 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.012598 30908 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:11.962882 31012 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:12.025410 30768 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:11.993698 30974 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:11.961529 30828 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:12.029458 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 98075.1us
I20260709 13:12:11.998631 31018 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:12.039569 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:12.042292 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:12.045707 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:12.047967 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:11.998903 30808 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.048895 28156 raft_consensus.cc:2376] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8. Re-sending same reply.
I20260709 13:12:12.014544 30987 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.052768 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:12.047175 30837 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:12.056555 30973 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:12:12.058903 30837 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:12.059821 30837 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:12.060037 30973 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:12.050930 31011 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:12:12.061160 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.063169 28160 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:12:12.063766 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:12.046751 30942 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:12.053171 31013 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:12:12.070663 30986 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.073304 30206 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:12.073757 30973 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:12.077857 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:12.080981 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:12.082914 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:12.083935 28156 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 3: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:12.085896 31014 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:12:12.087702 28153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:53998
I20260709 13:12:12.088328 28153 raft_consensus.cc:606] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:12.088639 28153 raft_consensus.cc:614] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:12.091168 28084 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:12:12.094905 31024 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.095568 31024 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.054721 30614 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.099179 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:12.099912 28083 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:12.100835 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:12.101533 28083 raft_consensus.cc:2368] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:12.101348 30614 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:12.102442 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:12.103297 28083 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:12.107306 28763 raft_consensus.cc:2368] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7.
I20260709 13:12:12.112305 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:12.042055 30974 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.054773 30766 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:12:12.121130 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:12.122073 28227 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:12.052357 30828 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.119428 28077 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:12.128334 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:12.130316 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:12.127786 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:12.134913 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:12.135728 28083 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:12.137763 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:12.138691 28077 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:12:12.140409 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:12.147814 28156 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:12.148479 28077 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:12.127513 30614 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:12.161125 28234 raft_consensus.cc:2410] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:12:12.143530 30810 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:12.166725 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:12.167202 28083 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:12.168187 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:12.170102 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:12.170548 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:12.170907 31019 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:12.171531 31019 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:12.171684 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:12.171906 30974 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:12.172323 28157 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:12.172446 31019 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.176076 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.177096 28227 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:12.151937 31024 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:12.154219 30902 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:12.153607 30973 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:12.143056 30731 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.164281 30955 raft_consensus.cc:2764] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:12:12.191933 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:12.180836 31023 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:12:12.196445 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.197913 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.199282 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.199889 30986 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:12.200079 31024 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:12.200696 31024 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.201658 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.202912 31024 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:12.155987 30614 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.200394 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.168704 30809 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:12.106036 30990 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.182896 28083 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:12:12.210594 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:12.217878 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:12.221768 28227 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:12:12.223522 30614 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:12.224463 30977 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.979s	user 0.003s	sys 0.002s
I20260709 13:12:12.224666 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.225296 28235 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:12.225900 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 497) took 1271 ms. Trace:
I20260709 13:12:12.226487 28078 rpcz_store.cc:276] 0709 13:12:10.954082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.954326 (+   244us) service_pool.cc:224] Handling call
0709 13:12:10.954778 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:10.955065 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:10.955102 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:10.955665 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:12:11.077540 (+121875us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:11.077597 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:11.077722 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:12.179488 (+1101766us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:12.225394 (+ 45906us) raft_consensus.cc:1685] finished
0709 13:12:12.225427 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:12.225818 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9591552,"thread_start_us":97722,"threads_started":1,"wal-append.queue_time_us":224232}
I20260709 13:12:12.228268 30614 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:12.229220 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:12.229831 28078 raft_consensus.cc:2376] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3. Re-sending same reply.
I20260709 13:12:12.231199 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:12.232156 28078 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:12.228909 30614 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.233397 30731 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:12.234156 30614 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:12.184674 28157 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:12.239984 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:12.241030 28050 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:12.241627 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:12.242278 28083 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
W20260709 13:12:12.244884 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 249405us
W20260709 13:12:12.246035 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0819460739999158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4958s for a response
I20260709 13:12:12.247503 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1621) took 1181 ms. Trace:
I20260709 13:12:12.247869 28078 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:12.249392 28052 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.247880 28085 rpcz_store.cc:276] 0709 13:12:11.066499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.095398 (+ 28899us) service_pool.cc:224] Handling call
0709 13:12:11.095815 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:11.110351 (+ 14536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.247274 (+1136923us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b54000e29d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:12.247447 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32018176}
I20260709 13:12:12.251469 28085 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:12.252228 28227 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:12.256918 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.257704 28085 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:12.269095 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:12.270133 28085 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:12.277323 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:12.265019 31030 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:12.284474 30915 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:12.285030 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:12.285059 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.285768 30896 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.286250 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:12.287432 31015 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:12.287593 28157 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:12:12.288220 30896 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:12.289513 28079 raft_consensus.cc:2368] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4.
I20260709 13:12:12.195531 30902 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.305222 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:12.306512 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:12.308215 28155 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:12.256713 30614 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:12.316706 28085 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:12.256577 30804 raft_consensus.cc:2764] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
W20260709 13:12:12.300158 30963 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.439s	user 0.008s	sys 0.000s
I20260709 13:12:12.321636 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.313409 30748 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:12:12.290313 30888 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:12:12.338441 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 100517us
I20260709 13:12:12.335141 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:12.288805 31015 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:12:12.265137 30659 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:12:12.347754 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:12.348042 28085 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:12.279280 30206 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:12:12.350373 28762 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:12.356817 28086 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:12.361021 28086 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:12:12.358440 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.326534 30915 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:12.363394 28227 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:12.363338 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.335421 30614 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:12.318922 30908 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:12:12.369768 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:12.376596 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:12.367403 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:12.350934 31002 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:12.382295 31020 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:12:12.383121 31020 consensus_queue.cc:1243] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:12:12.386175 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.386899 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.387840 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.387226 28232 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:12.390962 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.391592 28228 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:12.391906 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:12.345353 30982 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:12:12.400254 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.400516 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:12.362920 30968 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:12.409833 28228 raft_consensus.cc:2376] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4. Re-sending same reply.
I20260709 13:12:12.402580 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:12.393252 30892 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:12.417402 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.415232 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:12.411213 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73300.2us
I20260709 13:12:12.418152 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.419137 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.419548 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.392772 28230 raft_consensus.cc:2376] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7. Re-sending same reply.
I20260709 13:12:12.368901 30828 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:12.430141 31016 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:12.421972 28083 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:12.421010 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:12.432655 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.423985 30837 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.438690 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.427287 28155 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:12:12.428808 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.440472 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:12.384541 30915 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:12.442196 28235 pending_rounds.cc:85] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 4
I20260709 13:12:12.442855 28235 pending_rounds.cc:107] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:12:12.443128 28235 raft_consensus.cc:2889] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:12.445305 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:12.451162 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.413023 30968 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:12.416944 31018 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:12.458011 28159 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:12:12.431571 30888 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:12.385170 30804 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:12:12.449198 30828 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:12.438301 30892 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:12.450073 30915 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:12.452816 30543 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:12:12.474709 28228 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:12.453416 30982 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:12.475749 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:12.453547 31017 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:12.455672 30968 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:12.489030 28228 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:12.477296 30915 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:12.462949 30947 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:12.491819 28233 raft_consensus.cc:2410] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:12.475823 30614 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:12.480955 31035 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:12:12.458498 30884 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:12:12.461615 30896 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:12:12.503440 28228 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.492844 30888 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:12.464221 31018 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.518157 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:12.521263 28228 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:12.494592 30828 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:12.490257 30982 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:12.502776 31017 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:12.525501 28083 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: [5.8-7.9]
W20260709 13:12:12.526618 28199 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.647s (SENT)
I20260709 13:12:12.538609 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.523525 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:12.527817 28085 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:12:12.541365 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:12.541325 28228 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:12:12.532338 28233 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:12:12.529028 30947 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:12:12.542680 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:12.543856 28199 leader_election.cc:400] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:12.544134 28199 leader_election.cc:403] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:12:12.545271 28228 pending_rounds.cc:85] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 3
I20260709 13:12:12.521646 30888 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:12.528386 30768 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:12.547906 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.555941 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2009) took 2085 ms. Trace:
I20260709 13:12:12.558537 28228 pending_rounds.cc:107] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:12:12.531633 30982 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:12.562047 31042 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.188s	user 0.003s	sys 0.000s
I20260709 13:12:12.565994 28228 raft_consensus.cc:2889] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:12.566846 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.548372 31002 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
W20260709 13:12:12.553785 31027 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.469s	user 0.002s	sys 0.000s
I20260709 13:12:12.567924 30885 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:12.568277 30837 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:12.545147 30892 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.544570 30908 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:12:12.570474 31012 consensus_meta.cc:287] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
I20260709 13:12:12.548146 31015 raft_consensus.cc:2764] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:12:12.581023 30828 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.540827 30902 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:12.585150 30987 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:12.588404 31018 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:12.564463 28160 rpcz_store.cc:276] 0709 13:12:10.471160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.471769 (+   609us) service_pool.cc:224] Handling call
0709 13:12:10.472146 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:12.060668 (+1588522us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b540019d5a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5eedc0 00007f59ce6791c0 00007f59ce67a231 00007f59ce676d8f 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c
0709 13:12:12.078161 (+ 17493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.555117 (+476956us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b540019d5a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5eedc0 00007f59ce67aa02 00007f59ce676dd6 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:12.555681 (+   564us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b54003935d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:12.555876 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681612416}
I20260709 13:12:12.564713 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:12.595897 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.664516 30914 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:12:12.664862 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69032.9us
I20260709 13:12:12.665122 30914 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:12.666149 30914 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.596221 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:12.667528 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 71367.3us
I20260709 13:12:12.668272 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:12.669055 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 104423us
I20260709 13:12:12.669781 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:12.613983 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 147666us
I20260709 13:12:12.548584 30947 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:12.564465 30888 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:12.608832 30748 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:12.650801 30909 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:12.661119 31017 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.671594 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 447) took 1766 ms. Trace:
I20260709 13:12:12.680249 28152 rpcz_store.cc:276] 0709 13:12:10.904940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.104330 (+199390us) service_pool.cc:224] Handling call
0709 13:12:11.107818 (+  3488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:11.294527 (+186709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.671366 (+1376839us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400051c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:12.671537 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417813376}
I20260709 13:12:12.672184 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:12.673147 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:12.673359 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.675135 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:12.682533 28082 raft_consensus.cc:2368] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:12.682963 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1971) took 1252 ms. Trace:
I20260709 13:12:12.687985 28160 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:12:12.688246 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:12.683314 28082 rpcz_store.cc:276] 0709 13:12:11.430483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.430605 (+   122us) service_pool.cc:224] Handling call
0709 13:12:12.672057 (+1241452us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552eee 0000000000375939 00007f59d35c61e0 00007f59d368f0c4 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:12.682909 (+ 10852us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3063168}
I20260709 13:12:12.690832 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.701311 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.701452 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:12.704320 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:12.705124 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 252854us
I20260709 13:12:12.732846 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:12.746263 28160 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:12.755859 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.757522 28762 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:12.626677 31012 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:12.682206 31018 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:12.678562 31015 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:12.678869 31002 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:12.682364 30986 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:12.782346 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69713.4us
I20260709 13:12:12.694144 30885 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:12.746964 31024 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:12.745914 30947 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:12.754051 30780 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:12.793566 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:12.809312 30837 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:12.698940 30920 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.112s
I20260709 13:12:12.745277 30882 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:12:12.828677 31016 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:12.845932 30920 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:12.846294 31016 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:12.754266 30748 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:12:12.683861 30968 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.846570 30920 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:12.759666 30908 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:12:12.806730 31015 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:12.859658 30920 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:12.845471 30837 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:12.820120 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:12.814872 30986 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:12.864940 30882 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:12.823386 30885 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:12.814359 30731 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:12:12.871763 30986 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:12.846706 31016 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:12.933455 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:12.884920 30885 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:12.861007 30837 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:12.905661 28757 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 8: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:12:12.873431 31062 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:12:12.878818 30882 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:12.863037 31015 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:13.003957 30882 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.005424 30882 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:13.007565 31012 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:13.005584 30968 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:13.034850 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:13.039562 31055 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
W20260709 13:12:13.041747 31066 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 13:12:13.042687 31056 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
W20260709 13:12:13.042822 31063 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.187s	user 0.002s	sys 0.000s
W20260709 13:12:13.065469 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 358647us
I20260709 13:12:13.012189 30987 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:12:13.080606 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:13.080010 30892 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 13:12:13.097306 28079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1983) took 1186 ms (client timeout 1572 ms). Trace:
I20260709 13:12:13.097469 28078 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:13.096750 30206 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:13.101270 30206 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:13.102218 30206 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.103490 30206 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:13.106182 30804 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:13.106817 30804 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:13.110018 31044 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.756s	user 0.002s	sys 0.001s
I20260709 13:12:13.110280 30804 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:13.078924 30837 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:13.099897 28079 rpcz_store.cc:269] 0709 13:12:11.911317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.948892 (+ 37575us) service_pool.cc:224] Handling call
0709 13:12:13.085169 (+1136277us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b54002972d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:13.097228 (+ 12059us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":744772352}
I20260709 13:12:13.072019 30809 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:12:13.099321 30543 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:13.068960 30982 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:13.169656 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 479549us
I20260709 13:12:13.145545 30543 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:13.171180 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:13.185825 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 497604us
I20260709 13:12:13.067212 31016 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.188220 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:13.190203 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:13.194149 30748 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:13.206701 30888 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.002s
W20260709 13:12:13.184495 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.252777 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:13.254781 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:13.255739 28232 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:13.258590 30888 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:13.272694 28763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35896 (request call id 748) took 2068 ms (client timeout 1676 ms). Trace:
W20260709 13:12:13.273247 28763 rpcz_store.cc:269] 0709 13:12:11.204655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.204810 (+   155us) service_pool.cc:224] Handling call
0709 13:12:12.060998 (+856188us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:13.272413 (+1211415us) spinlock_profiling.cc:243] Waited 9.81 ms on lock 0x7b54004371d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:13.272608 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39190272}
W20260709 13:12:13.259215 30915 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.004s
I20260709 13:12:13.276081 28230 pending_rounds.cc:85] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 2
I20260709 13:12:13.282091 28230 pending_rounds.cc:107] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:12:13.282490 28230 raft_consensus.cc:2889] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:13.286069 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 95959.4us
I20260709 13:12:13.286510 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 454) took 1547 ms. Trace:
I20260709 13:12:13.226038 30543 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.287016 28158 rpcz_store.cc:276] 0709 13:12:11.739693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.749170 (+  9477us) service_pool.cc:224] Handling call
0709 13:12:11.749574 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:11.874364 (+124790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:11.874394 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:11.893913 (+ 19519us) log.cc:844] Serialized 26 byte log entry
0709 13:12:12.030986 (+137073us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:12.031021 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.553483 (+522462us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b24001b8b88. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4109c4 00007f59cb40eacb 00007f59cb5c9dc9 00007f59cb5c9a55 00007f59cb5c909d 00007f59ce1fb10e 00007f59ce21288c 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221
0709 13:12:13.286186 (+732703us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400254c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:13.286242 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:13.286290 (+    48us) raft_consensus.cc:1685] finished
0709 13:12:13.286305 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.286449 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324913152,"thread_start_us":117973,"threads_started":1,"wal-append.queue_time_us":147863}
I20260709 13:12:13.292893 28228 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:13.265522 31018 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:13.283581 30915 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:12:13.299014 30876 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.887s	user 0.010s	sys 0.003s
I20260709 13:12:13.283422 30888 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:13.283162 30485 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.008s	sys 0.030s
I20260709 13:12:13.300639 30876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:56420 (request call id 741) took 4896 ms. Trace:
W20260709 13:12:13.301721 30485 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.009s	sys 0.030s
I20260709 13:12:13.301947 30876 rpcz_store.cc:276] 0709 13:12:08.420783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.426813 (+  6030us) service_pool.cc:224] Handling call
0709 13:12:09.126148 (+699335us) ts_tablet_manager.cc:934] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
0709 13:12:09.990147 (+863999us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59cb56d0ac 00007f59d43c7ab9 00007f59cb58969b 00007f59cb586caa 00007f59cb586bd2 00007f59d35eb131
0709 13:12:11.698088 (+1707941us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:12:12.355452 (+657364us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:12.363417 (+  7965us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:12:13.299604 (+936187us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:12:13.299992 (+   388us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:13.300404 (+   412us) ts_tablet_manager.cc:989] Replica registered.
0709 13:12:13.300542 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":26663,"spinlock_wait_cycles":119983488,"tablet-copy.queue_time_us":543575,"thread_start_us":1585037,"threads_started":2}
I20260709 13:12:13.304792 30876 tablet_copy_client.cc:806] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:13.305617 30876 tablet_copy_client.cc:670] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:13.300705 30915 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:13.307356 31018 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:13.289877 30909 consensus_meta.cc:287] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.590s	user 0.013s	sys 0.000s
I20260709 13:12:13.314419 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:13.310266 31072 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:13.316233 31073 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:13.318125 30915 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:13.318881 30915 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.325017 28161 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:12:13.325594 28161 pending_rounds.cc:85] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 4
I20260709 13:12:13.325876 28161 pending_rounds.cc:107] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:12:13.326148 28161 raft_consensus.cc:2889] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:13.329841 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:13.336300 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:13.345674 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:13.349864 28084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 1974) took 1902 ms (client timeout 1647 ms). Trace:
I20260709 13:12:13.326884 31030 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:13.350605 28084 rpcz_store.cc:269] 0709 13:12:11.447751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.448453 (+   702us) service_pool.cc:224] Handling call
0709 13:12:13.349609 (+1901156us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54000e0454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:13.349758 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1470431616}
I20260709 13:12:13.353031 28153 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:13.334208 30968 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:13.335861 30914 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:12:13.356063 30915 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:13.357210 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:13.334035 30768 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
W20260709 13:12:13.384948 28160 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.012s	sys 0.014s
I20260709 13:12:13.401969 28760 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:13.349838 31002 raft_consensus.cc:476] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 13:12:13.375703 31030 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:13.418503 31037 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.070s	user 0.004s	sys 0.000s
I20260709 13:12:13.421324 31016 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:13.441179 30808 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:13.394455 30968 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:13.447903 28160 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.012s	sys 0.014s
I20260709 13:12:13.460983 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 464) took 1220 ms. Trace:
W20260709 13:12:13.469009 28198 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.013s	sys 0.017s
W20260709 13:12:13.469738 28198 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.014s	sys 0.017s
W20260709 13:12:13.470204 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 700017us
W20260709 13:12:13.479926 28199 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.572s (SENT)
I20260709 13:12:13.482903 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:13.457459 31079 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:12:13.577003 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 178049us
I20260709 13:12:13.634514 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:13.650955 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:13.636907 30908 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:13.657783 31039 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.292s	user 0.016s	sys 0.010s
I20260709 13:12:13.657966 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1923) took 2337 ms. Trace:
I20260709 13:12:13.658330 28161 rpcz_store.cc:276] 0709 13:12:11.320253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.328470 (+  8217us) service_pool.cc:224] Handling call
0709 13:12:11.328956 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:13.326685 (+1997729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.657722 (+331037us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b5400237154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:13.657906 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4582151552}
W20260709 13:12:13.660869 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.661116 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 504) took 1942 ms. Trace:
I20260709 13:12:13.661587 28080 rpcz_store.cc:276] 0709 13:12:11.718925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.719132 (+   207us) service_pool.cc:224] Handling call
0709 13:12:11.719477 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:11.740772 (+ 21295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:11.740834 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:11.751895 (+ 11061us) log.cc:844] Serialized 26 byte log entry
0709 13:12:12.234482 (+482587us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b18003f2920. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:12:12.346592 (+112110us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:12.346629 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.346703 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:13.657146 (+1310443us) spinlock_profiling.cc:243] Waited 84.9 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:13.660854 (+  3708us) raft_consensus.cc:1685] finished
0709 13:12:13.660875 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.661021 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3767,"raft.queue_time_us":6473,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":10331,"spinlock_wait_cycles":198271232,"thread_start_us":111917,"threads_started":1,"wal-append.queue_time_us":613919}
W20260709 13:12:13.663038 28234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 486) took 2449 ms (client timeout 2172 ms). Trace:
W20260709 13:12:13.663439 28234 rpcz_store.cc:269] 0709 13:12:11.213620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.213735 (+   115us) service_pool.cc:224] Handling call
0709 13:12:13.662849 (+2449114us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b54002dafd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:13.662974 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175781632}
I20260709 13:12:13.663980 30895 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:13.665942 31041 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.315s	user 0.002s	sys 0.000s
I20260709 13:12:13.498998 30885 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.668654 30908 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:13.670099 31082 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:12:13.670368 30885 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:13.641137 30973 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:13.670872 30908 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:13.671236 30973 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.673642 30973 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:13.675877 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1946) took 1522 ms. Trace:
I20260709 13:12:13.676601 31084 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:13.677263 31084 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.676331 28155 rpcz_store.cc:276] 0709 13:12:12.153705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.440002 (+286297us) service_pool.cc:224] Handling call
0709 13:12:12.440339 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:12.440745 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:13.039234 (+598489us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b18003bce60. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce68a5e5 00007f59ce68a568
0709 13:12:13.292679 (+253445us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.476053 (+183374us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.477640 (+  1587us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:13.498420 (+ 20780us) log.cc:844] Serialized 26 byte log entry
0709 13:12:13.498585 (+   165us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:13.498616 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.498692 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:13.675628 (+176936us) raft_consensus.cc:1685] finished
0709 13:12:13.675654 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.675820 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63786,"prepare.queue_time_us":430412,"prepare.run_cpu_time_us":143,"prepare.run_wall_time_us":15188,"raft.queue_time_us":22896,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":80204,"spinlock_wait_cycles":9489792,"thread_start_us":412100,"threads_started":2,"wal-append.queue_time_us":275671}
I20260709 13:12:13.668978 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 458) took 1519 ms. Trace:
I20260709 13:12:13.564177 30986 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.686661 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 456) took 1925 ms. Trace:
I20260709 13:12:13.683271 28156 rpcz_store.cc:276] 0709 13:12:12.149982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.150110 (+   128us) service_pool.cc:224] Handling call
0709 13:12:12.165448 (+ 15338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:12.168234 (+  2786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:12.168264 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:12.185070 (+ 16806us) log.cc:844] Serialized 26 byte log entry
0709 13:12:12.282493 (+ 97423us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:12.282536 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:12.282606 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:13.665343 (+1382737us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:13.666866 (+  1523us) raft_consensus.cc:1685] finished
0709 13:12:13.666900 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.668895 (+  1995us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5101,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":4673,"spinlock_wait_cycles":233200512,"thread_start_us":43536,"threads_started":1,"wal-append.queue_time_us":68111}
I20260709 13:12:13.645076 30748 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:13.661192 30891 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.703877 30748 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.704133 30891 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.706807 30748 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:13.707540 30735 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:13.707939 30735 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:13.708896 30735 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.689594 28154 rpcz_store.cc:276] 0709 13:12:11.761101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:11.841269 (+ 80168us) service_pool.cc:224] Handling call
0709 13:12:11.868003 (+ 26734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:11.955840 (+ 87837us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:11.973890 (+ 18050us) log.cc:844] Serialized 14 byte log entry
0709 13:12:12.036755 (+ 62865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:12.036813 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:12.036858 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.686357 (+1649499us) spinlock_profiling.cc:243] Waited 87.6 ms on lock 0x7b540011b054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:13.686418 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.686588 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30254,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":2968,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":3639,"spinlock_wait_cycles":201504512,"thread_start_us":62422,"threads_started":1,"wal-append.queue_time_us":156553,"wal-append.run_cpu_time_us":1462,"wal-append.run_wall_time_us":1055661}
W20260709 13:12:13.702023 31015 consensus_meta.cc:287] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.001s
I20260709 13:12:13.702764 30986 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:13.703342 30968 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:13.700383 31030 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:13.714670 30891 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:13.746750 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2002) took 1229 ms. Trace:
W20260709 13:12:13.749192 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.747313 28077 rpcz_store.cc:276] 0709 13:12:12.517600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.518905 (+  1305us) service_pool.cc:224] Handling call
0709 13:12:12.519388 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:12.523462 (+  4074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:12.523509 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:12.527899 (+  4390us) log.cc:844] Serialized 26 byte log entry
0709 13:12:12.614853 (+ 86954us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:12.615313 (+   460us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.745980 (+1130667us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b180001bca0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:12:13.746211 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.746322 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:13.746381 (+    59us) raft_consensus.cc:1685] finished
0709 13:12:13.746480 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.746671 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18604,"prepare.queue_time_us":167,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":1541,"raft.queue_time_us":4384,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":4799,"spinlock_wait_cycles":26210560,"thread_start_us":84754,"threads_started":1,"wal-append.queue_time_us":316749}
I20260709 13:12:13.757639 28762 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:13.760900 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:13.765091 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:13.765091 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:13.722680 30902 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:13.774564 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.787570 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2025) took 1090 ms. Trace:
I20260709 13:12:13.776909 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:13.755033 30990 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:13.721306 31015 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.744971 30968 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.780242 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:13.777299 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:13.769856 30837 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.783270 31084 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:13.788349 31046 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.782807 30902 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:13.791126 31098 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.768712 30908 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
W20260709 13:12:13.842638 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.797159 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:13.814369 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:13.845628 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 80395.2us
I20260709 13:12:13.846262 31098 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.848878 30731 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:13.850221 30456 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:13.850358 30731 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:13.824625 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:13.822407 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 752) took 1256 ms. Trace:
I20260709 13:12:13.789486 28160 rpcz_store.cc:276] 0709 13:12:12.697201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.755418 (+ 58217us) service_pool.cc:224] Handling call
0709 13:12:12.755833 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:12.756112 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:13.458444 (+702332us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.787092 (+328648us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:13.787162 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:13.787225 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.787321 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.787489 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15349,"prepare.queue_time_us":395974,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":770,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":976853,"threads_started":2,"wal-append.queue_time_us":334643}
I20260709 13:12:13.784013 30734 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:12:13.793635 31036 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.363s	user 0.012s	sys 0.003s
I20260709 13:12:13.783742 30828 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.842023 28226 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:13.850711 30456 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:13.852891 28760 rpcz_store.cc:276] 0709 13:12:12.566246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.566488 (+   242us) service_pool.cc:224] Handling call
0709 13:12:12.566781 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:13.451332 (+884551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.822147 (+370815us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54006149d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:13.822344 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1907629824}
I20260709 13:12:13.853335 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:13.829231 30735 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:13.855777 28762 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 13:12:13.812139 31064 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.951s	user 0.013s	sys 0.000s
I20260709 13:12:13.857977 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:13.858389 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:13.861325 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:13.862927 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:13.837805 30206 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.864480 28082 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:13.865873 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:13.866554 28085 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:12:13.868861 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.874126 30746 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:13.869329 28156 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:12:13.862555 30734 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.841974 31046 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.871012 28079 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:12:13.836370 30990 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.840024 30837 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.873762 28077 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:13.873840 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:13.845228 31081 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.887750 30828 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.927591 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:13.895639 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:13.943907 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:13.874301 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:13.897382 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:13.848577 31098 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:13.900341 28084 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:13.869246 28160 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:12:13.899689 28086 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:13.962224 28160 pending_rounds.cc:85] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 4
I20260709 13:12:13.902614 28083 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:13.901240 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:13.962944 28154 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:13.902456 30955 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:13.898994 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:13.904524 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:13.905413 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:13.851343 31084 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:13.838320 30968 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.916018 28758 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:13.907330 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:13.906502 30746 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:13.872408 30456 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:13.925251 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:13.927367 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:13.962513 28160 pending_rounds.cc:107] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.5
I20260709 13:12:13.978221 28227 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:13.978499 28160 raft_consensus.cc:2917] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Aborting config change with OpId 4.5: Aborted: Op aborted by new leader
I20260709 13:12:13.967516 28156 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:13.980053 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:13.873085 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:13.963793 28760 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:12:13.906023 30206 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.955826 31002 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:12:14.041553 28234 raft_consensus.cc:2410] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:12:13.983541 28155 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:13.988476 28160 raft_consensus.cc:2889] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:13.981662 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:14.290778 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:43180 (request call id 13) took 3719 ms. Trace:
I20260709 13:12:15.014261 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:14.255815 30746 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:14.059451 31109 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:12:14.398766 31073 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:13.969466 31084 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.970181 30882 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:14.060184 30734 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:14.242631 31016 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.767s	user 0.006s	sys 0.004s
I20260709 13:12:14.243575 30543 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:14.400041 31002 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:14.099064 31103 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 1.079s
I20260709 13:12:13.990531 31081 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.971009 30955 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.093153 30909 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.129962 31084 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:14.529676 28201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.023s	sys 0.021s
W20260709 13:12:15.130899 28201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.174s	user 0.040s	sys 0.033s
I20260709 13:12:14.399416 28228 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:15.132283 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1670) took 1443 ms. Trace:
W20260709 13:12:14.101684 31097 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:12:15.022274 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2035) took 1126 ms. Trace:
I20260709 13:12:14.100960 30968 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.024112 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2039) took 1121 ms. Trace:
I20260709 13:12:14.291427 28160 rpcz_store.cc:276] 0709 13:12:10.533992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:10.766907 (+232915us) service_pool.cc:224] Handling call
0709 13:12:10.916986 (+150079us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b58000614d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5d30aa 00007f59ce655447 00007f59ce67270a 00007f59ce67376a 00007f59ce67291b 00007f59ce67e03c 00007f59ce67d6b3 00007f59d363dd29 00007f59ccbbc1d4 00007f59ccbbc162
0709 13:12:11.340465 (+423479us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b58007ce5d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5d30aa 00007f59ce655447 00007f59ce67270a 00007f59ce67376a 00007f59ce67291b 00007f59ce67e03c 00007f59ce67d6b3 00007f59d363dd29 00007f59ccbbc1d4 00007f59ccbbc162
0709 13:12:11.430085 (+ 89620us) log.cc:844] Serialized 506 byte log entry
0709 13:12:12.221361 (+791276us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b540019f8a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f0062 00007f59ce5d0eaa 00007f59ce5d7d53 00007f59ce5d7cd9 00007f59ce5d7c69 00007f59ce5d7c31 00007f59ce5d6dfd 000000000038f354 000000000038f199 00007f59cb58a475
0709 13:12:12.653885 (+432524us) spinlock_profiling.cc:243] Waited 7.1 ms on lock 0x7b540019f8a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f0062 00007f59ce5d0eaa 00007f59ce5d7d53 00007f59ce5d7cd9 00007f59ce5d7c69 00007f59ce5d7c31 00007f59ce5d6dfd 000000000038f354 000000000038f199 00007f59cb58a475
0709 13:12:13.144434 (+490549us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b540019f8a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f0062 00007f59ce5d0eaa 00007f59ce5d7d53 00007f59ce5d7cd9 00007f59ce5d7c69 00007f59ce5d7c31 00007f59ce5d6dfd 000000000038f354 000000000038f199 00007f59cb58a475
0709 13:12:13.798095 (+653661us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b18003a8640. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5ecdbe 00007f59ce5f7930 00007f59ce5f78d8
0709 13:12:14.253222 (+455127us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":305,"mutex_wait_us":28698,"raft.queue_time_us":17703,"raft.run_cpu_time_us":8104,"raft.run_wall_time_us":3234331,"spinlock_wait_cycles":1523374592,"thread_start_us":129238,"threads_started":1,"wal-append.queue_time_us":196910,"wal-append.run_cpu_time_us":18309,"wal-append.run_wall_time_us":3223551}
I20260709 13:12:15.085927 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1669) took 1397 ms. Trace:
I20260709 13:12:14.052155 30456 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:15.093562 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1939) took 1828 ms. Trace:
W20260709 13:12:14.102784 31096 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 13:12:14.458792 30735 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.000s
I20260709 13:12:14.098767 30837 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:14.106156 30990 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:15.123553 30779 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:15.093725 30206 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.133606 30748 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.127421 30543 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:14.250087 30828 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.147455 31103 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:15.134452 28155 rpcz_store.cc:276] 0709 13:12:13.895583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.982481 (+ 86898us) service_pool.cc:224] Handling call
0709 13:12:13.983079 (+   598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:14.252925 (+269846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.022212 (+769287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.180487 31084 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:15.135016 28158 rpcz_store.cc:276] 0709 13:12:13.902355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.985765 (+ 83410us) service_pool.cc:224] Handling call
0709 13:12:15.023459 (+1037694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.023682 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.023718 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.023776 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:15.023840 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.023929 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.024053 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.019502 31046 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.176470 28231 rpcz_store.cc:276] 0709 13:12:13.688181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.290301 (+602120us) service_pool.cc:224] Handling call
0709 13:12:15.085306 (+795005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.085497 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.085520 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.085574 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:15.085637 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.085711 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.085854 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.121409 30973 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:15.177673 28230 rpcz_store.cc:276] 0709 13:12:13.272077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.275407 (+  3330us) service_pool.cc:224] Handling call
0709 13:12:13.275827 (+   420us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:12:13.312949 (+ 37122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:13.312979 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:12:13.497681 (+184702us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b2c0004c898. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:13.498456 (+   775us) log.cc:844] Serialized 78 byte log entry
0709 13:12:13.827672 (+329216us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:14.101207 (+273535us) log.cc:844] Serialized 14 byte log entry
0709 13:12:14.342774 (+241567us) log.cc:844] Serialized 14 byte log entry
0709 13:12:14.343017 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:14.343110 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:15.093295 (+750185us) raft_consensus.cc:1685] finished
0709 13:12:15.093321 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.093506 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":267024,"prepare.queue_time_us":191884,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":8136,"raft.queue_time_us":376442,"raft.run_cpu_time_us":521,"raft.run_wall_time_us":21915,"spinlock_wait_cycles":4226304,"thread_start_us":524019,"threads_started":2,"wal-append.queue_time_us":246174}
I20260709 13:12:15.124689 31002 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:15.131335 30892 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.012s	sys 0.009s
I20260709 13:12:15.093081 30768 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:15.142385 30746 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:15.129978 31081 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.147984 30780 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:15.130666 30942 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:15.134565 31104 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.147425 30804 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:15.132757 28228 rpcz_store.cc:276] 0709 13:12:13.688492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.398736 (+710244us) service_pool.cc:224] Handling call
0709 13:12:14.399175 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.131908 (+732733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.132208 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:15.185515 31123 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.929s	user 0.002s	sys 0.000s
I20260709 13:12:15.209057 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 503) took 1490 ms. Trace:
I20260709 13:12:15.177258 30955 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:15.152271 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.130488 30909 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.130700 30882 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.195659 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 472) took 1290 ms. Trace:
I20260709 13:12:15.209472 28228 rpcz_store.cc:276] 0709 13:12:13.718388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.208080 (+1489692us) service_pool.cc:224] Handling call
0709 13:12:15.208450 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.208678 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.208720 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.208785 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:15.208818 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.208868 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.208997 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.188609 30748 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.198839 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:15.199999 28231 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:15.186074 30779 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:15.199000 30892 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.014s	sys 0.009s
I20260709 13:12:15.204172 31002 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.205369 30543 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:15.190212 31103 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.193094 31084 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.211874 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 504) took 1493 ms. Trace:
I20260709 13:12:15.202910 30780 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:15.210731 28158 rpcz_store.cc:276] 0709 13:12:13.904928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.194477 (+1289549us) service_pool.cc:224] Handling call
0709 13:12:15.194820 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.195329 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.195360 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.195408 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:15.195444 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.195503 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.195612 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.215566 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 499) took 1502 ms. Trace:
I20260709 13:12:15.214016 31016 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:15.228719 31016 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:13.461596 28152 rpcz_store.cc:276] 0709 13:12:12.240399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.681133 (+440734us) service_pool.cc:224] Handling call
0709 13:12:12.745146 (+ 64013us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:12.745361 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:13.185237 (+439876us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.428989 (+243752us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.457488 (+ 28499us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:13.457553 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:13.457612 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.457696 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:13.457857 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":479,"prepare.queue_time_us":374817,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":5178,"raft.queue_time_us":1658,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":17876,"thread_start_us":654126,"threads_started":2,"wal-append.queue_time_us":226269}
I20260709 13:12:15.231801 31016 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:15.201484 30768 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:15.217392 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1938) took 1955 ms. Trace:
I20260709 13:12:15.208869 30804 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:15.220219 28228 rpcz_store.cc:276] 0709 13:12:13.718666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.211010 (+1492344us) service_pool.cc:224] Handling call
0709 13:12:15.211320 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.211519 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.211547 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.211590 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:15.211625 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.211685 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.211821 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.208032 30942 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.226508 30779 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.232405 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:15.197882 30973 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.237730 28231 rpcz_store.cc:276] 0709 13:12:13.713676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.196916 (+1483240us) service_pool.cc:224] Handling call
0709 13:12:15.197263 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.211352 (+ 14089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.215475 (+  4123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.215137 30968 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:15.241622 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 505) took 1522 ms. Trace:
I20260709 13:12:15.237275 28232 rpcz_store.cc:276] 0709 13:12:13.262380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.262492 (+   112us) service_pool.cc:224] Handling call
0709 13:12:13.262853 (+   361us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:13.281420 (+ 18567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:13.281450 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:13.286293 (+  4843us) log.cc:844] Serialized 52 byte log entry
0709 13:12:13.728468 (+442175us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:14.050983 (+322515us) log.cc:844] Serialized 14 byte log entry
0709 13:12:15.012709 (+961726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.012819 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:15.216965 (+204146us) raft_consensus.cc:1685] finished
0709 13:12:15.216989 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.217167 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47997,"prepare.queue_time_us":116781,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":3058,"raft.queue_time_us":260793,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":71826,"thread_start_us":702893,"threads_started":2,"wal-append.queue_time_us":587945}
I20260709 13:12:15.209707 31104 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.235590 31002 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:15.240322 30804 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:15.315445 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 301240us
I20260709 13:12:15.231827 30882 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:15.219082 31126 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:12:15.344507 31096 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.077s	user 0.001s	sys 0.008s
I20260709 13:12:15.345772 31084 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:15.346284 30942 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.346838 31129 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:12:15.377003 31127 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.157s	user 0.003s	sys 0.012s
I20260709 13:12:15.347683 30909 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.348538 31081 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:15.349035 31131 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.349730 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2023) took 2665 ms. Trace:
I20260709 13:12:15.349273 30837 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:15.238159 30780 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.384505 30837 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.385556 30780 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:15.387873 30837 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.349936 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 509) took 1615 ms. Trace:
I20260709 13:12:15.396761 28234 rpcz_store.cc:276] 0709 13:12:13.734396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.927378 (+192982us) service_pool.cc:224] Handling call
0709 13:12:15.349726 (+1422348us) spinlock_profiling.cc:243] Waited 98.5 ms on lock 0x7b54002dafd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:15.349877 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226523008}
I20260709 13:12:15.369701 28228 rpcz_store.cc:276] 0709 13:12:13.718968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.240381 (+1521413us) service_pool.cc:224] Handling call
0709 13:12:15.240855 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.241121 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.241162 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.241228 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:15.241281 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.241365 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.241536 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.380861 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 507) took 1661 ms. Trace:
I20260709 13:12:15.368322 28152 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 1.
I20260709 13:12:15.401273 28231 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:15.400652 28232 rpcz_store.cc:276] 0709 13:12:13.719528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.379777 (+1660249us) service_pool.cc:224] Handling call
0709 13:12:15.380148 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.380396 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.380431 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.380488 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:15.380535 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.380606 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.380785 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.403527 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 508) took 1683 ms. Trace:
I20260709 13:12:15.403792 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 513) took 1602 ms. Trace:
I20260709 13:12:15.404135 28231 rpcz_store.cc:276] 0709 13:12:13.719806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.380175 (+1660369us) service_pool.cc:224] Handling call
0709 13:12:15.380604 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.401675 (+ 21071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.403444 (+  1769us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.404464 28232 rpcz_store.cc:276] 0709 13:12:13.801198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.402782 (+1601584us) service_pool.cc:224] Handling call
0709 13:12:15.403114 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.403366 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.403404 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.403462 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:15.403506 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.403583 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.403716 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:15.407944 28231 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:15.408869 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 514) took 1555 ms. Trace:
W20260709 13:12:15.377908 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 228066us
W20260709 13:12:15.362622 31124 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.161s	user 0.008s	sys 0.000s
I20260709 13:12:15.389220 31081 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.389700 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:15.382850 31129 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.383564 28159 rpcz_store.cc:276] 0709 13:12:12.684213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.688985 (+  4772us) service_pool.cc:224] Handling call
0709 13:12:12.747063 (+ 58078us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:13.258336 (+511273us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:13.265976 (+  7640us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.722049 (+456073us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:13.723963 (+  1914us) log.cc:844] Serialized 26 byte log entry
0709 13:12:13.724104 (+   141us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:13.728953 (+  4849us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.745900 (+ 16947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:13.745977 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:15.271356 (+1525379us) raft_consensus.cc:1685] finished
0709 13:12:15.344713 (+ 73357us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.349647 (+  4934us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77105,"prepare.queue_time_us":11288,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":93,"raft.queue_time_us":78770,"raft.run_cpu_time_us":406,"raft.run_wall_time_us":26502,"thread_start_us":849905,"threads_started":2,"wal-append.queue_time_us":506977}
I20260709 13:12:15.420831 28158 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:15.393035 31131 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.409274 28231 rpcz_store.cc:276] 0709 13:12:13.854143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.406743 (+1552600us) service_pool.cc:224] Handling call
0709 13:12:15.407173 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.408486 (+  1313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.408794 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:15.417709 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.416139 30909 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.413924 31084 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:15.435258 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.437077 31122 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.443964 31103 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.446067 30973 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.427591 30955 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:15.462746 31016 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:15.448457 31081 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.402524 31126 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.463336 31016 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.466501 31126 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.467718 28228 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:15.469883 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 512) took 1678 ms. Trace:
I20260709 13:12:15.470521 28228 rpcz_store.cc:276] 0709 13:12:13.791363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.400085 (+1608722us) service_pool.cc:224] Handling call
0709 13:12:15.400406 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.468253 (+ 67847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.469499 (+  1246us) spinlock_profiling.cc:243] Waited 66.8 ms on lock 0x7b54001f54d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:15.469794 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153621248}
I20260709 13:12:15.428167 30987 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:15.478703 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:15.478824 30987 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.453233 30768 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.454214 30909 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.455042 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:15.455855 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:15.456679 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:15.381711 30968 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.483136 28760 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 2: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:15.458278 28152 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:15.460064 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 501) took 1742 ms. Trace:
I20260709 13:12:15.462178 30955 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.450932 31129 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.477463 31122 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.486106 28230 rpcz_store.cc:276] 0709 13:12:13.717729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.196242 (+1478513us) service_pool.cc:224] Handling call
0709 13:12:15.196512 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.196679 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:15.264595 (+ 67916us) log.cc:844] Serialized 14 byte log entry
0709 13:12:15.459704 (+195109us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.459757 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:15.459819 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.459887 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.460011 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":67803,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":67,"raft.queue_time_us":1194,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":257418,"threads_started":2,"wal-append.queue_time_us":210248}
I20260709 13:12:15.494663 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.489180 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.489687 28231 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:12:15.495755 28231 pending_rounds.cc:85] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 3
I20260709 13:12:15.496024 28231 pending_rounds.cc:107] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:12:15.491058 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 510) took 1745 ms. Trace:
I20260709 13:12:15.491415 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.496176 28228 raft_consensus.cc:2410] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:12:15.492108 30987 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.492408 31131 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.496604 28232 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:15.496232 28231 raft_consensus.cc:2889] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:15.496384 28234 rpcz_store.cc:276] 0709 13:12:13.745412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.397876 (+1652464us) service_pool.cc:224] Handling call
0709 13:12:15.398268 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:15.398650 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.398680 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:15.399171 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:12:15.460930 (+ 61759us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:15.460971 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.461038 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:15.490866 (+ 29828us) raft_consensus.cc:1685] finished
0709 13:12:15.490894 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.491009 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":119,"thread_start_us":61243,"threads_started":1,"wal-append.queue_time_us":90138}
I20260709 13:12:15.501569 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.498963 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.502290 28232 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:15.502578 28231 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:15.503120 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:15.520829 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.521812 28233 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:15.522279 31122 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.527552 31016 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:15.527675 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.532182 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.534593 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.492420 31104 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.472867 31084 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:15.543212 31084 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:15.544103 28233 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 4: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:15.546345 28228 raft_consensus.cc:2410] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:12:15.472211 30882 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:15.548012 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44958.7us
I20260709 13:12:15.547252 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 477) took 1633 ms. Trace:
I20260709 13:12:15.550290 31129 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:15.550856 31129 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:15.553177 31129 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:15.555879 30955 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.557915 30909 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.558175 31015 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:15.559864 30968 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:15.550194 28159 rpcz_store.cc:276] 0709 13:12:13.913633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.420891 (+1507258us) service_pool.cc:224] Handling call
0709 13:12:15.424206 (+  3315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:15.426128 (+  1922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:15.426160 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:15.432225 (+  6065us) log.cc:844] Serialized 26 byte log entry
0709 13:12:15.476553 (+ 44328us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:15.476617 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.476770 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:15.547031 (+ 70261us) raft_consensus.cc:1685] finished
0709 13:12:15.547054 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.547195 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1497,"raft.queue_time_us":3530,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":9846,"thread_start_us":20364,"threads_started":1,"wal-append.queue_time_us":88774}
I20260709 13:12:15.560824 30982 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:15.561511 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:15.561792 30804 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.562525 28159 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 6: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:15.564035 28155 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:15.567193 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:15.566870 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:15.547744 30746 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:15.628897 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:15.602977 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 1.74127e+06us
I20260709 13:12:15.616400 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:15.629456 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:15.636710 28232 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:15.567957 30882 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.641831 30882 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.642459 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:15.643225 28762 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:15.648672 28077 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:15.650761 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.631299 30804 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.613075 28086 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:12:15.632688 31020 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.690536 28762 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:15.634445 30819 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:15.655310 28086 pending_rounds.cc:85] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 7
I20260709 13:12:15.811266 28086 pending_rounds.cc:107] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:12:15.683826 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:15.737658 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 59578.2us
W20260709 13:12:15.728448 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.693804 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:15.712615 28082 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:12:15.689574 30748 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
W20260709 13:12:15.741458 28052 connection.cc:256] RPC call timeout handler was delayed by -1.5566453209999054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 13:12:15.788846 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 130582us
I20260709 13:12:15.812985 28086 raft_consensus.cc:2889] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:15.829356 28052 connection.cc:256] RPC call timeout handler was delayed by -1.391045320999865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
I20260709 13:12:15.716408 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:15.817319 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:15.830222 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:15.689975 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54190.1us
I20260709 13:12:15.831760 28049 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:15.748035 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:15.715056 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.653861 31119 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:15.833148 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.655021 28080 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:15.743050 30791 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.842957 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:15.680922 31046 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.741804 31053 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:15.833946 28231 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:15.837810 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:15.859572 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1951) took 3652 ms. Trace:
I20260709 13:12:15.687130 30985 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:15.638231 30746 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.810364 31129 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:15.910676 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 76105.6us
I20260709 13:12:15.839109 30982 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:15.913590 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 1995) took 3980 ms. Trace:
I20260709 13:12:15.925918 28078 rpcz_store.cc:276] 0709 13:12:11.933256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.249121 (+315865us) service_pool.cc:224] Handling call
0709 13:12:13.097194 (+848073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:13.100114 (+  2920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.913270 (+2813156us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54002972d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:15.913496 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1949500288}
W20260709 13:12:15.856158 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 167387us
I20260709 13:12:15.858362 30814 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:15.931515 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:15.848975 30804 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:15.829016 30819 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.814757 31020 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.850704 30735 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 13:12:15.932435 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:15.940022 28232 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:15.940022 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:15.937490 30838 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:12:15.902285 30882 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:12:15.851665 30791 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.888679 31046 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.930968 30814 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:15.952152 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:15.935194 30828 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:15.936553 30804 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:15.912525 31140 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:12:15.849258 31119 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.905423 31053 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:15.956241 31120 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:15.956212 30814 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:15.923404 30985 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.981074 31120 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:15.997853 31120 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.999468 31046 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:16.003085 30819 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:15.959720 30828 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.010929 30804 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.016820 31053 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:16.018761 31129 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:15.952613 31051 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:12:16.022002 31020 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:16.118360 30985 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:16.155798 31119 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:16.236925 31020 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:16.237816 31020 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:16.255625 31020 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:16.251612 31155 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:16.258164 31155 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:16.259981 31155 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:16.264072 31155 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:16.303870 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 457459us
I20260709 13:12:15.931444 28085 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:16.317634 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 479) took 1219 ms. Trace:
I20260709 13:12:16.318666 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:16.318243 28159 rpcz_store.cc:276] 0709 13:12:15.097876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.563276 (+465400us) service_pool.cc:224] Handling call
0709 13:12:15.563605 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.563841 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:15.848058 (+284217us) log.cc:844] Serialized 14 byte log entry
0709 13:12:16.316616 (+468558us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.316794 (+   178us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:16.316939 (+   145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.317076 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.317375 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65700,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":62,"thread_start_us":705804,"threads_started":2,"wal-append.queue_time_us":314713}
I20260709 13:12:16.321815 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:16.326982 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:16.327312 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:16.330420 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:16.332345 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 408906us
I20260709 13:12:16.241420 30902 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:16.378260 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 480) took 1280 ms. Trace:
I20260709 13:12:16.378937 28155 rpcz_store.cc:276] 0709 13:12:15.098186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.190705 (+ 92519us) service_pool.cc:224] Handling call
0709 13:12:16.377689 (+1186984us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54001978d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:16.378013 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810978688}
I20260709 13:12:16.379940 31152 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:12:16.382629 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 473) took 2470 ms. Trace:
I20260709 13:12:16.383273 28158 rpcz_store.cc:276] 0709 13:12:13.912431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.270744 (+1358313us) service_pool.cc:224] Handling call
0709 13:12:15.351072 (+ 80328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.421366 (+ 70294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.381840 (+960474us) spinlock_profiling.cc:243] Waited 61 ms on lock 0x7b5400238554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:16.382380 (+   540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140338560}
I20260709 13:12:16.399132 30791 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:16.402518 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 1941) took 3112 ms. Trace:
I20260709 13:12:16.403275 28229 rpcz_store.cc:276] 0709 13:12:13.287390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.288988 (+  1598us) service_pool.cc:224] Handling call
0709 13:12:13.289359 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:13.313395 (+ 24036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:13.313432 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:15.971975 (+2658543us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b2c0004c898. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:15.972433 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:12:16.398867 (+426434us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:16.398980 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.399171 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:16.399262 (+    91us) raft_consensus.cc:1685] finished
0709 13:12:16.399280 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.399726 (+   446us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23630,"raft.queue_time_us":26454,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":84,"spinlock_wait_cycles":3518208,"thread_start_us":418916,"threads_started":1,"wal-append.queue_time_us":391782}
W20260709 13:12:16.359874 28228 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.000s	sys 0.011s
I20260709 13:12:16.363183 30898 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:16.416822 28228 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.000s	sys 0.012s
W20260709 13:12:16.421423 28051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.017s	sys 0.038s
W20260709 13:12:16.422042 28051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.017s	sys 0.038s
I20260709 13:12:16.421344 30898 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.424718 31087 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:16.425786 30898 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:16.426277 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.434495 30992 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:16.436205 30206 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:16.437377 31053 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.438347 31053 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.439527 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.440943 28232 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:16.442525 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.440966 31048 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:16.414600 31152 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:12:16.473117 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:16.476471 28154 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:16.477931 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:16.485255 28154 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
W20260709 13:12:16.487165 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 70168.7us
I20260709 13:12:16.490034 28159 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:16.449141 30974 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.490347 28154 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:16.474088 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:16.488488 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:16.488417 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.497277 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1663) took 2898 ms. Trace:
I20260709 13:12:16.466775 31020 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:16.499230 28226 rpcz_store.cc:276] 0709 13:12:13.586690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.586916 (+   226us) service_pool.cc:224] Handling call
0709 13:12:13.640020 (+ 53104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:13.868029 (+228009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.484064 (+2616035us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54001f4fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:16.484567 (+   503us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464110848}
I20260709 13:12:16.508148 28158 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:16.510246 28230 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:16.511672 31053 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:16.514968 30746 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:16.518532 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 474) took 2605 ms. Trace:
I20260709 13:12:16.521111 31053 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:16.523402 28152 rpcz_store.cc:276] 0709 13:12:13.912777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.407518 (+1494741us) service_pool.cc:224] Handling call
0709 13:12:15.410371 (+  2853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.485667 (+ 75296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.517810 (+1032143us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b54003769d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:16.518320 (+   510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109057280}
I20260709 13:12:16.532274 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:16.530949 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet c71e4401306a4e84a111e5443be7652c: Aborted: Op aborted by new leader
I20260709 13:12:16.494623 30974 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:16.535450 28161 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:16.535194 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:16.544791 30746 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:16.545885 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 713999us
I20260709 13:12:16.545917 30746 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:16.547497 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:16.540334 30804 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.557118 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:16.560372 28153 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:16.548703 30746 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:16.561048 28153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2037) took 2664 ms (client timeout 2052 ms). Trace:
W20260709 13:12:16.563402 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 134013us
I20260709 13:12:16.561576 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:16.554600 31053 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:16.563437 28153 rpcz_store.cc:269] 0709 13:12:13.896240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.964574 (+ 68334us) service_pool.cc:224] Handling call
0709 13:12:16.526085 (+2561511us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:16.560934 (+ 34849us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5204736}
I20260709 13:12:16.574283 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:16.574548 30804 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.575702 31053 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:16.576213 31053 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:16.576709 31053 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:16.578298 30974 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:16.579547 28730 connection.cc:256] RPC call timeout handler was delayed by -1.6863810830000148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 13:12:16.581643 28730 connection.cc:256] RPC call timeout handler was delayed by -1.4562974230000236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
W20260709 13:12:16.581995 28730 connection.cc:256] RPC call timeout handler was delayed by -1.253721293000126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
I20260709 13:12:16.582790 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:16.586622 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.592372 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:16.594800 30804 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:16.598727 28152 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:12:16.603399 31117 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:16.603960 31117 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:16.604318 31117 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:16.608197 30814 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.613s	user 0.014s	sys 0.000s
I20260709 13:12:16.608573 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1684) took 1076 ms. Trace:
I20260709 13:12:16.609128 28233 rpcz_store.cc:276] 0709 13:12:15.531636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:15.534290 (+  2654us) service_pool.cc:224] Handling call
0709 13:12:16.608077 (+1073787us) spinlock_profiling.cc:243] Waited 8.54 ms on lock 0x7b5400391cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:16.608359 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19652992}
I20260709 13:12:16.609108 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:16.617197 30819 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:16.617856 30819 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:16.618350 30819 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:16.618414 28233 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:12:16.618911 28233 pending_rounds.cc:85] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 2
I20260709 13:12:16.619226 28233 pending_rounds.cc:107] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:12:16.619529 28233 raft_consensus.cc:2889] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:16.627575 28079 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:12:16.628854 30896 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.000s	sys 0.015s
W20260709 13:12:16.629419 30896 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.000s	sys 0.015s
I20260709 13:12:16.630832 30828 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:16.642427 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45175.3us
I20260709 13:12:15.872298 28157 rpcz_store.cc:276] 0709 13:12:12.207616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.645628 (+438012us) service_pool.cc:224] Handling call
0709 13:12:12.645936 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:12.646150 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:12.916003 (+269853us) log.cc:844] Serialized 14 byte log entry
0709 13:12:12.938123 (+ 22120us) log.cc:844] Serialized 14 byte log entry
0709 13:12:13.662242 (+724119us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b180031c4e0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a522 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:12:15.858966 (+2196724us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b180031c4e0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59ce22dd70 00007f59ce221293 00007f59ce216cea 00007f59ce21723a 00007f59ce681a2a 00007f59ce6910bf 00007f59ce691038
0709 13:12:15.859219 (+   253us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:15.859270 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:15.859315 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:15.859383 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:15.859514 (+   131us) inbound_call.cc:177] Queueing success response
0709 13:12:15.869412 (+  9898us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b180031c4e0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a522 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
Metrics: {"mutex_wait_us":1317,"prepare.queue_time_us":237774,"prepare.run_cpu_time_us":6339,"prepare.run_wall_time_us":32864,"raft.queue_time_us":971,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":15436,"spinlock_wait_cycles":32981888,"thread_start_us":269391,"threads_started":1}
I20260709 13:12:16.649096 30768 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:16.649855 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2028) took 2789 ms. Trace:
I20260709 13:12:16.650030 30768 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.650352 28160 rpcz_store.cc:276] 0709 13:12:13.860194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:13.860640 (+   446us) service_pool.cc:224] Handling call
0709 13:12:13.860915 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:15.181193 (+1320278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.649284 (+1468091us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b5400196ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:16.649628 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4694912}
W20260709 13:12:16.653668 31170 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:12:16.654913 30768 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:16.645826 28049 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:16.659509 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:16.662990 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:16.658356 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55520.9us
I20260709 13:12:16.668550 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.670677 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.670991 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:16.671456 30902 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:16.672375 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.673507 28230 raft_consensus.cc:2410] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:16.677595 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 471) took 2772 ms. Trace:
I20260709 13:12:16.678611 31104 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:16.679364 31104 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:16.679812 31104 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:16.686167 31169 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.146s	user 0.000s	sys 0.002s
I20260709 13:12:16.681190 28155 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:16.687309 28229 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:16.681411 30902 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:16.688403 30902 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:16.689702 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 750) took 4133 ms. Trace:
I20260709 13:12:16.690024 30902 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:16.690164 28758 rpcz_store.cc:276] 0709 13:12:12.556617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:12.563861 (+  7244us) service_pool.cc:224] Handling call
0709 13:12:12.627870 (+ 64009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:13.967278 (+1339408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.689099 (+2721821us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54005f0fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:16.689415 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2957409152}
I20260709 13:12:16.691776 30902 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:16.694876 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:16.696012 28762 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:16.698297 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:16.699254 28762 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:16.701251 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.702734 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.701277 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:16.615053 30814 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.704006 28083 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:16.704628 28085 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:16.707719 30814 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:16.708876 30659 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:16.678275 28156 rpcz_store.cc:276] 0709 13:12:13.904578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:14.255148 (+350570us) service_pool.cc:224] Handling call
0709 13:12:16.675874 (+2420726us) spinlock_profiling.cc:243] Waited 2.95 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:12:16.676307 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:16.676945 (+   638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:16.676980 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:16.677073 (+    93us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:16.677132 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:16.677224 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:16.677373 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6783872}
I20260709 13:12:16.701222 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:16.703205 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.718354 31074 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:16.709010 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 63541.2us
W20260709 13:12:16.715127 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 44332.7us
I20260709 13:12:16.727921 28079 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:16.733387 31033 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:16.735167 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 137238us
I20260709 13:12:16.735780 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.736423 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:16.736766 28231 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:16.738695 28730 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:16.740200 30812 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:16.718879 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.741153 30812 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:16.766460 31074 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.768880 30891 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:12:16.724622 30746 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:16.772315 30746 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:16.774667 30746 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:16.775213 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.775961 28080 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:16.760668 30659 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.777171 30659 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:16.779585 30659 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:16.765280 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:16.741385 31053 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.788483 28083 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:16.796765 31033 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:16.805159 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.811329 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.817255 28078 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 13:12:16.818426 30768 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:16.818466 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.819939 28080 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:16.824851 31074 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:16.825747 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.828646 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:16.832437 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 86297.2us
I20260709 13:12:16.838995 28231 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:12:16.839828 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:16.845237 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 100075us
I20260709 13:12:16.847036 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.847368 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:16.847810 28233 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:16.848533 31033 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:16.847295 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:16.851872 30884 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.852514 30884 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.845098 30812 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:12:16.854866 30884 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:16.858528 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:16.851886 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:16.861357 30768 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:16.861886 30768 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:16.862319 30768 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:16.860602 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:16.867575 30914 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:16.867537 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 53944.8us
I20260709 13:12:16.882489 30659 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:16.883490 30659 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:16.886056 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.887164 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.887571 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.888027 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.888432 28083 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:16.889293 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:16.890321 28077 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:16.890075 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.891382 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.891631 28086 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:16.892232 28079 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:16.893491 30659 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:16.883037 28229 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:12:16.868691 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:16.906162 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.908635 31125 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:16.909070 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:16.911000 30659 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:16.910590 30892 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:16.913625 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:16.899719 28229 pending_rounds.cc:85] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 7
I20260709 13:12:16.926371 28229 pending_rounds.cc:107] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:12:16.926925 28229 raft_consensus.cc:2889] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:16.927438 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 69675.6us
I20260709 13:12:16.932063 28078 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
W20260709 13:12:16.920068 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45290.6us
I20260709 13:12:16.935750 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:16.877856 30819 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:16.912084 31153 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:16.948122 31053 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:16.954242 30206 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:16.958503 30884 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:16.875741 31074 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:16.961208 31074 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:16.963877 31074 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:16.970992 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 44179.6us
I20260709 13:12:16.982914 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:16.985459 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:16.925724 30659 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:16.998571 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.001839 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.003659 28079 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:17.008919 30804 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:17.010416 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:17.015424 28125 connection.cc:256] RPC call timeout handler was delayed by -2.3795060850000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
I20260709 13:12:16.996206 30659 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:17.018687 28082 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
W20260709 13:12:17.016043 28125 connection.cc:256] RPC call timeout handler was delayed by -2.2949060849998659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19210000000000002s for a response
I20260709 13:12:17.019974 31155 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:12:17.023259 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 47248.1us
I20260709 13:12:17.023624 28079 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:17.024242 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 155690us
I20260709 13:12:16.996461 31074 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:17.028707 31104 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:17.030246 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:17.018898 30746 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:17.037600 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:17.038753 30882 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:17.039422 30882 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:17.040156 28081 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:17.040444 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:17.053718 30984 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:17.035539 31053 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:17.054590 30984 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:17.054919 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:17.041496 31155 consensus_queue.cc:1059] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:17.061877 31155 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:17.062489 31155 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:17.063354 30892 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:17.063853 30892 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:17.064284 30892 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:17.065377 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:17.066934 28229 raft_consensus.cc:2410] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:12:17.047179 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:17.048775 31074 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:17.039444 28079 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:17.054684 31053 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:17.041513 30746 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:17.056959 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.058755 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:17.059077 30768 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:17.060097 30819 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:17.019251 31117 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:17.073560 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.073904 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.074520 27984 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 3, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 3 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:17.079829 31074 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:17.032771 30902 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:17.085229 31053 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:17.055375 31020 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:17.092747 28080 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
W20260709 13:12:17.104414 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57334us
I20260709 13:12:17.109490 30814 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:17.132829 30814 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:17.133878 27982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 275) took 13369 ms (client timeout 15000 ms). Trace:
I20260709 13:12:17.135303 30814 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:17.135385 31074 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:17.136087 31074 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:17.136629 28078 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:17.137959 28082 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:17.125108 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:17.128382 30902 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:17.138890 30876 tablet_copy_client.cc:538] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:12:17.139117 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49771.1us
I20260709 13:12:17.140760 30902 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:17.140936 31153 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:17.141423 28230 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:17.141610 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.141784 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.142477 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.142836 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:17.142452 28082 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:17.143808 28077 raft_consensus.cc:2410] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:17.143702 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.144162 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.142879 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.140621 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:17.063915 31155 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:17.145074 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.161022 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.147143 28078 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:17.147233 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.147962 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:17.147284 28125 connection.cc:256] RPC call timeout handler was delayed by -2.1545055379999667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
I20260709 13:12:17.152460 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.157437 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:17.144691 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.126817 30914 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:17.172006 30914 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:17.172374 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.159732 31033 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:12:17.174821 28757 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:17.174528 27984 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index -1 to 3, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 3 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:12:17.169682 31125 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:12:17.177242 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.177709 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.178678 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.179647 28764 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 8: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:17.179930 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:17.134289 27982 rpcz_store.cc:269] 0709 13:12:03.769967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.966876 (+196909us) service_pool.cc:224] Handling call
0709 13:12:17.133634 (+13166758us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:11.538872 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:11.701212 (+162340us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:11.725924 (+ 24712us) write_op.cc:435] Acquired schema lock
0709 13:12:11.739010 (+ 13086us) tablet.cc:662] Decoding operations
0709 13:12:11.993448 (+254438us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:12.026895 (+ 33447us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:12.029151 (+  2256us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:12:12.285395 (+256244us) tablet.cc:701] Row locks acquired
0709 13:12:12.285447 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:12:12.285612 (+   165us) write_op.cc:270] Start()
0709 13:12:12.285754 (+   142us) write_op.cc:276] Timestamp: P: 1783602732285548 usec, L: 0
0709 13:12:12.285774 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:12:12.286470 (+   696us) log.cc:844] Serialized 7885 byte log entry
0709 13:12:12.556931 (+270461us) spinlock_profiling.cc:243] Waited 64.5 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:13.455839 (+898908us) op_driver.cc:464] REPLICATION: finished
0709 13:12:14.059851 (+604012us) write_op.cc:301] APPLY: starting
0709 13:12:14.059950 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:14.060403 (+   453us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:14.060427 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:14.208015 (+147588us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:15.093821 (+885806us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 13:12:15.121332 (+ 27511us) write_op.cc:312] APPLY: finished
0709 13:12:15.127125 (+  5793us) log.cc:844] Serialized 139 byte log entry
0709 13:12:15.201224 (+ 74099us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:15.203594 (+  2370us) write_op.cc:454] Released schema lock
0709 13:12:15.205276 (+  1682us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:15.235185 (+ 29909us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:15.239730 (+  4545us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1574698,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":530064,"apply.run_cpu_time_us":54415,"apply.run_wall_time_us":1153856,"num_ops":20,"prepare.queue_time_us":193780,"prepare.run_cpu_time_us":32087,"prepare.run_wall_time_us":808247,"raft.queue_time_us":629482,"raft.run_cpu_time_us":25604,"raft.run_wall_time_us":460770,"replication_time_us":1169869,"spinlock_wait_cycles":148437248,"thread_start_us":1155127,"threads_started":5,"wal-append.queue_time_us":175411,"wal-append.run_cpu_time_us":86968,"wal-append.run_wall_time_us":3138656}]]}
I20260709 13:12:17.181519 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.160329 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:17.182655 28125 connection.cc:256] RPC call timeout handler was delayed by -2.0132055379999656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
W20260709 13:12:17.183115 28125 connection.cc:256] RPC call timeout handler was delayed by -1.8179060850000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2451s for a response
I20260709 13:12:17.153399 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.176468 27984 catalog_manager.cc:5697] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 4 to 5, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 5 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:17.138306 31074 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:17.129290 31020 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:17.153705 31104 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:17.146842 30746 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:17.172457 30914 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:17.154383 31117 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:17.153003 30984 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:17.184870 30898 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.148255 30882 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:17.187562 30814 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:17.220583 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.189000 31153 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:17.202574 31020 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:17.223294 28082 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:12:17.233350 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.214747 30891 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:17.236356 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 79062.9us
I20260709 13:12:17.242650 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.201867 30902 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:12:17.224397 30814 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:17.244673 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:17.249024 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 93013.9us
I20260709 13:12:17.250299 28763 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:17.221290 27984 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:17.254604 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.235021 31155 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:17.236825 30891 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:17.263105 28125 connection.cc:256] RPC call timeout handler was delayed by -1.6400621109999065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20120000000000002s for a response
W20260709 13:12:17.263705 28125 connection.cc:256] RPC call timeout handler was delayed by -1.494009510999831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 13:12:17.264144 28125 connection.cc:256] RPC call timeout handler was delayed by -1.4336060849998375s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2878s for a response
I20260709 13:12:17.226529 30898 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:17.235819 31104 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:17.268187 28083 raft_consensus.cc:2410] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:12:17.268126 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.251245 30814 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:17.278188 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.260986 30891 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:17.282289 28082 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:17.281967 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.286635 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.258216 30659 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:17.294016 30902 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:17.294549 30902 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:17.294992 30659 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:17.299108 28077 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:17.312615 28156 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:17.311411 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.317301 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:17.319948 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.323650 28050 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:17.327164 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.281945 31104 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:17.327509 31187 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:17.321503 28160 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:17.333299 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.348199 28232 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:17.348053 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.348829 30898 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:17.349557 31051 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:17.349964 31051 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:17.350313 31051 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:17.352208 31186 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.352797 31186 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:17.354950 31186 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:17.355950 31104 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:17.321951 31125 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:12:17.388747 27984 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:17.358932 31117 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:12:17.390666 27984 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 4 to 5, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 5 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:17.373396 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:17.392946 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 69373.3us
I20260709 13:12:17.377170 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.361878 30884 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:17.330060 30768 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:17.369226 28198 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:17.393114 28157 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
W20260709 13:12:17.400693 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 74851.4us
I20260709 13:12:17.402575 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.404088 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.405033 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.405949 28085 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:17.392241 27984 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:17.412197 28154 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 3.
I20260709 13:12:17.413686 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.414121 30882 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.414382 28152 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:17.414772 30882 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:17.425601 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.435243 30882 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:17.439441 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.412480 30984 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:12:17.446144 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:17.447693 31179 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.216s	user 0.004s	sys 0.000s
I20260709 13:12:17.448324 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:17.465898 28755 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:12:17.465888 28124 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:17.469628 28050 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:17.475311 28232 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:17.430001 31186 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:17.487989 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:17.486714 31117 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:17.494028 31117 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:17.494500 31117 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:17.494889 30804 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:17.495347 30804 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:17.495695 30804 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:17.497040 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.497735 28154 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:17.499451 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.500589 28156 raft_consensus.cc:2410] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:17.502753 30884 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:17.494915 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:17.513278 28154 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:17.513302 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.516180 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.409911 30892 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:17.516882 28156 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:17.436151 30768 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:17.523729 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.521219 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:17.502637 31125 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:17.429589 31191 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:12:17.586690 28080 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:17.619410 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.625527 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.591090 27984 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 5 to 6, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 6 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:17.486063 30659 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.132s
I20260709 13:12:17.572043 31125 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:17.639019 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.658532 28760 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:17.586498 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.658764 27984 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 6, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 6 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:17.636921 28156 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:17.667747 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.671072 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:17.675503 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 85185.3us
I20260709 13:12:17.587682 30814 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:17.585243 30768 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:17.673339 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:17.692018 28233 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.005s	sys 0.030s
W20260709 13:12:17.692790 28233 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.006s	sys 0.030s
I20260709 13:12:17.696982 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:17.708474 28152 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:17.696871 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:17.711951 28160 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:17.714298 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.714257 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.713676 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:17.759841 27984 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 6 to 7, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 7 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:17.764772 28153 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:17.777475 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 256308us
W20260709 13:12:17.779130 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:17.783077 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.698720 31104 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:17.788230 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 111259us
I20260709 13:12:17.788208 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:17.817768 28156 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:17.819177 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.829962 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.834988 28154 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:12:17.837514 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.847772 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.826885 30819 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:17.848483 28154 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
W20260709 13:12:17.838634 31053 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.000s
I20260709 13:12:17.863299 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:17.870200 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.873034 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:17.876312 31200 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:17.786293 31125 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:17.881482 31202 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:12:17.895684 28160 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:17.898744 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:17.876935 30814 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:17.870975 27984 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 7 to 8, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 8 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:17.918583 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:17.919143 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 599905us
I20260709 13:12:17.867193 30819 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:17.922514 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:17.854568 30659 consensus_queue.cc:1243] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:17.901690 30898 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.898622 31200 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:17.865468 31053 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:17.939265 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:17.943513 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:17.970534 28154 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:17.921751 30892 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:17.935851 30891 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.001s
I20260709 13:12:17.869689 31218 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:12:17.921820 31117 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:17.945943 27984 catalog_manager.cc:5697] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 4 to 6, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 6 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:17.984289 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 62927.1us
I20260709 13:12:18.008675 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:18.303339 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:18.473121 28730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.003s	sys 0.007s
I20260709 13:12:18.030781 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:18.487784 28730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.004s	sys 0.007s
I20260709 13:12:18.056694 28153 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
W20260709 13:12:18.340754 28085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.005s	sys 0.004s
I20260709 13:12:18.488381 28160 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:18.023587 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:18.158344 28086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.007s	sys 0.045s
W20260709 13:12:18.473233 28084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.001s	sys 0.010s
W20260709 13:12:18.494248 28086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.010s	sys 0.059s
W20260709 13:12:18.494491 28084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.001s	sys 0.010s
I20260709 13:12:18.495783 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 554) took 1029 ms. Trace:
I20260709 13:12:18.384600 30876 tablet_bootstrap.cc:492] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
W20260709 13:12:18.488976 28085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.006s	sys 0.005s
I20260709 13:12:18.060632 30902 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:18.513510 30902 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:18.513616 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 529) took 1294 ms. Trace:
I20260709 13:12:18.514132 28234 rpcz_store.cc:276] 0709 13:12:17.218991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.232240 (+ 13249us) service_pool.cc:224] Handling call
0709 13:12:17.243424 (+ 11184us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:17.262777 (+ 19353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.262827 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:17.284446 (+ 21619us) log.cc:844] Serialized 26 byte log entry
0709 13:12:17.523310 (+238864us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:17.523363 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.526809 (+  3446us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.513363 (+986554us) raft_consensus.cc:1685] finished
0709 13:12:18.513387 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.513535 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40403,"raft.queue_time_us":87671,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":15728,"thread_start_us":191665,"threads_started":1,"wal-append.queue_time_us":1173757}
I20260709 13:12:18.516211 31193 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:18.516659 31193 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:18.518180 28764 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:18.517717 31193 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.518476 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:18.519145 28759 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:18.519346 31193 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:18.522375 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:18.289789 31214 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.501257 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:18.527442 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:18.528468 28755 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 1. Current term is 3. Ops: []
I20260709 13:12:18.063524 30819 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:18.531167 28727 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.011s	sys 0.014s
W20260709 13:12:18.532115 28727 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.011s	sys 0.014s
W20260709 13:12:18.021364 31207 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:12:18.497308 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:18.494095 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:18.450714 31155 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.186s	user 0.012s	sys 0.001s
I20260709 13:12:17.969645 31200 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:18.023422 31205 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 13:12:18.030453 31212 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:12:18.543629 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56675.4us
I20260709 13:12:18.544025 28049 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:18.034467 30804 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.000s
I20260709 13:12:18.526517 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:18.009068 30768 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:18.530186 31214 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:18.349440 31020 consensus_meta.cc:287] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.104s	user 0.012s	sys 0.000s
I20260709 13:12:18.482478 31125 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:18.533864 28156 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 6: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:18.023097 31187 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:18.495303 27984 catalog_manager.cc:5697] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 4 to 6, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 6 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:18.535156 28153 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: [6.8-8.9]
W20260709 13:12:18.444519 30914 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.212s	user 0.007s	sys 0.007s
I20260709 13:12:18.289151 31216 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:18.540829 28759 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:18.542543 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1648) took 1396 ms. Trace:
I20260709 13:12:18.543426 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1646) took 1403 ms. Trace:
W20260709 13:12:18.472922 31219 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.002s	sys 0.007s
W20260709 13:12:18.358546 31051 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.008s	user 0.013s	sys 0.000s
I20260709 13:12:18.548640 28082 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:12:17.981647 30898 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:18.563759 28079 rpcz_store.cc:276] 0709 13:12:17.139814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.169507 (+ 29693us) service_pool.cc:224] Handling call
0709 13:12:18.536011 (+1366504us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:12:18.536261 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.543044 (+  6783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:18.543079 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.543133 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:18.543176 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.543246 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.543370 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25464064}
I20260709 13:12:18.496176 28086 rpcz_store.cc:276] 0709 13:12:17.466363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.498471 (+ 32108us) service_pool.cc:224] Handling call
0709 13:12:17.498777 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.498980 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:17.499457 (+   477us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.495461 (+996004us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:18.495511 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:18.495554 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.495613 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.495733 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":593,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":36,"raft.queue_time_us":201065,"raft.run_wall_time_us":16242,"spinlock_wait_cycles":16341632,"thread_start_us":995225,"threads_started":2,"wal-append.queue_time_us":190126}
I20260709 13:12:18.551182 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:18.570643 28234 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:18.551658 30746 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:18.546720 30814 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:18.549302 30804 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.552451 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:18.552987 31234 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:18.557113 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 556) took 1063 ms. Trace:
I20260709 13:12:17.981868 30891 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:18.566041 31219 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.003s	sys 0.008s
I20260709 13:12:18.564173 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:18.042212 28083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25d46d07733d450e897e3a72d765e511"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
W20260709 13:12:18.559306 28125 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.811s (SENT) [suppressed 18 similar messages]
I20260709 13:12:18.561527 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:18.567003 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.595389 28083 raft_consensus.cc:606] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:18.596606 28083 raft_consensus.cc:614] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:18.556504 31155 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:18.563666 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.601632 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:18.563216 28077 rpcz_store.cc:276] 0709 13:12:17.140363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.171561 (+ 31198us) service_pool.cc:224] Handling call
0709 13:12:17.211640 (+ 40079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.305940 (+ 94300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.536221 (+1230281us) spinlock_profiling.cc:243] Waited 87.2 ms on lock 0x7b5400207e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:18.536510 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200504576}
I20260709 13:12:18.600178 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:18.602759 28157 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:12:18.573855 31053 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:18.560305 31187 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:18.606681 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:18.581911 28084 rpcz_store.cc:276] 0709 13:12:17.493981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.658664 (+164683us) service_pool.cc:224] Handling call
0709 13:12:17.671377 (+ 12713us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:17.683118 (+ 11741us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.683172 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:17.698068 (+ 14896us) log.cc:844] Serialized 26 byte log entry
0709 13:12:18.495936 (+797868us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:18.495972 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.496053 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.556880 (+ 60827us) raft_consensus.cc:1685] finished
0709 13:12:18.556902 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.557054 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24168,"raft.queue_time_us":258741,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6878,"thread_start_us":668449,"threads_started":1,"wal-append.queue_time_us":759002}
I20260709 13:12:18.564633 31215 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:18.560776 31020 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:18.611552 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:18.612532 28079 raft_consensus.cc:2410] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:12:18.612967 28152 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:18.594183 30543 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.613978 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:18.566344 30914 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:18.634619 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 82237.7us
W20260709 13:12:18.612411 31235 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:12:18.632799 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:18.609560 31219 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.610205 31187 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:18.638259 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2042) took 1357 ms. Trace:
I20260709 13:12:18.573041 31051 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:18.641304 28081 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 8. Ops: []
I20260709 13:12:18.638505 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:18.641966 28077 raft_consensus.cc:2368] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9.
I20260709 13:12:18.642262 28078 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:18.645977 28078 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:18.641463 28155 rpcz_store.cc:276] 0709 13:12:17.281073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.284440 (+  3367us) service_pool.cc:224] Handling call
0709 13:12:17.314866 (+ 30426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:17.322192 (+  7326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.322223 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:17.342065 (+ 19842us) log.cc:844] Serialized 26 byte log entry
0709 13:12:17.694287 (+352222us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:17.694341 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.694475 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.638006 (+943531us) raft_consensus.cc:1685] finished
0709 13:12:18.638033 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.638200 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":293853,"threads_started":1,"wal-append.queue_time_us":388663}
I20260709 13:12:18.646759 28153 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:18.650928 28159 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:18.646329 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:18.635782 30884 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:18.643592 30814 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.611869 31155 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.645028 31217 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:18.603132 31216 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.661880 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 768) took 1506 ms. Trace:
I20260709 13:12:18.602087 31234 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:18.659992 31213 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:18.678380 31231 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:18.666225 28760 rpcz_store.cc:276] 0709 13:12:17.156803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.157049 (+   246us) service_pool.cc:224] Handling call
0709 13:12:17.157434 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.665359 (+507925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.658484 (+993125us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400531e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:18.658667 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057744256}
I20260709 13:12:18.633025 31215 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.680620 31020 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:18.667102 31217 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:18.682451 31217 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:18.682493 31215 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:18.682983 31217 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:18.683029 31051 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:18.661600 30814 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:18.681027 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 553) took 1218 ms. Trace:
I20260709 13:12:18.684866 31234 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:18.685060 28085 rpcz_store.cc:276] 0709 13:12:17.462214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.466508 (+  4294us) service_pool.cc:224] Handling call
0709 13:12:17.504521 (+ 38013us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:17.517253 (+ 12732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.517281 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:17.591564 (+ 74283us) log.cc:844] Serialized 52 byte log entry
0709 13:12:18.510886 (+919322us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:18.511194 (+   308us) log.cc:844] Serialized 14 byte log entry
0709 13:12:18.511389 (+   195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.511463 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.680810 (+169347us) raft_consensus.cc:1685] finished
0709 13:12:18.680829 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.680971 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1818,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":156599,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":7427,"thread_start_us":840092,"threads_started":1,"wal-append.queue_time_us":978140}
I20260709 13:12:18.686348 30814 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:18.686930 31193 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:18.665452 30884 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.683588 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:18.688261 28083 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:18.689090 30768 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:18.689476 30884 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:18.646106 31219 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.650125 31053 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:12:18.692528 31219 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:18.693514 31217 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.694084 28158 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:18.697425 31214 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:18.693310 30804 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:18.699313 28226 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:18.696038 27984 catalog_manager.cc:5697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:18.701108 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:18.687397 28085 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:18.710836 28083 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:18.636636 30543 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.701977 27984 catalog_manager.cc:5697] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 5 to 6, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 6 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:18.698788 30898 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:18.693406 31053 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:18.673481 28078 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.718762 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:18.718986 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:18.719846 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:18.760427 28157 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
W20260709 13:12:18.760810 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 201501us
I20260709 13:12:18.727903 28234 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:18.733189 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:18.734654 31209 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.898s	user 0.013s	sys 0.019s
I20260709 13:12:18.733250 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:18.736402 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:18.737756 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:18.687376 31193 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:18.715917 31117 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:18.747494 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:18.749769 30746 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:18.764274 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:18.717025 30884 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:18.767468 28230 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:18.767760 28228 raft_consensus.cc:2410] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:12:18.766009 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 119757us
W20260709 13:12:18.765534 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49886.1us
W20260709 13:12:18.766340 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 204874us
I20260709 13:12:18.768935 28233 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:12:18.835181 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 63790.4us
I20260709 13:12:18.836109 28157 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
I20260709 13:12:18.837807 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:18.837977 30746 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:18.839509 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:18.849184 31215 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:18.851483 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:18.854410 31246 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:18.776851 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 51849.8us
I20260709 13:12:18.859184 28049 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:18.765560 31053 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:18.770380 31193 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:18.856017 31234 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:18.870901 27984 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 5 to 6, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 6 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:18.885550 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46135.4us
I20260709 13:12:18.885874 27984 catalog_manager.cc:5697] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:18.887111 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:18.890125 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:18.892987 31117 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:18.894923 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:18.768069 31214 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 13:12:18.873230 30891 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:18.906095 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:18.903491 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:18.876595 31217 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:18.877918 31218 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:18.883471 30814 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:18.770530 31236 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 13:12:18.925261 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:18.863255 31246 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:18.930088 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:18.867386 30884 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:18.926381 28125 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 2.012s (SENT)
W20260709 13:12:18.940059 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 53054.9us
I20260709 13:12:18.939661 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:18.940711 28125 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:18.942135 28760 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 8: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:18.912459 31213 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:18.917259 31234 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:18.950153 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:18.940652 30914 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:18.957597 31245 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:18.963115 31245 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:18.963587 31245 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:18.963982 31245 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:18.964592 31200 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.965605 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1982) took 1359 ms. Trace:
I20260709 13:12:18.966109 28159 rpcz_store.cc:276] 0709 13:12:17.606670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.606898 (+   228us) service_pool.cc:224] Handling call
0709 13:12:17.659282 (+ 52384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.651410 (+992128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.965323 (+313913us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b5400255154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:18.965521 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280194304}
I20260709 13:12:18.961290 30543 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:18.969419 31204 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.150s	user 0.000s	sys 0.013s
W20260709 13:12:18.969725 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:18.970541 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.972012 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2056) took 1258 ms. Trace:
I20260709 13:12:18.972205 28160 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:12:18.961714 31242 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:18.961144 31230 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:18.967240 31200 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:18.948630 31213 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:12:18.987251 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:18.941936 31217 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:12:18.965731 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1632) took 1830 ms. Trace:
I20260709 13:12:18.948158 30891 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:18.977097 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 523) took 1819 ms. Trace:
I20260709 13:12:18.972522 28152 rpcz_store.cc:276] 0709 13:12:17.713942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.782724 (+ 68782us) service_pool.cc:224] Handling call
0709 13:12:18.971732 (+1189008us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54003769d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:18.971939 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1906469760}
I20260709 13:12:18.976963 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.016806 31241 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:12:19.013424 30884 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:19.011780 28080 rpcz_store.cc:276] 0709 13:12:17.135320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.170156 (+ 34836us) service_pool.cc:224] Handling call
0709 13:12:17.585504 (+415348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.606288 (+ 20784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:18.965540 (+1359252us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54001ad450. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:18.965676 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935605120}
I20260709 13:12:19.011770 28229 rpcz_store.cc:276] 0709 13:12:17.158081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.158218 (+   137us) service_pool.cc:224] Handling call
0709 13:12:17.158676 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:17.264884 (+106208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.264918 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:17.313659 (+ 48741us) log.cc:844] Serialized 26 byte log entry
0709 13:12:17.517544 (+203885us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:17.517590 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:17.517666 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:18.965130 (+1447464us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b48000600d8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:12:18.968601 (+  3471us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:18.976845 (+  8244us) raft_consensus.cc:1685] finished
0709 13:12:18.976872 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:18.977029 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72744,"raft.run_cpu_time_us":21239,"raft.run_wall_time_us":1732109,"spinlock_wait_cycles":30422144,"thread_start_us":223910,"threads_started":2,"wal-append.queue_time_us":292462}
I20260709 13:12:18.950683 31053 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:19.024773 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 105739us
I20260709 13:12:19.025094 31254 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:19.025619 31254 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:19.026928 31254 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.027551 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1635) took 1891 ms. Trace:
W20260709 13:12:19.029564 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 90006.2us
I20260709 13:12:19.030879 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.032037 28763 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:19.028004 28081 rpcz_store.cc:276] 0709 13:12:17.136397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.140835 (+  4438us) service_pool.cc:224] Handling call
0709 13:12:17.141138 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:18.645513 (+1504375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.027277 (+381764us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540027bfd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:19.027488 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3449616768}
I20260709 13:12:18.926975 31218 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:18.988976 31213 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:18.989504 30746 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:19.042856 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:19.028278 31254 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:19.044399 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 765) took 1892 ms. Trace:
I20260709 13:12:18.998589 31230 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:19.064486 30914 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:18.984977 31234 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.043051 31236 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:19.114471 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.115262 28235 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:12:19.115166 31236 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.074827 31219 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:19.075059 28081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:12:19.116847 28081 raft_consensus.cc:606] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:19.117254 28081 raft_consensus.cc:614] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:19.119115 30898 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:19.119482 31104 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:19.119585 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:19.075897 28761 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 1. Current term is 3. Ops: []
I20260709 13:12:19.120409 30898 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:19.120886 30814 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:19.063868 28758 rpcz_store.cc:276] 0709 13:12:17.151753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.151919 (+   166us) service_pool.cc:224] Handling call
0709 13:12:17.152350 (+   431us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:17.322533 (+170183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.322571 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:18.645383 (+1322812us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b2c00047778. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:18.703844 (+ 58461us) log.cc:844] Serialized 52 byte log entry
0709 13:12:18.759727 (+ 55883us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:18.776478 (+ 16751us) log.cc:844] Serialized 14 byte log entry
0709 13:12:19.039566 (+263088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.039701 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:19.039768 (+    67us) raft_consensus.cc:1685] finished
0709 13:12:19.044102 (+  4334us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.044321 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":72114,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":121343,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":7652,"spinlock_wait_cycles":4939136,"thread_start_us":459532,"threads_started":3,"wal-append.queue_time_us":143696}
I20260709 13:12:19.121467 30814 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:19.078379 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:19.122004 30902 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:19.122507 30902 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:19.122577 30485 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:19.123083 30485 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.123425 30485 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:19.085232 30206 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:19.123485 30902 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.075496 31234 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.090158 31020 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:19.094110 30984 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:19.125480 30984 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:19.122627 30814 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.126117 30902 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:19.096804 31249 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:12:19.097864 31230 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:19.126787 31251 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:12:19.101703 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:19.104987 31248 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:12:19.105959 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.109561 31245 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.112437 31218 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.113648 31053 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.074164 30731 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:19.119999 31104 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.076439 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:19.121183 30768 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:12:19.122313 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44010.9us
W20260709 13:12:19.144852 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 241894us
I20260709 13:12:19.145607 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:19.088770 31187 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.131728 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:19.133124 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.146977 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:19.148203 28762 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.013433 31246 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.143493 31234 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:19.147078 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:19.043859 30746 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:19.150230 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:19.150079 28081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:12:19.156132 28081 raft_consensus.cc:606] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:19.121479 30898 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:19.156666 28081 raft_consensus.cc:614] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:19.127595 31216 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:19.160995 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 205085us
I20260709 13:12:19.167518 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:19.170885 28758 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:19.175508 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.176378 28157 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:19.165069 31259 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.177712 31259 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.127931 30891 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:12:19.179876 31033 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.178030 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 774) took 2001 ms. Trace:
I20260709 13:12:19.184970 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:19.125320 31020 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.177234 31255 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:19.173053 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:19.166440 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:19.181368 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.190954 28152 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:19.172434 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:19.144928 30768 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:19.159492 30746 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.171381 31231 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:19.171418 30914 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:19.202792 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.182078 31215 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:19.171906 30804 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:19.130663 31213 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.150900 30206 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:12:19.179924 31259 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.209393 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:19.185981 31214 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:19.180477 31033 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.214425 31263 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:12:19.184864 28756 rpcz_store.cc:276] 0709 13:12:17.176671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.219763 (+ 43092us) service_pool.cc:224] Handling call
0709 13:12:17.244784 (+ 25021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:17.262628 (+ 17844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.268230 (+  5602us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:17.269813 (+  1583us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:17.281850 (+ 12037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.177658 (+1895808us) spinlock_profiling.cc:243] Waited 6.65 ms on lock 0x7b54007e0254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:19.177711 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.177857 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15290240}
I20260709 13:12:19.216084 31245 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:19.217933 30884 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:19.151422 30731 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.207170 31242 raft_consensus.cc:2764] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:12:19.220824 30804 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:19.212152 31180 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.939s	user 0.008s	sys 0.018s
I20260709 13:12:19.211185 31215 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.216905 30768 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.225935 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.232877 31213 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:19.246477 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.232904 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:19.248984 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 75752.8us
I20260709 13:12:19.126626 30984 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.272512 31020 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:19.273281 30984 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:19.272598 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 59606.3us
I20260709 13:12:19.273901 31187 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:19.258818 31263 consensus_queue.cc:1243] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:19.274384 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.242494 28226 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:19.210255 31200 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:19.210815 30543 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:19.256593 28080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:12:19.279004 28080 raft_consensus.cc:606] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:19.248301 31153 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.628s	user 0.018s	sys 0.022s
I20260709 13:12:19.254151 30884 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:19.262202 30914 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:19.237629 31231 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:19.296240 30914 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:19.297222 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.296957 31231 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.297830 28155 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:19.250666 30892 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:19.300055 30892 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:19.300864 30884 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:19.301155 31263 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:19.302389 31263 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:19.302825 31263 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:19.300509 30892 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.271375 28086 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:19.275235 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 72584.2us
I20260709 13:12:19.258054 31214 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:19.312464 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.312970 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:19.313558 28156 raft_consensus.cc:2410] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:12:19.313432 31214 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.314522 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:19.314680 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.315478 31257 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:19.316087 31257 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:19.316143 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.316393 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:19.277379 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 111003us
I20260709 13:12:19.280835 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.282361 30804 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:19.284782 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:19.285388 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.346405 28232 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:19.285897 31245 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:19.287004 31187 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:19.287343 30543 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:19.278710 30891 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:19.287403 31193 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.356674 31051 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:19.357174 31051 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:19.357193 28084 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 13:12:19.358037 31051 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.358588 31193 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:19.289496 31033 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.290271 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:19.269313 31215 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:19.269838 28233 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:19.305264 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 1989) took 1494 ms. Trace:
I20260709 13:12:19.306537 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.308202 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.311349 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:19.311764 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:19.320297 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.367549 28153 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.343222 28155 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:19.331748 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:19.292109 31153 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.701s	user 0.019s	sys 0.022s
I20260709 13:12:19.348932 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:19.271225 30898 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:12:19.353921 28231 raft_consensus.cc:2410] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:12:19.329891 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:19.360028 28227 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:19.300081 31155 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:19.363575 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:19.359268 31051 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:12:19.317067 31213 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.316947 31242 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:19.372169 28078 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:19.372428 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55122.1us
I20260709 13:12:19.365284 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.378433 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:19.270388 30485 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.376432 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.377238 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:19.380043 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.381036 31252 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:12:19.381227 31213 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.382174 31252 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:19.382572 31252 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:19.356705 31104 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.385878 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:19.386533 28157 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.387777 28229 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:12:19.383545 31213 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.389595 31193 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:19.397112 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.397702 31267 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.397974 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.398918 28232 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:19.400355 31245 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:19.400925 31245 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.401342 31245 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:19.402907 31238 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.763s	user 0.004s	sys 0.000s
I20260709 13:12:19.403491 30814 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:19.407612 31074 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:19.408113 31074 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:19.398367 31267 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.366380 28161 rpcz_store.cc:276] 0709 13:12:17.810358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.810491 (+   133us) service_pool.cc:224] Handling call
0709 13:12:17.831254 (+ 20763us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:17.846875 (+ 15621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:17.846929 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:19.096305 (+1249376us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b2c000b54b8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:19.096842 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:12:19.206191 (+109349us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:19.206245 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:19.206321 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:19.292484 (+ 86163us) raft_consensus.cc:1685] finished
0709 13:12:19.292505 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:19.305154 (+ 12649us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7241,"raft.run_cpu_time_us":509,"raft.run_wall_time_us":34603,"spinlock_wait_cycles":7126400,"thread_start_us":108837,"threads_started":1,"wal-append.queue_time_us":191099}
I20260709 13:12:19.413602 28762 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:19.416922 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.417461 28153 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:19.421756 28078 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:12:19.383576 31252 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:19.424907 31221 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:12:19.378571 31033 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:19.426833 31033 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.427217 31033 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:19.427841 30884 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.390532 31020 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:19.418594 28124 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:19.434387 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.435989 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:19.436805 28229 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:19.440975 28761 raft_consensus.cc:2368] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:19.448386 28155 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:12:19.449190 28084 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:19.448464 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.430869 30884 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.380138 31231 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:19.384351 31242 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:19.455396 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.457109 28161 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:19.457528 28155 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:19.458006 28155 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:19.392405 30543 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.459740 31242 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.308979 31263 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.462212 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:19.464195 28161 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:19.465483 30731 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.466156 31267 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.467655 31200 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:19.461731 30892 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.475997 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:19.479585 28157 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:19.455302 30891 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:19.485952 28082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1658) took 2149 ms (client timeout 1913 ms). Trace:
I20260709 13:12:19.487386 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:19.486675 28082 rpcz_store.cc:269] 0709 13:12:17.326799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.326918 (+   119us) service_pool.cc:224] Handling call
0709 13:12:19.476463 (+2149545us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540017c354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:19.476556 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2807152512}
W20260709 13:12:19.488493 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 110178us
I20260709 13:12:19.492414 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:19.492995 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:19.494660 30768 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:19.494829 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:19.497051 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.482729 31200 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:19.499270 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 765) took 1469 ms. Trace:
I20260709 13:12:19.499001 31200 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.499624 28763 rpcz_store.cc:276] 0709 13:12:18.029989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:18.030413 (+   424us) service_pool.cc:224] Handling call
0709 13:12:19.499064 (+1468651us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54004d21d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:19.499193 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1208411392}
I20260709 13:12:19.500205 31200 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:19.501704 31104 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:19.319254 31257 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:19.502264 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.482472 31074 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.510746 31245 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.513105 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:19.513761 28760 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:19.518558 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 244226us
I20260709 13:12:19.513883 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:19.491338 30891 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:19.528270 30485 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.529418 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:19.529239 30891 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.531412 31269 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:12:19.532204 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.497360 28155 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.536327 28760 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:19.538657 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.541641 31234 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:19.542191 31234 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:19.517375 30814 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:19.549571 28124 leader_election.cc:400] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:19.551338 28124 leader_election.cc:403] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:19.550256 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.552343 31234 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:19.556423 28155 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:19.561919 30892 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.520009 30746 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.564827 31263 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:19.566340 31215 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.569216 31267 raft_consensus.cc:993] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:12:19.569766 31267 raft_consensus.cc:1081] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:12:19.570850 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:19.572883 31236 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.566774 30746 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:19.574095 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.565193 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.537806 31117 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:12:19.552636 31269 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:19.593729 28158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2059) took 1720 ms (client timeout 2161 ms). Trace:
I20260709 13:12:19.556008 31153 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 13:12:19.594144 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 104495us
I20260709 13:12:19.586364 31104 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:19.603952 28158 rpcz_store.cc:269] 0709 13:12:17.873160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.873329 (+   169us) service_pool.cc:224] Handling call
0709 13:12:19.593571 (+1720242us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b5400317ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:19.593676 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1576033536}
I20260709 13:12:19.618005 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:19.587004 31215 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.589689 30804 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:19.627753 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:19.591455 31273 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:12:19.625975 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 103646us
I20260709 13:12:19.636114 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.626310 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.626390 28155 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:12:19.639891 28155 pending_rounds.cc:85] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 3
I20260709 13:12:19.640424 28155 pending_rounds.cc:107] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:12:19.642198 28155 raft_consensus.cc:2889] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:19.634279 28229 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:19.618980 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.587172 31242 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.646394 31242 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:19.651618 31033 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.607547 30814 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:19.655593 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:19.656282 28161 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:19.657336 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.658645 28230 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
W20260709 13:12:19.670459 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 71039.1us
I20260709 13:12:19.671276 28050 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:19.672351 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 159239us
I20260709 13:12:19.672932 28050 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:19.673136 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.675045 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:19.675809 28155 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.681391 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:19.682618 28229 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:19.687942 28232 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 8. Ops: []
I20260709 13:12:19.689865 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.692107 28228 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:12:19.669035 31218 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:19.690910 28227 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:19.703339 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:19.705478 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 69497.3us
I20260709 13:12:19.706413 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:19.677310 31215 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:19.713997 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.710304 28155 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:19.716984 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:19.678604 30914 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:19.678921 30814 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:19.709550 28762 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 7. Ops: []
I20260709 13:12:19.707373 28231 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:19.695497 31267 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:19.725679 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:19.726131 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:19.726320 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:19.729986 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:19.702180 30819 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:19.736819 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.699579 31218 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:19.736413 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.735522 28125 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:19.776135 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46220.8us
W20260709 13:12:19.778421 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.781474 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:19.783881 28228 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:19.726917 31155 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:19.820189 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:19.794378 28233 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:19.843497 28232 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:19.733634 30814 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:19.848455 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.877862 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.878858 28229 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:19.739661 30819 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:19.905181 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 174947us
W20260709 13:12:19.905732 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 149173us
I20260709 13:12:19.907626 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.916075 28229 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:19.917366 30814 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:19.891211 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:19.821569 31275 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
W20260709 13:12:19.941037 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:19.823468 30914 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:19.918426 31213 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.784405 31267 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:19.977952 31213 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.978565 31267 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:19.980320 31213 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:19.735634 31269 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:19.919435 31281 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:12:19.926921 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:19.985387 31282 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:12:19.985486 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 58631us
I20260709 13:12:19.986929 31260 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:19.987453 31260 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:19.904714 31270 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:12:19.904918 31218 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.988377 31260 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.995412 31218 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.995615 31234 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:19.880493 28756 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:19.736291 31219 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.171s
I20260709 13:12:19.779693 31245 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:19.906806 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:19.820441 31263 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.784264 31215 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:19.905671 28157 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:19.940896 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:20.001102 28159 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:19.787187 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 51750.3us
I20260709 13:12:19.917876 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:19.944033 28226 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:19.945226 28158 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:12:19.875293 30876 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:12:19.907662 30819 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:19.974291 28760 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:19.871625 30891 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.813724 30804 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:19.975556 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:19.991348 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:20.006134 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 88369.8us
I20260709 13:12:20.008414 28762 raft_consensus.cc:2376] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7. Re-sending same reply.
I20260709 13:12:20.010092 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:20.010848 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:20.011615 28230 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:12:20.012317 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:20.015990 28125 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:20.017313 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 76480.7us
I20260709 13:12:20.018400 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:20.015520 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.022702 28226 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:20.010468 31246 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:20.037037 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49316.5us
I20260709 13:12:20.037624 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:20.010080 31215 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:20.038345 28226 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:20.038349 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:20.025118 31033 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:20.032955 30884 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:12:19.998481 31266 consensus_queue.cc:1175] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "11f10c12c03e47dcac88b050d4c1431b" responder_term: 8 status { last_received { term: 8 index: 8 } last_committed_idx: 7 last_received_current_leader { term: 8 index: 8 } }
I20260709 13:12:20.010246 31213 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:12:20.032815 31153 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:20.040551 31219 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:20.009829 30891 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.047034 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:20.052973 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:20.059787 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.060508 28158 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:20.009038 31263 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.028851 31053 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:12:20.065367 28227 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:20.046284 31254 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:20.056479 31210 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:20.061035 28228 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:20.051437 30884 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.072502 31210 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:20.073016 30884 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:20.073340 30891 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:20.073617 31210 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.075516 31263 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:20.069852 31215 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.070858 31273 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:20.081809 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:20.084137 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:20.085541 28158 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:20.091493 28158 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:20.092427 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:20.093309 28230 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:20.095501 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:20.098414 28161 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:20.098027 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:20.103147 31104 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:20.117692 30731 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:20.120532 30731 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:20.121311 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.126142 28231 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:20.128417 28160 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:20.132948 28762 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:12:20.133296 28762 pending_rounds.cc:85] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 7
I20260709 13:12:20.133525 28762 pending_rounds.cc:107] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:12:20.133716 28762 raft_consensus.cc:2889] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:20.134021 28762 pending_rounds.cc:107] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:12:20.134204 28762 raft_consensus.cc:2889] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:20.135413 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:20.135982 28234 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:20.125191 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.137594 28159 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:20.139590 28226 raft_consensus.cc:2410] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:12:20.056708 31276 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:12:20.145896 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:20.148948 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.151527 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:20.154145 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.154937 28226 raft_consensus.cc:2376] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9. Re-sending same reply.
I20260709 13:12:20.144805 31213 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
W20260709 13:12:20.157835 31274 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.676s	user 0.005s	sys 0.000s
I20260709 13:12:20.161809 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:20.163530 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.170816 28234 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:20.177589 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.094708 30804 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.153416 31245 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:20.147373 30819 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.158022 30884 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.150844 31053 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:12:20.147858 31217 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:12:20.172269 31215 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:20.182751 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:20.189220 31193 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:20.189644 31193 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:20.190060 31193 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:20.194037 30884 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:20.173058 31216 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:20.185087 31280 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:20.185658 31276 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:20.194805 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:20.179150 30731 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.233482 31276 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:20.233918 31276 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:20.234288 31276 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:20.234666 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:20.236850 31200 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:20.237205 31237 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:20.237335 31200 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.237666 31237 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:20.237826 31200 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:20.238564 31237 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.239418 28761 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:20.239841 31237 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:20.235890 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:20.242408 31051 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:20.242677 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:20.243427 31255 raft_consensus.cc:2764] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:12:20.244108 31255 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:20.244318 31216 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:20.244493 31255 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:20.245009 31255 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:20.245430 31216 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:20.202486 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.249902 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:20.250485 28230 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:20.222579 31234 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:20.252472 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:20.224431 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:20.253153 28228 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:20.226192 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 210272us
I20260709 13:12:20.226872 30914 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:20.183144 31245 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:20.250517 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.257997 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:20.218526 30819 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:20.255044 28760 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:20.268559 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:20.253949 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.266106 28230 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:20.279443 28229 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:20.280766 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:20.281522 28229 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:20.282725 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:20.291906 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:20.279393 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 123799us
I20260709 13:12:20.299431 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:20.300601 30892 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:20.301370 30892 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:20.302579 30892 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:20.319998 31242 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.006s	sys 0.005s
W20260709 13:12:20.320465 31242 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.006s	sys 0.005s
I20260709 13:12:20.323710 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:20.325403 28227 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:12:20.278906 30876 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.062s	user 0.004s	sys 0.000s
I20260709 13:12:20.328297 30876 tablet_bootstrap.cc:492] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:20.331359 30814 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:12:20.333782 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48676.2us
I20260709 13:12:20.335338 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:20.344717 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:20.345693 28160 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:20.284716 31234 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.292137 28234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39712
I20260709 13:12:20.352262 28234 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:20.352792 28234 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:20.353299 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:20.353878 28161 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:20.360025 28160 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:20.295617 30902 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:20.295126 31215 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:20.284330 31155 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:20.298203 30819 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:20.336632 31217 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:20.382166 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:20.387483 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:20.295774 31273 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:12:20.348670 31280 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:20.394587 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 140415us
I20260709 13:12:20.346278 30884 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:12:20.398533 28159 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:20.398589 28229 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:20.392499 31117 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:20.368468 31234 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.393378 30819 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:20.407617 30876 tablet_bootstrap.cc:492] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:12:20.423556 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:20.410089 28234 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.408869 31210 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:20.390506 31155 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.423327 31117 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:20.431109 30819 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.417248 30814 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:12:20.431332 30898 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:20.415197 31276 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.432416 31117 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.436169 31276 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:20.437039 31252 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:20.439455 28230 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:20.374754 31245 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.446265 28161 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:20.401443 31255 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:20.450073 31308 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:12:20.453168 31301 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:12:20.461212 28234 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:20.467478 28157 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:20.472505 31214 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.447811 31210 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.473129 31214 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.475207 31210 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:20.477037 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.477339 28078 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:20.477839 28229 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:20.478796 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.426679 31193 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.483982 31193 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:20.484812 30876 ts_tablet_manager.cc:1403] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 2.098s	user 0.041s	sys 0.034s
I20260709 13:12:20.485716 28235 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:20.444147 31242 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:20.495317 28228 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:20.496704 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:12:20.487013 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 108833us
I20260709 13:12:20.487284 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:20.448457 30898 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.490830 30876 raft_consensus.cc:359] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:20.493338 28231 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:20.485916 31299 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:12:20.505312 30898 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.507565 30898 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:20.510058 31255 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:20.511991 31155 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:20.513202 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.514245 28234 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:20.514118 31053 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:20.516223 31053 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:20.517343 31053 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.513247 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:20.521003 28234 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:20.521255 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:20.522313 31255 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:20.523242 31221 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:20.523881 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:20.524396 31221 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:20.524842 31221 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:20.525972 31221 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.526563 31217 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:20.527446 31221 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:20.524887 31255 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:20.529042 31255 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:20.529592 31255 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:20.531950 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 108455us
I20260709 13:12:20.525985 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:20.541538 28125 leader_election.cc:400] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9.
I20260709 13:12:20.549708 28125 leader_election.cc:403] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:20.551697 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:20.475247 31214 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:20.557822 31273 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9.
I20260709 13:12:20.497201 31242 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:20.605921 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64536.1us
W20260709 13:12:20.570704 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:20.625269 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.534665 30898 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.548024 30876 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:12:20.664489 31242 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:20.675966 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:20.652480 31321 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:20.668823 31074 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:20.689152 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:20.558303 30984 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 13:12:20.695925 28079 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:20.696946 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:20.652803 31316 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:20.703249 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:20.707438 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:20.710240 28125 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:20.711390 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:20.711692 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:20.712656 28077 raft_consensus.cc:2410] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:20.716249 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:20.683102 31215 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:20.720289 28079 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:20.665182 31200 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:12:20.723697 28199 connection.cc:256] RPC call timeout handler was delayed by -2.1452337579999039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 13:12:20.729195 28199 connection.cc:256] RPC call timeout handler was delayed by -2.0973763360000248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21610000000000001s for a response
W20260709 13:12:20.729698 28199 connection.cc:256] RPC call timeout handler was delayed by -1.4610763359999055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2868s for a response
I20260709 13:12:20.734404 28230 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:20.734977 28230 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:20.742585 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:20.750612 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:20.763350 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:20.764535 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:20.743345 28230 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.767351 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:20.764460 28079 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:20.770629 28230 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:20.774525 28078 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:20.770748 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:20.681315 30898 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.782459 30898 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:20.783160 28086 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:20.783321 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:20.771593 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 61449.5us
I20260709 13:12:20.770229 31193 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:20.717172 30876 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:20.697528 31321 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.721372 30902 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:12:20.718628 31260 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:12:20.790402 31321 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:20.791515 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:20.718951 31293 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:12:20.792583 28228 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:20.719355 31273 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:12:20.798476 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:20.800004 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 100910us
I20260709 13:12:20.816809 28230 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:20.818279 28086 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:20.724922 31316 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.836547 28078 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:20.838083 28084 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:20.768906 30768 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:20.846094 28086 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:20.847225 28762 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:20.851686 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:20.843308 30768 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:20.859198 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.859941 28229 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:20.860625 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:20.863565 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:20.863653 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:20.865413 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:20.866178 28158 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:20.867220 28085 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:20.867604 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:20.868234 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 100075us
I20260709 13:12:20.868891 28158 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:20.868983 28086 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: []
I20260709 13:12:20.871107 28230 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:20.871639 28230 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:20.872328 28230 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.868595 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:20.868909 28199 leader_election.cc:336] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.161s (SENT)
I20260709 13:12:20.875968 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:20.878008 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:20.881064 28230 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:20.884408 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:20.903925 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:20.904239 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:20.904958 28158 raft_consensus.cc:2368] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7.
I20260709 13:12:20.903925 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:20.907814 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:20.908577 28158 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:12:20.909097 28159 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:20.914812 28153 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:12:20.922788 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:20.867513 30768 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:20.924482 28152 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:20.928329 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:20.929435 28153 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:12:20.930656 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 61764.6us
I20260709 13:12:20.934985 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:20.927510 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 82580.1us
I20260709 13:12:20.883664 31020 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.853343 31255 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:20.941223 28762 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:20.924093 31276 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:20.962463 28231 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:20.925962 31263 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:12:20.950161 31020 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:20.969969 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:20.970186 31322 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:12:20.970745 28233 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:20.971091 30884 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:20.974254 28079 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:20.978617 31316 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:21.009994 31328 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
W20260709 13:12:21.011323 31310 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.583s	user 0.003s	sys 0.000s
W20260709 13:12:21.012053 31326 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:12:21.012925 30884 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:21.007850 31074 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.013978 30884 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:20.924259 31053 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:12:20.983760 28234 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:21.019524 28153 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:20.985688 30876 ts_tablet_manager.cc:1434] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.500s	user 0.015s	sys 0.006s
I20260709 13:12:20.987188 28084 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:20.988519 28231 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:20.991804 28229 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:20.992918 28158 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:20.999694 28230 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:20.979164 28082 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:21.017771 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:21.018467 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 90607.2us
I20260709 13:12:20.984563 28077 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:21.020277 28228 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:20.984923 30768 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:21.027099 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:21.028280 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:21.014202 31074 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.030539 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:21.031637 28159 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:21.024273 28231 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:21.023433 31330 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:12:21.035117 28231 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.037097 28158 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:21.037120 28230 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:21.023044 31200 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:21.022205 30876 ts_tablet_manager.cc:1467] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Tablet startup took 536ms
I20260709 13:12:21.040619 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:21.041623 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:21.042168 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.043610 28082 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:12:21.044134 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.044099 28077 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:21.046087 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.046640 28084 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:21.053773 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:21.055190 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:21.056120 28231 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:21.058255 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:21.060276 28082 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:21.052516 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:21.061856 28158 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:21.068956 31321 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:21.070303 31321 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:21.071009 31321 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:21.072341 31321 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:21.029670 31263 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:12:21.087985 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:21.051637 31316 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:21.089988 28158 raft_consensus.cc:2368] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7.
I20260709 13:12:21.114389 28159 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:21.117287 28234 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:21.062561 31200 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:21.088820 31020 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:21.151270 31334 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 13:12:21.152271 31343 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:12:21.162626 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 129422us
I20260709 13:12:21.153752 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:21.164297 28159 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:21.147370 31276 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:21.177448 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:21.178340 28159 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:21.181623 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:21.182263 28158 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:21.165184 30884 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
W20260709 13:12:21.187532 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:21.190708 31329 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:12:21.194355 31293 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.195080 31293 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:21.193223 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 135263us
I20260709 13:12:21.197845 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:21.198747 28234 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:21.201804 31332 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.202780 31332 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:21.222585 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.172715 30984 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:12:21.226356 31293 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:21.233675 31242 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 13:12:21.234202 31242 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.235603 28226 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:21.217934 31255 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:21.236938 30898 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:21.239483 31332 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:21.043382 30876 ts_tablet_manager.cc:1469] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Trace:
0709 13:12:08.420783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:08.426813 (+  6030us) service_pool.cc:224] Handling call
0709 13:12:09.126148 (+699335us) ts_tablet_manager.cc:934] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
0709 13:12:09.990147 (+863999us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59cb56d0ac 00007f59d43c7ab9 00007f59cb58969b 00007f59cb586caa 00007f59cb586bd2 00007f59d35eb131
0709 13:12:11.698088 (+1707941us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:12:12.355452 (+657364us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:12.363417 (+  7965us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:12:13.299604 (+936187us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:12:13.299992 (+   388us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:13.300404 (+   412us) ts_tablet_manager.cc:989] Replica registered.
0709 13:12:13.300542 (+   138us) inbound_call.cc:177] Queueing success response
0709 13:12:13.838126 (+537584us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b1800408ee0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb42eed8 00007f59cb589de2 00007f59d35eed23 00007f59d35ee55d 00007f59d368a3d8 00007f59d36bbf4e 00007f59d36a0fdd
0709 13:12:18.382938 (+4544812us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:12:20.485338 (+2102400us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:12:20.488117 (+  2779us) tablet_replica.cc:261] Starting instrumentation
0709 13:12:20.488820 (+   703us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":26663,"mutex_wait_us":33143,"raft.queue_time_us":30374,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":22782,"spinlock_wait_cycles":123641984,"tablet-copy.queue_time_us":543575,"tablet-download-pool.queue_time_us":5200,"tablet-download-pool.run_cpu_time_us":2135,"tablet-download-pool.run_wall_time_us":3752584,"thread_start_us":1585037,"threads_started":2}
I20260709 13:12:21.240756 31335 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.168421 31345 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:12:21.241357 31335 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:21.243561 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.244282 31234 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:21.244516 31335 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:21.219194 31263 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:12:21.246868 31242 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:21.247879 28158 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:21.269752 28763 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:21.271596 28763 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:21.271814 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 213647us
I20260709 13:12:21.270092 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:21.272876 31316 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:21.266724 31335 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:21.275208 31255 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:21.275545 31200 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:21.277314 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:21.278419 31255 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:21.278828 31255 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:21.279798 31255 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:21.291113 28079 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:12:21.291635 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:21.284446 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.293061 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.293651 28077 raft_consensus.cc:2368] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8.
I20260709 13:12:21.294432 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.295383 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.295930 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:21.295989 28079 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:21.297942 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.304452 28084 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:21.269939 28756 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:21.269309 31349 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.282920 31255 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:21.281248 31263 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.319465 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.306943 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:21.321288 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.325037 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.331463 28085 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:21.282222 31200 consensus_queue.cc:1243] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:12:21.328655 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.306296 31316 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:21.334775 28232 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:21.274912 31335 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.342214 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:21.338531 31316 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:21.341952 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64717.6us
I20260709 13:12:21.307559 31020 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:21.344521 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:21.348313 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.346010 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:21.355170 31200 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:21.355528 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:21.356352 28084 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:12:21.356594 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:21.358039 31350 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:21.358551 31350 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:21.360455 28234 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:21.360994 28234 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:21.345798 31332 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:21.336404 31263 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.349049 31242 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:21.362533 31350 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:21.375155 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.355837 31200 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:21.376168 28081 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:21.377140 28234 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:21.367030 31332 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:21.384552 28234 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:21.349262 31020 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.389796 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.390738 28086 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:21.333817 31349 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.388401 31332 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:21.413018 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.414526 28080 raft_consensus.cc:2410] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:12:21.414809 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2012) took 1164 ms. Trace:
I20260709 13:12:21.411399 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:21.417709 31242 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:21.416698 28235 rpcz_store.cc:276] 0709 13:12:20.250565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.253634 (+  3069us) service_pool.cc:224] Handling call
0709 13:12:20.278273 (+ 24639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:20.493146 (+214873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:21.414611 (+921465us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54001f4ad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:21.414759 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474417536}
I20260709 13:12:21.417209 28081 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:21.419009 31242 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:21.409284 31355 raft_consensus.cc:2764] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:12:21.424408 28230 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:21.440210 28234 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:21.429092 30819 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.454448 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:21.455152 28085 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:21.454453 28086 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 8. Ops: []
W20260709 13:12:21.522851 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 166319us
I20260709 13:12:21.523538 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:21.526206 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:21.529808 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.540982 31193 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:21.541594 31193 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:21.545599 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:21.622502 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66953.6us
W20260709 13:12:21.521960 31326 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:12:21.639842 31361 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:12:21.642143 31349 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:21.642786 31313 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.214s	user 0.003s	sys 0.020s
W20260709 13:12:21.643412 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:21.644822 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.648048 31346 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.651881 30902 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:21.409610 31316 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:21.542550 30884 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:21.652470 30902 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:21.544948 31355 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:21.677855 31350 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:21.678709 31365 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:21.679766 31367 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:12:21.680146 31362 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:12:21.691659 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 551) took 1635 ms. Trace:
I20260709 13:12:21.694144 28763 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.697703 31364 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:21.664667 31346 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.707068 31074 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:21.450847 31200 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:21.538558 30819 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.748957 31346 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:21.741386 28233 rpcz_store.cc:276] 0709 13:12:20.055865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.056129 (+   264us) service_pool.cc:224] Handling call
0709 13:12:20.069294 (+ 13165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.001931 (+932637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:21.691312 (+689381us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b54000e8b54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:21.691572 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2072405760}
I20260709 13:12:21.542524 31193 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:12:21.751750 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:21.752187 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.745333 28763 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:21.733588 31350 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:21.754050 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2031) took 1383 ms. Trace:
I20260709 13:12:21.754516 28160 rpcz_store.cc:276] 0709 13:12:20.370444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.370734 (+   290us) service_pool.cc:224] Handling call
0709 13:12:20.371152 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:20.392335 (+ 21183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:20.392380 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:20.392438 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:20.392482 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:21.753822 (+1361340us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b54000a2dd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:21.753869 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:21.753991 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48072960}
I20260709 13:12:21.756839 28234 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:21.754099 28233 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:21.720929 31335 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:21.761886 28231 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:21.720570 31355 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:21.768385 31355 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:21.771162 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.775592 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:21.776448 28231 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
W20260709 13:12:21.782492 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.790206 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:21.790361 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:21.790431 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 240005us
I20260709 13:12:21.793963 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:21.793998 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:21.794673 28158 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:21.794917 28160 raft_consensus.cc:2410] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:21.795949 28157 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:21.757850 31293 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:21.820923 28231 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:21.823148 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:21.823879 28157 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:21.776767 31263 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:21.845247 28763 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:21.830497 30902 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:21.847606 30902 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:12:21.852054 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:21.855098 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.858682 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:21.859926 28160 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:21.861642 28158 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:21.862629 31213 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:21.863125 31213 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:21.863771 28158 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:21.863822 28156 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:21.864532 28156 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:21.864589 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:21.865286 28152 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
W20260709 13:12:21.868680 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 414307us
W20260709 13:12:21.869369 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.894786 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:21.900682 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2023) took 1566 ms. Trace:
W20260709 13:12:21.906735 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.908027 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:21.909020 28160 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 2.
I20260709 13:12:21.914278 28233 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:21.740866 31365 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.856650 31020 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:21.831934 31293 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.744089 31367 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.762228 31364 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:21.864008 31213 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:22.079342 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.901392 28227 rpcz_store.cc:276] 0709 13:12:20.334055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.334211 (+   156us) service_pool.cc:224] Handling call
0709 13:12:20.334671 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:20.335099 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:20.335467 (+   368us) log.cc:844] Serialized 14 byte log entry
0709 13:12:21.702889 (+1367422us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b1800489be0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a
0709 13:12:21.703129 (+   240us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:21.703636 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:12:21.703800 (+   164us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:21.703839 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:21.703914 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:21.859692 (+155778us) raft_consensus.cc:1685] finished
0709 13:12:21.900356 (+ 40664us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:21.900587 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":206533,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":2094,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":194,"spinlock_wait_cycles":3586944,"thread_start_us":221257,"threads_started":1,"wal-append.queue_time_us":430959}
W20260709 13:12:21.997471 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.918926 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:21.950907 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:21.920331 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:22.138162 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:21.920553 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 45935.1us
I20260709 13:12:21.919684 28156 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:22.177261 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 220655us
W20260709 13:12:22.253325 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.175551 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:22.407612 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:22.399979 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:22.488319 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:21.994283 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:22.154402 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:22.384949 31346 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.016s	sys 0.005s
I20260709 13:12:22.188880 31193 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.396674 31242 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:22.372309 31376 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:21.980790 31074 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.141363 28160 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:21.973006 31316 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:22.037246 31357 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
W20260709 13:12:22.000080 31292 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:12:22.089699 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-b99bb21dbacb4d63839f1403ef9b08b9 received from {username='slave'} at 127.0.0.1:36768
I20260709 13:12:22.537356 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:22.000694 31342 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.500s
I20260709 13:12:22.479828 28158 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:22.481980 31276 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:22.251941 31375 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:12:22.509815 31020 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:22.437112 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 518264us
I20260709 13:12:22.556358 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:22.558005 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:22.126519 31377 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.559942 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:22.494415 28078 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:22.530995 31074 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.440847 31213 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:22.424088 31187 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:22.460489 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.176569 31293 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:22.506136 28157 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:22.574478 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 561) took 1303 ms. Trace:
I20260709 13:12:22.465780 31350 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:22.523576 31193 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.144186 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:22.465981 31364 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.519601 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:22.520146 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:22.092624 30984 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.403s
I20260709 13:12:22.480408 31263 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:22.501176 31381 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:12:22.115193 31335 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.380s
W20260709 13:12:22.502969 31355 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.734s	user 0.012s	sys 0.004s
I20260709 13:12:22.178054 30814 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.336s
I20260709 13:12:22.533789 31316 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.535140 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:22.000552 31380 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.522s
I20260709 13:12:22.188515 31370 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.363s
I20260709 13:12:21.938714 31214 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.493218 28156 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:22.561614 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:22.600085 28078 raft_consensus.cc:2368] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5.
I20260709 13:12:22.140370 31365 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.562521 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:22.563086 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:22.547272 31332 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.116s	user 0.014s	sys 0.000s
I20260709 13:12:22.577183 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:22.436251 31378 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:22.603839 28158 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:22.574925 28758 rpcz_store.cc:276] 0709 13:12:21.271525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.272128 (+   603us) service_pool.cc:224] Handling call
0709 13:12:21.306486 (+ 34358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:21.306847 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:21.306875 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:21.313366 (+  6491us) log.cc:844] Serialized 26 byte log entry
0709 13:12:21.338394 (+ 25028us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:21.338453 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:21.338549 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:22.573987 (+1235438us) raft_consensus.cc:1685] finished
0709 13:12:22.574014 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:22.574410 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7080,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":8594,"thread_start_us":24648,"threads_started":1,"wal-append.queue_time_us":78810}
I20260709 13:12:22.524226 31242 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.579504 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:22.579046 28199 leader_election.cc:400] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7.
I20260709 13:12:22.612522 28199 leader_election.cc:403] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:12:22.527127 31376 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.564381 31200 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:22.590963 28160 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:22.584839 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-b99bb21dbacb4d63839f1403ef9b08b9 on tablet b99bb21dbacb4d63839f1403ef9b08b9 with peer 70996712c3c942468a6006602de6477f
I20260709 13:12:22.559402 31276 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.582899 31350 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:22.585050 31364 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:22.611070 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2030) took 1698 ms. Trace:
I20260709 13:12:22.562978 31020 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.578564 31187 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.615877 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2077) took 1673 ms. Trace:
I20260709 13:12:22.624411 28153 rpcz_store.cc:276] 0709 13:12:20.942152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.995552 (+ 53400us) service_pool.cc:224] Handling call
0709 13:12:21.019352 (+ 23800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.019963 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.615666 (+1595703us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b5400376ed0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:22.615822 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38555520}
W20260709 13:12:22.616489 28229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1721) took 1886 ms (client timeout 1561 ms). Trace:
W20260709 13:12:22.625811 28229 rpcz_store.cc:269] 0709 13:12:20.730459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.739152 (+  8693us) service_pool.cc:224] Handling call
0709 13:12:22.616326 (+1877174us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b54001c70d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:22.616426 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193268480}
I20260709 13:12:22.623191 28228 rpcz_store.cc:276] 0709 13:12:20.912183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.912390 (+   207us) service_pool.cc:224] Handling call
0709 13:12:20.938770 (+ 26380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.026342 (+ 87572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.610773 (+1584431us) spinlock_profiling.cc:243] Waited 81.2 ms on lock 0x7b54000dedd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:22.610977 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186725248}
I20260709 13:12:22.618782 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:22.566097 31377 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.601567 31367 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.613911 31346 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.667013 28156 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:12:22.667429 28156 pending_rounds.cc:85] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 2
I20260709 13:12:22.667654 28156 pending_rounds.cc:107] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:12:22.668254 28156 raft_consensus.cc:2889] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:22.629895 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:22.632349 31385 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:12:22.635109 31020 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.636754 31276 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.636814 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 554) took 1945 ms. Trace:
I20260709 13:12:22.637077 31386 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.637737 31397 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:22.641551 31408 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:12:22.645732 31321 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:12:22.648154 30819 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.649009 31396 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.649096 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 560) took 1383 ms. Trace:
I20260709 13:12:22.649681 31215 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:22.651101 30768 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:22.652467 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1683) took 1346 ms. Trace:
I20260709 13:12:22.607347 31214 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.653443 28763 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 3.
I20260709 13:12:22.654896 31376 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:22.655813 31335 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:22.621779 31365 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7.
I20260709 13:12:22.599898 31263 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.657873 31193 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.658066 31388 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:22.658469 31387 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:22.659287 31074 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.660687 31200 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:22.596061 31355 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:22.669778 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.671423 28226 rpcz_store.cc:276] 0709 13:12:20.690940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:20.691062 (+   122us) service_pool.cc:224] Handling call
0709 13:12:20.691524 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.236001 (+544477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.636544 (+1400543us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b5400305554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:22.636750 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250802560}
W20260709 13:12:22.674721 31379 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:12:22.675089 31406 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:22.758436 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1691) took 1327 ms. Trace:
I20260709 13:12:22.758652 31406 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:22.675871 31383 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:12:22.758960 28081 rpcz_store.cc:276] 0709 13:12:21.436555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.440829 (+  4274us) service_pool.cc:224] Handling call
0709 13:12:21.441223 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:21.666230 (+225007us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:21.666264 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:21.673407 (+  7143us) log.cc:844] Serialized 26 byte log entry
0709 13:12:21.907378 (+233971us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:22.389142 (+481764us) log.cc:844] Serialized 14 byte log entry
0709 13:12:22.427604 (+ 38462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.427718 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:22.758160 (+330442us) raft_consensus.cc:1685] finished
0709 13:12:22.758186 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:22.758362 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":265382,"prepare.queue_time_us":340174,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":137799,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":3832,"thread_start_us":637640,"threads_started":3,"wal-append.queue_time_us":904496}
W20260709 13:12:22.676297 30902 consensus_meta.cc:287] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.828s	user 0.001s	sys 0.009s
I20260709 13:12:22.763203 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2067) took 1097 ms. Trace:
I20260709 13:12:22.763692 28085 rpcz_store.cc:276] 0709 13:12:21.666029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.666193 (+   164us) service_pool.cc:224] Handling call
0709 13:12:21.666697 (+   504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:21.667195 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:21.667236 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:21.721462 (+ 54226us) log.cc:844] Serialized 52 byte log entry
0709 13:12:22.015084 (+293622us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:12:22.015147 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.015258 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:22.762911 (+747653us) raft_consensus.cc:1685] finished
0709 13:12:22.762939 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:22.763131 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32440,"raft.queue_time_us":9155,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":93,"thread_start_us":250174,"threads_started":1,"wal-append.queue_time_us":733529}
W20260709 13:12:22.676667 31382 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:12:22.702874 31396 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:22.625360 31399 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:12:22.767560 31396 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:22.678642 31377 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.768170 31396 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.680205 31405 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:12:22.681474 30884 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.007s	user 0.010s	sys 0.003s
I20260709 13:12:22.769330 30884 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:22.683104 28080 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:22.685700 31316 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.771006 28086 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:22.771693 31396 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:22.611992 31332 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:22.658481 31187 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:22.691540 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 91836.5us
I20260709 13:12:22.690864 28761 rpcz_store.cc:276] 0709 13:12:21.265313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.265462 (+   149us) service_pool.cc:224] Handling call
0709 13:12:21.271051 (+  5589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.354194 (+ 83143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:21.354248 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:21.354307 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:21.354353 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.648830 (+1294477us) spinlock_profiling.cc:243] Waited 82.8 ms on lock 0x7b5400562cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:22.648882 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:22.649033 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190437248}
I20260709 13:12:22.699288 28084 rpcz_store.cc:276] 0709 13:12:21.305665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.306497 (+   832us) service_pool.cc:224] Handling call
0709 13:12:21.306786 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:21.357068 (+ 50282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.652200 (+1295132us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b54000e0454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:22.652405 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113187712}
I20260709 13:12:22.783324 28125 leader_election.cc:400] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8.
I20260709 13:12:22.783824 28125 leader_election.cc:403] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:22.689638 31397 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:12:22.714289 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 88113.2us
W20260709 13:12:22.711522 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 85251.1us
W20260709 13:12:22.716871 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.693518 31215 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:22.670239 31346 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.753917 31412 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:22.679464 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49611.3us
I20260709 13:12:22.851346 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:22.744921 31393 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.084s	user 0.001s	sys 0.001s
W20260709 13:12:22.795817 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:22.704418 31335 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:22.823402 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 385845us
I20260709 13:12:22.862501 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:22.854238 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:22.857721 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:22.866451 28226 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:22.867175 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 86181.5us
I20260709 13:12:22.690042 31386 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:22.784564 31397 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:22.874183 28199 leader_election.cc:400] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7.
I20260709 13:12:22.876426 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:22.868891 30731 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:22.856261 31187 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:22.881299 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:22.868093 31316 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:22.871132 31399 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:12:22.869692 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 75096us
I20260709 13:12:22.867866 31242 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.895967 31408 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:22.897388 31408 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:22.897894 31408 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:22.898232 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:22.898993 28227 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:22.900148 31406 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:22.888849 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 105687us
I20260709 13:12:22.745831 31388 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:22.866237 31405 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:22.898782 31408 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:22.901178 31405 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.902007 31408 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:22.902493 31388 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:22.903461 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:22.903887 31405 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.889117 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:22.905198 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:22.905376 31405 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:22.906651 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:22.907917 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:22.907747 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:22.908963 31388 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:22.909401 31405 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:22.910113 31388 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:22.910729 31388 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:22.911744 31388 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:22.912637 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:22.875321 31380 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:22.883309 31365 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:22.928608 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45839us
I20260709 13:12:22.916080 28084 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:22.930594 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:22.936079 28079 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:22.873051 31376 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:12:22.932178 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:22.888108 31200 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:22.936782 28078 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:22.884938 30731 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:22.947268 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:22.880584 31410 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:22.934115 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:22.936118 28085 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:22.878203 31193 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:22.937986 31408 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:22.855733 31412 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.940850 31356 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:12:22.944141 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:22.953255 31405 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:23.019918 28078 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 8: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:22.944518 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.021925 31414 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8.
I20260709 13:12:23.022466 31419 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:23.023049 31214 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:22.892580 30768 raft_consensus.cc:2764] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:12:22.752097 31387 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:23.024288 31411 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:12:22.946349 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:22.950199 30731 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:22.916107 31316 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:22.950915 31399 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:22.887220 30814 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:12:23.019914 31416 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.138s
W20260709 13:12:23.025550 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 91506.3us
I20260709 13:12:23.052664 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:23.053925 31263 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:22.989609 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.019098 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:22.957335 30485 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:22.893466 31215 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.034111 31386 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:23.023593 31415 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.149s
I20260709 13:12:23.019968 31405 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:22.954807 31410 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:22.940048 31365 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.041913 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2035) took 1249 ms. Trace:
I20260709 13:12:23.045511 31332 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:23.044303 31397 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.104811 31397 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:22.928026 31335 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.106012 31335 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:22.937384 31380 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.110670 31380 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:22.958014 31193 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.112632 30892 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:23.113057 31193 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:23.071094 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:23.114467 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:23.115434 31215 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:23.115633 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:23.116402 30914 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:23.117122 30914 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:23.117825 30914 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:23.118171 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:23.118839 28227 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:23.119479 31396 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:23.073066 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:23.120568 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47567.6us
I20260709 13:12:23.113095 30892 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:23.121722 30892 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.123023 30892 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:23.123782 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 605) took 1205 ms. Trace:
I20260709 13:12:23.125478 31242 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:23.126009 31242 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:23.031976 31388 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:23.126873 31280 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.127190 31242 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:23.127507 31280 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.056933 31399 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.077989 28161 rpcz_store.cc:276] 0709 13:12:21.792659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.792807 (+   148us) service_pool.cc:224] Handling call
0709 13:12:21.793069 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:22.354614 (+561545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:22.354834 (+   220us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:22.473832 (+118998us) log.cc:844] Serialized 26 byte log entry
0709 13:12:22.583703 (+109871us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:22.583787 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.583902 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.041665 (+457763us) raft_consensus.cc:1685] finished
0709 13:12:23.041689 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.041837 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":579628,"raft.run_cpu_time_us":441,"raft.run_wall_time_us":41496,"thread_start_us":521745,"threads_started":2,"wal-append.queue_time_us":164152}
I20260709 13:12:23.057670 31408 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.130020 31408 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.124274 28086 rpcz_store.cc:276] 0709 13:12:21.917988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.919336 (+  1348us) service_pool.cc:224] Handling call
0709 13:12:21.936607 (+ 17271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:22.771405 (+834798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.123434 (+352029us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54003aa2d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:23.123693 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1918476672}
I20260709 13:12:23.132364 31408 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:23.080160 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:23.136744 28228 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:23.081037 31412 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:23.082849 31387 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:23.075348 31367 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:12:23.083329 30731 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.085413 31316 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:23.086942 31365 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:23.068109 31416 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:23.087175 31405 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:23.153967 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 246117us
I20260709 13:12:23.155118 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:23.054546 31245 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:23.073532 31074 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:12:23.074195 31276 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:12:23.090047 31410 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:23.075919 31386 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:23.071563 31321 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:23.068820 31355 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:23.099246 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 563) took 1335 ms. Trace:
I20260709 13:12:23.168622 28234 rpcz_store.cc:276] 0709 13:12:21.764942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.765094 (+   152us) service_pool.cc:224] Handling call
0709 13:12:21.765471 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:21.983836 (+218365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:21.983885 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:21.983944 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:21.983990 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.098934 (+1114944us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54003921d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:23.099008 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.099181 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501480832}
I20260709 13:12:23.102061 31419 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:23.068452 31346 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:23.125522 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:23.056883 30485 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:23.139981 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:23.171283 28127 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:23.025641 31187 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:23.142303 31347 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.890s	user 0.005s	sys 0.006s
I20260709 13:12:23.157372 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.170539 28227 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:23.153718 31416 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:23.176087 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:23.177776 28761 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:23.182453 28125 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:23.187044 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.188912 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.167740 31387 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:23.189643 28153 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:23.190580 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.191340 28158 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:23.190490 28154 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:23.193327 28125 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:23.194530 28161 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:12:23.196144 28152 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:23.198122 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.177695 31394 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:23.200232 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.181564 31388 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:23.207466 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.207549 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.208060 28152 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 2.
I20260709 13:12:23.201300 28161 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:23.188334 31273 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:23.187789 31396 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:23.194386 31419 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:23.215751 31266 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:23.215924 30485 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.217144 30485 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:23.179723 31187 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:23.222280 28260 heartbeater.cc:646] Failed to heartbeat to 127.26.83.254:42913 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.26.83.254:42913 timed out after 15.000s (SENT)
I20260709 13:12:23.204706 31408 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:23.233695 30731 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:23.236483 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:23.237169 28226 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:23.227936 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:23.241300 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:23.243283 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:23.244362 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:23.244944 28152 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:23.253824 31403 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.672s	user 0.000s	sys 0.006s
I20260709 13:12:23.262634 31187 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:23.228477 31273 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:23.264017 31273 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.265146 31231 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:23.219157 31416 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.267452 31416 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:23.218919 31396 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.271260 31396 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:23.246359 31292 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:23.273963 31292 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.274458 31292 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:23.278004 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2038) took 1484 ms. Trace:
I20260709 13:12:23.279183 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.278606 28159 rpcz_store.cc:276] 0709 13:12:21.793948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.794064 (+   116us) service_pool.cc:224] Handling call
0709 13:12:21.837950 (+ 43886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:21.963463 (+125513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:21.963511 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:22.160406 (+196895us) log.cc:844] Serialized 26 byte log entry
0709 13:12:22.444583 (+284177us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:22.444673 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:22.444903 (+   230us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:23.253045 (+808142us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:23.277598 (+ 24553us) raft_consensus.cc:1685] finished
0709 13:12:23.277632 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.277920 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":648369,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":10788,"spinlock_wait_cycles":3396864,"thread_start_us":211664,"threads_started":2,"wal-append.queue_time_us":283053}
I20260709 13:12:23.282366 28152 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:23.283824 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.284483 28762 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:23.288409 28226 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:23.225416 30984 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:12:23.296492 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.247140 31335 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:23.299250 28235 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:23.299436 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.299657 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.301453 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.304067 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:23.305099 31266 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:23.305797 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.307627 31266 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:23.308138 31074 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.292769 31396 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:23.311033 28228 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:12:23.309528 28230 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:12:23.316071 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:23.308911 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 79407.9us
I20260709 13:12:23.330483 28228 pending_rounds.cc:85] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 4
I20260709 13:12:23.331826 28228 pending_rounds.cc:107] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:12:23.332099 28228 raft_consensus.cc:2889] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:23.331971 28125 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:23.315320 28229 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:23.313304 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.310297 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:23.334157 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:23.292661 31370 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.296518 31276 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.341635 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.341635 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:23.342366 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:23.343029 28762 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:23.343226 28234 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:23.340515 31415 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:23.346374 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:23.347208 28233 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:23.299280 31335 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.302704 31255 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.352658 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.353397 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:23.355248 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.355353 28049 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:23.355734 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:23.369261 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:23.372501 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.308857 31200 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.404407 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.405411 28762 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
W20260709 13:12:23.554569 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 202451us
I20260709 13:12:23.554739 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:23.478497 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.557389 28233 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:23.557910 28082 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:23.323316 31280 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:23.556510 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:23.336021 31399 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:12:23.477689 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.568660 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.345829 31415 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.348184 31395 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:23.358340 31374 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:23.573166 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 99021.8us
I20260709 13:12:23.569439 28078 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
W20260709 13:12:23.570685 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 231788us
I20260709 13:12:23.576169 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.578475 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.579396 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.585402 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.308656 31074 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.351485 31335 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:23.475610 31332 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:12:23.589727 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:23.591061 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 237731us
I20260709 13:12:23.584380 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:23.594007 28125 leader_election.cc:400] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5.
I20260709 13:12:23.594327 28125 leader_election.cc:403] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:12:23.594864 28229 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:12:23.567179 31419 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:23.554141 31356 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:23.475018 31237 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:23.340621 31242 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.601269 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:23.365360 31255 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.577545 31415 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:23.605257 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.605526 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.652009 28078 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:23.606304 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.607295 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:23.608508 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:23.609305 31423 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.530s	user 0.008s	sys 0.000s
I20260709 13:12:23.653896 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.654681 28759 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:23.654970 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:23.578840 31214 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:23.661912 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.665382 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.593087 31332 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.618758 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.578151 31053 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.348629 31370 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.613740 30902 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 4. Current term is 5.
I20260709 13:12:23.577358 31374 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.623906 28233 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:23.594607 31340 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:23.570937 31408 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:23.349212 31276 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:23.653791 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.308637 31266 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.660638 31410 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:23.626664 28156 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:23.814783 28156 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:23.646121 31237 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.818899 31237 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:23.819358 31242 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:23.667490 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.668627 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:23.668715 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:23.820972 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 200857us
I20260709 13:12:23.722633 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.649654 31394 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:23.575949 31200 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.626932 31388 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:23.777400 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 97484.2us
I20260709 13:12:23.849152 28198 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:23.850649 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:23.674893 31419 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:23.682017 31332 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:23.803486 31276 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:23.797760 31408 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.687314 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:23.813515 31410 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:23.825764 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 248534us
I20260709 13:12:23.667783 31234 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.144s
I20260709 13:12:23.824965 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:23.887444 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 62556.5us
I20260709 13:12:23.770951 31053 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.888506 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:23.650939 31356 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.849145 28156 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.776314 31340 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:23.854424 31370 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:23.870079 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 566) took 2042 ms. Trace:
I20260709 13:12:23.861559 31419 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.776964 31374 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:23.676916 31214 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:23.843576 31423 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:12:23.884151 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:23.890358 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:23.871308 31187 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:23.858386 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.853415 31242 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:23.862387 28198 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:23.899152 31217 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:23.899696 28231 rpcz_store.cc:276] 0709 13:12:21.828012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.829794 (+  1782us) service_pool.cc:224] Handling call
0709 13:12:21.830173 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:22.175279 (+345106us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:22.175310 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:22.175348 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:22.175380 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.869730 (+1694350us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54001c75d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:23.869805 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:23.869995 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":793104768}
I20260709 13:12:23.847055 31213 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:12:23.872813 31388 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.908676 28728 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:23.912698 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 50424.1us
W20260709 13:12:23.901526 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 212392us
W20260709 13:12:23.921339 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.921830 28198 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:23.897248 30814 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:23.873189 31394 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.887698 31410 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:23.903635 31419 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:23.912132 31187 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:23.921830 28049 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:23.899502 31340 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:23.944952 30884 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:23.945115 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:23.945562 30884 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:23.946090 30884 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:23.916517 31276 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:23.930253 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:23.916929 31217 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:23.933182 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:23.919721 28230 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:23.949653 31389 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:23.928167 31321 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:23.922799 31219 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:23.920483 31231 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:23.909277 31345 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:12:23.952915 28232 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:23.929399 28233 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:23.960826 28077 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:12:23.947654 31187 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:23.975605 31441 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:12:23.975703 31263 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:12:23.975941 31438 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:12:23.976307 31263 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:23.977321 31263 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:23.978785 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.954912 31217 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:23.987140 30485 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:23.989828 30485 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:23.990267 30485 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:23.993062 31444 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:12:23.997898 31292 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.723s	user 0.013s	sys 0.001s
I20260709 13:12:23.998827 31292 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:23.992486 31349 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.001752 31349 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.004248 31349 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.005494 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.950081 31276 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:23.959304 31074 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:23.993878 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:24.004563 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:23.980607 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.024188 28161 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:24.013006 28158 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 4.
I20260709 13:12:23.958806 31321 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.009209 28758 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 8: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:24.021348 31276 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:24.024138 28154 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:24.036154 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:24.033888 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:24.041249 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 47484.8us
I20260709 13:12:24.050577 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.034518 31321 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:24.054977 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:24.063275 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.014031 31389 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:12:24.064049 28158 raft_consensus.cc:2410] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:12:24.041182 31349 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.047904 31444 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:24.069706 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.072125 28161 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
W20260709 13:12:24.072903 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.050855 31292 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:24.075245 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.061975 31200 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:24.050081 31345 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:12:24.071556 31444 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:24.097587 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 171457us
I20260709 13:12:24.074820 31408 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.075218 31356 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.101071 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:24.101238 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.106608 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:24.111845 31231 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:24.113564 31446 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:24.114277 30814 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:24.114650 31458 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:24.114892 31412 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.113559 31456 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:24.116756 31346 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:24.124756 31214 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.118191 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:24.121505 28759 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:24.074101 31349 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.099783 31444 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:24.126040 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:24.097837 31053 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.128213 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.097607 31389 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:24.151324 31389 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.126580 31442 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:24.134871 31446 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.155800 31389 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.157155 31446 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.135978 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:24.142418 31458 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.160353 31458 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.148818 28078 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:24.128539 31388 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:24.135242 31437 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:12:24.154261 28231 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
W20260709 13:12:24.146767 31435 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:12:24.161943 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:24.165326 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:24.168915 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:24.181630 30884 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.150863 31456 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.148205 31445 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:24.197965 31214 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:24.194419 28078 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:24.134192 30814 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.199663 31214 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.201337 30814 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.202064 31214 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.134675 31412 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.153462 31394 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.189971 31446 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:24.184221 30884 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.191987 31456 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:24.248793 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 132390us
I20260709 13:12:24.238704 31446 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.264631 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:24.229346 31419 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.193950 31231 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.263550 31446 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:24.292248 28728 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.692s (SENT) [suppressed 28 similar messages]
W20260709 13:12:24.303159 31460 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:12:24.293313 31374 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.314817 28154 raft_consensus.cc:2368] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:24.314755 31217 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.317358 31217 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.318830 31374 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:24.325968 31415 consensus_meta.cc:287] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.658s	user 0.005s	sys 0.006s
I20260709 13:12:24.326588 31415 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.329726 31415 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.326279 31446 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.333351 31446 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.282181 31458 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:24.331928 31340 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.336753 31394 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.369691 31346 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:24.370491 31346 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:24.375144 31436 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.551s	user 0.002s	sys 0.001s
I20260709 13:12:24.382422 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 551) took 1180 ms. Trace:
I20260709 13:12:24.382898 28153 rpcz_store.cc:276] 0709 13:12:23.202206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.202393 (+   187us) service_pool.cc:224] Handling call
0709 13:12:23.202622 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:23.215951 (+ 13329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:23.215986 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:23.216028 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:23.216057 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.382141 (+1166084us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b54003430d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:24.382186 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:24.382355 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30201088}
I20260709 13:12:24.344708 31217 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:12:24.388705 31346 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.390550 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1725) took 1269 ms. Trace:
I20260709 13:12:24.390996 28235 rpcz_store.cc:276] 0709 13:12:23.121607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.124853 (+  3246us) service_pool.cc:224] Handling call
0709 13:12:23.125214 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:23.299712 (+174498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.390297 (+1090585us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400306454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:24.390490 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399348608}
W20260709 13:12:24.395444 31335 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.777s	user 0.008s	sys 0.003s
I20260709 13:12:24.395789 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.395912 31335 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.406867 28152 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:24.398181 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2057) took 1037 ms. Trace:
I20260709 13:12:24.408934 28234 rpcz_store.cc:276] 0709 13:12:23.361128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.366130 (+  5002us) service_pool.cc:224] Handling call
0709 13:12:23.366600 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:23.651508 (+284908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:23.651555 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:23.662481 (+ 10926us) log.cc:844] Serialized 26 byte log entry
0709 13:12:24.111424 (+448943us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:24.111469 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.397744 (+286275us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54003bfbd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:24.397799 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:24.397850 (+    51us) raft_consensus.cc:1685] finished
0709 13:12:24.397863 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:24.398108 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1891,"raft.queue_time_us":6213,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":13711,"spinlock_wait_cycles":649467648,"thread_start_us":288287,"threads_started":1,"wal-append.queue_time_us":329741}
I20260709 13:12:24.398423 31419 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.400127 31346 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.401172 31412 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.307125 31461 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:12:24.426628 31335 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:24.402447 31394 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.429013 31394 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.306834 30814 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:12:24.326651 31214 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:12:24.337960 31276 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.433065 31276 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.342943 31213 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
W20260709 13:12:24.351626 31416 consensus_meta.cc:287] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.084s	user 0.014s	sys 0.000s
W20260709 13:12:24.406351 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 95810.8us
I20260709 13:12:24.396889 31349 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.435600 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:24.437203 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:24.438519 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:24.423173 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.440364 28152 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:24.443032 28155 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:12:24.444785 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:24.423990 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 131776us
I20260709 13:12:24.384402 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:24.445817 28153 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:24.518291 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.519487 28152 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
W20260709 13:12:24.520965 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 82503.8us
I20260709 13:12:24.525556 31255 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.526938 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:24.531145 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:24.531711 28152 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:12:24.532039 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:24.436218 31410 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.001s
I20260709 13:12:24.434597 31444 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.534533 28050 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:24.537492 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 554) took 1205 ms. Trace:
I20260709 13:12:24.538041 28159 rpcz_store.cc:276] 0709 13:12:23.332428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.332587 (+   159us) service_pool.cc:224] Handling call
0709 13:12:23.344217 (+ 11630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:23.344591 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:23.344619 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:24.393289 (+1048670us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b2c000b54b8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:24.393856 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:12:24.532179 (+138323us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:24.532218 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.532283 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:24.532337 (+    54us) raft_consensus.cc:1685] finished
0709 13:12:24.532352 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:24.537407 (+  5055us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23174,"raft.queue_time_us":1313,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":3765,"spinlock_wait_cycles":8341248,"thread_start_us":137944,"threads_started":1,"wal-append.queue_time_us":134406}
I20260709 13:12:24.539927 28153 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:24.533394 31231 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:24.526988 31214 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:24.542441 31214 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:24.544016 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:24.519949 31335 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.545858 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:24.542940 31214 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:24.548843 31345 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:24.517900 31471 raft_consensus.cc:2764] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:12:24.543709 31216 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:12:24.424813 31187 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.562834 31187 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:24.563051 31466 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.238s	user 0.003s	sys 0.000s
I20260709 13:12:24.531459 30814 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:12:24.553164 31335 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.565013 31444 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.566635 31335 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:24.565013 31332 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.678s	user 0.006s	sys 0.006s
I20260709 13:12:24.567612 31332 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.569753 31332 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.570595 28086 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:24.572233 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 619) took 1096 ms. Trace:
I20260709 13:12:24.553869 31340 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.530663 31349 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:24.537006 31410 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.557408 31458 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.589421 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.589610 31458 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:24.591532 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:24.591952 31458 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.558632 31321 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.592562 28077 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:24.594168 31321 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.573913 28086 rpcz_store.cc:276] 0709 13:12:23.475797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.475949 (+   152us) service_pool.cc:224] Handling call
0709 13:12:24.571997 (+1096048us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b540027c9d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:24.572142 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2277656960}
I20260709 13:12:24.552771 28153 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:24.595511 28082 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:24.595907 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.596684 28153 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:24.597620 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:24.598706 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.599897 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.601063 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.575800 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.576439 28152 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:12:24.586889 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:24.604342 28079 raft_consensus.cc:2368] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:24.594192 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.600446 28153 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:24.613298 28159 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:24.543205 31215 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:24.617247 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.621978 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.622997 28153 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 4.
I20260709 13:12:24.630084 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.632112 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.626493 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.640664 28763 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:24.641364 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.642120 28086 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:24.644335 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:24.645782 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:24.646507 28082 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:24.647432 28161 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.612135 31345 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:24.650377 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.650609 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.654698 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:24.655220 28153 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:24.656283 28086 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:24.656431 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.657753 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.664110 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.665652 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.666567 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:24.668133 28153 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:24.669724 28080 raft_consensus.cc:2368] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:24.676975 28082 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:24.678601 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:24.657770 31345 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:24.680431 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.683482 28159 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:24.681440 28161 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:24.685489 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.686789 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.687386 28158 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:24.688236 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.689320 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:24.680596 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.657600 31336 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:24.695143 31336 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.667553 31293 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:24.703738 28158 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:24.698854 28153 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:24.709983 28153 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:24.704401 31336 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.692158 31345 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:24.708870 31293 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.739415 31293 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.744424 28159 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:24.722455 31273 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:24.781095 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:24.783340 28079 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:24.784224 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:24.819175 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:24.883005 28152 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:12:24.879181 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56787us
I20260709 13:12:24.876528 30485 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.884977 30485 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:24.785266 31349 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.891404 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:24.784965 31273 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.905546 28152 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:24.886011 31263 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:24.911052 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:24.914198 28152 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 4: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:24.917480 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:24.908012 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:24.895295 31456 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:24.900776 31410 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:24.930039 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:24.917562 31477 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:12:24.936401 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:24.937484 28084 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 4: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:24.937721 31439 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.135s	user 0.003s	sys 0.016s
I20260709 13:12:24.942118 31273 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:24.948681 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:24.926438 30804 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:24.900326 31349 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:24.952527 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2045) took 1646 ms. Trace:
I20260709 13:12:24.925137 31394 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:24.955345 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 806) took 1334 ms. Trace:
I20260709 13:12:24.957895 28226 rpcz_store.cc:276] 0709 13:12:23.306340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.306493 (+   153us) service_pool.cc:224] Handling call
0709 13:12:23.306834 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:23.317085 (+ 10251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:23.317115 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:23.317572 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:12:23.554673 (+237101us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:23.554719 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:23.554798 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:24.937119 (+1382321us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:24.946905 (+  9786us) raft_consensus.cc:1685] finished
0709 13:12:24.946926 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:24.952463 (+  5537us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9840,"raft.queue_time_us":11799,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":7683456,"thread_start_us":236783,"threads_started":1,"wal-append.queue_time_us":466367}
I20260709 13:12:24.958230 28760 rpcz_store.cc:276] 0709 13:12:23.620579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.621591 (+  1012us) service_pool.cc:224] Handling call
0709 13:12:23.735410 (+113819us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:23.819679 (+ 84269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:23.819712 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:23.819754 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:23.819791 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.955101 (+1135310us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b54006b3d54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:24.955150 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:24.955283 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193278848}
I20260709 13:12:24.952215 30804 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:24.965991 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:24.956990 31462 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.612s	user 0.005s	sys 0.000s
I20260709 13:12:24.971529 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:24.973486 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:24.969741 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:24.982933 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:24.983971 30485 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:24.984337 30485 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:24.984619 30485 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:24.988559 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:25.007217 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:24.998898 30884 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:24.968396 30804 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:25.013716 30884 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.011099 31237 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:24.970527 31410 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:25.017762 31231 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:25.009368 31219 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:24.963696 31394 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.031518 31237 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:25.029027 31231 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:25.035929 31231 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:25.035543 31237 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.036908 31219 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:25.037901 31219 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.023483 30884 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:25.009213 31416 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:12:25.028331 31410 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:25.095405 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 88259.2us
I20260709 13:12:25.042954 31217 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:25.112382 30485 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.097437 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:25.079321 31410 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:25.115314 31217 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.128487 31461 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:25.147982 30485 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:25.142916 31217 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:25.148348 31481 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
W20260709 13:12:25.170974 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 116171us
W20260709 13:12:25.171084 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 147570us
I20260709 13:12:25.160475 31461 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:25.173590 28127 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.775s (SENT)
W20260709 13:12:25.174561 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:25.167984 31349 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.169428 31338 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:25.178319 31214 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.631s	user 0.006s	sys 0.006s
W20260709 13:12:25.187732 31483 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:12:25.200573 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:25.202593 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:25.206243 31394 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.206890 31345 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.208051 28080 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:25.187493 31214 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.213346 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:25.183609 31338 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.220094 31335 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:25.220990 31231 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:25.222517 31482 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:12:25.246305 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:25.247004 28081 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
W20260709 13:12:25.249876 28758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 803) took 1666 ms (client timeout 1817 ms). Trace:
W20260709 13:12:25.250455 28758 rpcz_store.cc:269] 0709 13:12:23.583586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.583805 (+   219us) service_pool.cc:224] Handling call
0709 13:12:25.249625 (+1665820us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54006b0154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:25.249804 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":955989376}
I20260709 13:12:25.255556 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 581) took 2019 ms. Trace:
I20260709 13:12:25.261320 28227 rpcz_store.cc:276] 0709 13:12:23.236075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.238854 (+  2779us) service_pool.cc:224] Handling call
0709 13:12:23.239433 (+   579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:23.239939 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:23.239981 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:23.245692 (+  5711us) log.cc:844] Serialized 26 byte log entry
0709 13:12:25.019397 (+1773705us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b1800095d40. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:12:25.162715 (+143318us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:25.163179 (+   464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.163275 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:25.233056 (+ 69781us) raft_consensus.cc:1685] finished
0709 13:12:25.233075 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:25.255473 (+ 22398us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":859,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":2752,"spinlock_wait_cycles":4702464,"thread_start_us":143027,"threads_started":1,"wal-append.queue_time_us":1936505}
I20260709 13:12:25.212278 31461 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:25.306797 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:25.326133 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 112867us
I20260709 13:12:25.274055 31394 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:25.442026 28763 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 807) took 1811 ms (client timeout 1787 ms). Trace:
W20260709 13:12:25.442629 28763 rpcz_store.cc:269] 0709 13:12:23.631111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.631322 (+   211us) service_pool.cc:224] Handling call
0709 13:12:25.441723 (+1810401us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54004371d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:25.441928 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1881306880}
I20260709 13:12:25.452160 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:25.452877 28758 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:25.446929 31345 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.429531 31394 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:25.476200 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2092) took 1857 ms. Trace:
I20260709 13:12:25.479885 28155 rpcz_store.cc:276] 0709 13:12:23.618671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.618785 (+   114us) service_pool.cc:224] Handling call
0709 13:12:23.619102 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:24.443710 (+824608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.475856 (+1032146us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b54003764d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:25.476108 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1878276864}
I20260709 13:12:25.494904 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:25.504961 28049 connection.cc:256] RPC call timeout handler was delayed by -1.1898514770000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1817s for a response
I20260709 13:12:25.514847 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:25.516052 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:25.516932 28763 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:25.485095 31338 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:25.521944 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:25.524979 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:25.485220 31273 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.538866 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:25.542972 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:25.543994 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:25.544821 28758 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:25.547379 31394 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:25.549264 30804 consensus_meta.cc:287] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.001s
I20260709 13:12:25.549726 30804 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.551949 30804 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.552695 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:25.555105 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:25.559209 28156 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:25.582124 28760 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:25.584798 28762 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:25.586181 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:25.529215 31501 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:25.592936 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:25.593683 28762 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:25.594635 28156 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:25.595329 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2090) took 1977 ms. Trace:
I20260709 13:12:25.595652 28156 rpcz_store.cc:276] 0709 13:12:23.618058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.618212 (+   154us) service_pool.cc:224] Handling call
0709 13:12:23.626514 (+  8302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:25.558520 (+1932006us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b58003b66d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5d30aa 00007f59ce655447 00007f59ce66dfe7 00007f59ce66f7de 00007f59ce679b49 00007f59ce67a416 00007f59ce676d8f 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4
0709 13:12:25.595058 (+ 36538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.595260 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1638356224}
I20260709 13:12:25.609699 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:25.611487 28127 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.577s (SENT)
I20260709 13:12:25.612521 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:25.613572 28084 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:25.614905 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:25.617142 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:25.617884 28161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2060) took 1667 ms (client timeout 1874 ms). Trace:
W20260709 13:12:25.618268 28161 rpcz_store.cc:269] 0709 13:12:23.950450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.126242 (+175792us) service_pool.cc:224] Handling call
0709 13:12:25.617734 (+1491492us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b540011e254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:25.617834 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123618560}
I20260709 13:12:25.619983 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:25.621410 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:25.605924 31377 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:12:25.637229 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 584) took 1527 ms. Trace:
I20260709 13:12:25.637506 31419 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.638041 31419 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:25.637686 28230 rpcz_store.cc:276] 0709 13:12:24.110191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.191932 (+ 81741us) service_pool.cc:224] Handling call
0709 13:12:24.192216 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:24.192418 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:24.233684 (+ 41266us) log.cc:844] Serialized 14 byte log entry
0709 13:12:25.636884 (+1403200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:25.636935 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:25.636978 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.637044 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:25.637166 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1402896,"prepare.queue_time_us":41231,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":40872,"threads_started":1,"wal-append.queue_time_us":14588,"wal-append.run_cpu_time_us":1037,"wal-append.run_wall_time_us":1027545}
W20260709 13:12:25.620781 31263 consensus_meta.cc:287] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.697s	user 0.012s	sys 0.001s
I20260709 13:12:25.640057 31276 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:25.640621 31276 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.641043 31276 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:25.623008 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:25.629662 28154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2103) took 2008 ms (client timeout 1967 ms). Trace:
I20260709 13:12:25.633489 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2055) took 2444 ms. Trace:
I20260709 13:12:25.541038 31273 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:25.635675 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 1.47366e+06us
I20260709 13:12:25.606279 31501 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.647907 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:25.635466 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2067) took 1590 ms. Trace:
I20260709 13:12:25.659030 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:25.659854 28084 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:25.648645 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:25.644801 28154 rpcz_store.cc:269] 0709 13:12:23.621524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.035870 (+414346us) service_pool.cc:224] Handling call
0709 13:12:25.629351 (+1593481us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400053554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:25.629556 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638742400}
I20260709 13:12:25.644884 28160 rpcz_store.cc:276] 0709 13:12:23.188842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:23.188954 (+   112us) service_pool.cc:224] Handling call
0709 13:12:23.189196 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:23.277279 (+ 88083us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:23.297970 (+ 20691us) log.cc:844] Serialized 14 byte log entry
0709 13:12:23.344382 (+ 46412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:23.344460 (+    78us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:23.344527 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:24.699331 (+1354804us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b180012be00. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419
0709 13:12:25.633201 (+933870us) spinlock_profiling.cc:243] Waited 87.8 ms on lock 0x7b540010bb54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:25.633249 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:25.633402 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72438,"prepare.queue_time_us":21447,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":108,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4733,"spinlock_wait_cycles":210942208,"thread_start_us":45893,"threads_started":1,"wal-append.queue_time_us":607559,"wal-append.run_cpu_time_us":2351,"wal-append.run_wall_time_us":1287013}
I20260709 13:12:25.668766 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:25.669661 28760 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:25.673348 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:25.623742 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:25.681574 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:25.682472 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:25.683257 28758 raft_consensus.cc:2368] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:25.686354 28154 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:25.687227 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:25.688294 31242 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.688808 31242 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:25.690965 31242 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:25.691506 31242 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.691942 31242 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.659597 28232 rpcz_store.cc:276] 0709 13:12:24.045300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.045833 (+   533us) service_pool.cc:224] Handling call
0709 13:12:24.046202 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:24.046453 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:24.061017 (+ 14564us) log.cc:844] Serialized 14 byte log entry
0709 13:12:24.146015 (+ 84998us) log.cc:844] Serialized 14 byte log entry
0709 13:12:25.634977 (+1488962us) spinlock_profiling.cc:243] Waited 955 us on lock 0x7b1800431e00. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:12:25.635130 (+   153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:25.635180 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:25.635224 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.635293 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:25.635408 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1487530,"prepare.queue_time_us":18399,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":2067,"raft.queue_time_us":1506149,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":1702,"spinlock_wait_cycles":2197248,"thread_start_us":69925,"threads_started":1,"wal-append.queue_time_us":134898,"wal-append.run_cpu_time_us":2711,"wal-append.run_wall_time_us":1204939}
I20260709 13:12:25.696951 31367 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:25.697299 28759 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 3 LEARNER]: Advancing to term 5
I20260709 13:12:25.697535 31367 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:25.699632 31367 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:25.704365 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 638) took 1031 ms. Trace:
I20260709 13:12:25.705157 28080 rpcz_store.cc:276] 0709 13:12:24.673475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.673840 (+   365us) service_pool.cc:224] Handling call
0709 13:12:24.690095 (+ 16255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:25.208421 (+518326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.704129 (+495708us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54002daad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:25.704303 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1190674816}
I20260709 13:12:25.711191 31276 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.712915 31380 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:25.714039 31380 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:25.714337 31380 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:25.714480 31214 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.715217 31380 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.717113 27982 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 3 to 6, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 6 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:25.718748 27982 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 5 to 8, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:25.649626 31461 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:25.721488 31410 consensus_meta.cc:287] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.006s
I20260709 13:12:25.721151 31461 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:25.728929 31410 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.731132 31410 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:25.732479 31217 consensus_meta.cc:287] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.564s	user 0.009s	sys 0.005s
I20260709 13:12:25.732957 31217 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:25.734201 31231 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.682464 28764 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:25.720242 27982 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 6, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 6 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:25.751740 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:25.752804 28081 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:25.756217 28760 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
W20260709 13:12:25.648252 30884 consensus_meta.cc:287] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.570s	user 0.013s	sys 0.005s
I20260709 13:12:25.745680 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:25.758615 28079 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:25.760370 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:25.777053 28153 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:25.763348 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:25.765911 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:25.769869 28082 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:25.769992 28085 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:12:25.772948 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:25.781912 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:25.782713 28080 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:25.782858 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:25.784165 28083 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:25.784991 31255 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:25.785554 31255 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.704312 31419 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:25.791965 31501 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.793220 31367 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:25.793785 31367 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.795795 31367 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.797533 31242 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.798869 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:25.716528 31380 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:25.800076 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:25.802320 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:25.815659 28234 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:25.803342 28764 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:25.791848 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:25.800462 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:25.805106 28230 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:25.807852 28084 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:25.819712 27982 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 8, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:25.805477 28759 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 5 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:25.809973 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:25.810138 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 64500.7us
I20260709 13:12:25.811846 28227 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:25.822103 27982 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 5 to 7, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 7 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:25.822916 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:25.813730 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:25.823853 28228 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:25.824100 27982 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 7, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 7 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:25.857002 28227 raft_consensus.cc:2368] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10.
W20260709 13:12:25.870151 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 62903.3us
I20260709 13:12:25.871884 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:25.757042 31321 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:25.880620 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 583) took 1770 ms. Trace:
I20260709 13:12:25.881424 28077 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:25.883322 28080 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:25.769758 31074 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.787808 31255 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:25.890436 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 68567.5us
I20260709 13:12:25.895383 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:25.902067 31445 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:25.902904 31276 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.809036 30884 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:25.798101 31273 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:25.905387 31074 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.853551 31338 raft_consensus.cc:2764] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:12:25.907582 31346 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:25.816879 31200 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.881771 28229 rpcz_store.cc:276] 0709 13:12:24.109949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.191498 (+ 81549us) service_pool.cc:224] Handling call
0709 13:12:24.191834 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:24.192033 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:24.192357 (+   324us) log.cc:844] Serialized 14 byte log entry
0709 13:12:24.562455 (+370098us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:25.738385 (+1175930us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b48000600d8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:12:25.880214 (+141829us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:25.880296 (+    82us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:25.880342 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.880430 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:25.880559 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1438334,"prepare.queue_time_us":919,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":5453,"raft.queue_time_us":1388746,"raft.run_cpu_time_us":1036,"raft.run_wall_time_us":70205,"spinlock_wait_cycles":121229568,"thread_start_us":237942,"threads_started":2,"wal-append.queue_time_us":132536,"wal-append.run_cpu_time_us":3551,"wal-append.run_wall_time_us":1238619}
I20260709 13:12:25.909080 28232 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:25.908911 31200 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.909924 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:25.910584 28229 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:25.911057 31200 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.911689 28235 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:25.853073 31245 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:25.853283 31504 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:25.912884 31321 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.913244 31504 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:25.913765 31504 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:25.914978 31321 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.874392 31340 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:12:25.916440 31340 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:25.869907 31266 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.917325 31245 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:25.919406 31245 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:25.917337 31266 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.920817 31377 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:25.921995 31377 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.922082 31266 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:25.922629 31377 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:25.923804 30884 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:25.911581 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:25.924865 31503 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:25.925326 31503 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:25.925664 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:25.901341 28227 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:25.888878 28230 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:25.928170 28233 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:12:25.928319 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:25.931511 28234 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:25.916788 31340 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:25.933156 28153 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
W20260709 13:12:25.935086 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 143294us
I20260709 13:12:25.929332 28227 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:25.796615 31412 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:12:25.938620 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:25.954970 28234 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:25.951257 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:25.961599 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:25.962574 28227 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:25.965915 28229 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:25.965983 28153 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:12:25.969080 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:25.907573 31074 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:25.909129 31346 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:25.971257 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:25.969156 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:25.936367 31255 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.950555 31349 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:25.977664 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 84277.9us
I20260709 13:12:25.983515 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:25.979461 28234 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:25.985797 28049 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:25.878760 31490 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:12:25.995851 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:25.968385 31200 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.006718 28156 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:26.000203 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:25.971737 31321 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:26.012415 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:26.013447 28229 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:25.976068 31276 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:25.976994 31346 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:25.980629 31242 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:26.014599 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:26.012327 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:25.976343 31187 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:25.991147 31370 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:25.981962 31349 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:26.038760 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:26.034716 28156 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:12:26.021016 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49840.6us
I20260709 13:12:26.014341 31412 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:26.042953 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:26.047333 28235 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:26.044458 28199 leader_election.cc:400] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:26.048099 28199 leader_election.cc:403] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:12:26.049381 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:26.049553 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:26.050218 28233 raft_consensus.cc:2410] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:12:25.994843 31255 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.021888 28049 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:26.013963 31464 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:26.045486 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:26.067847 28229 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:26.049396 28159 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:26.031430 31345 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:26.072178 28761 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:26.072970 28199 leader_election.cc:400] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:26.062898 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:26.073563 28199 leader_election.cc:403] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:12:26.069999 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:26.042128 31349 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:26.068410 28763 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:26.078055 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:26.079388 28233 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:26.034112 31514 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:26.085235 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:26.093138 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:26.014010 31200 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.105268 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:26.048955 31266 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:26.019928 31321 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.114338 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:26.115114 28080 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:26.031458 31276 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.072309 31345 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.066852 31394 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.049090 31242 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:12:26.041514 31187 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.158145 31321 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:26.073210 31155 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:12:26.087222 31416 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:12:26.161062 31370 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:26.161516 31416 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:26.161652 31370 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.132172 31345 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:26.121315 31514 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:26.093683 31464 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:26.168290 27982 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.088879 31074 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 13:12:26.170106 27982 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:26.127599 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.068897 31412 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:26.018589 31340 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.173491 31412 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:26.175312 31505 log.cc:927] Time spent T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:12:26.175730 31340 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:26.072371 31511 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:26.177116 31511 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.177551 31511 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:26.112888 30884 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:12:26.192997 30884 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:26.193418 30884 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:26.166992 31394 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:26.196345 31394 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:26.151124 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:26.118628 31501 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:26.162083 31416 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:26.170416 31200 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:26.092912 31456 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.084s
W20260709 13:12:26.138273 31515 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:12:26.117967 31231 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:26.137131 30804 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:26.178921 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:12:26.255254 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 76410.3us
I20260709 13:12:26.256431 31210 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:26.256278 30804 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.256927 31210 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:26.257366 31210 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:26.258797 30804 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.259485 31231 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.261210 28052 leader_election.cc:400] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10.
I20260709 13:12:26.261878 31231 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.262719 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:26.263857 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:26.172044 27982 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 6, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 6 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.265069 31456 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:26.265479 31456 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:26.265898 31456 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:26.265655 27982 catalog_manager.cc:5697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.267113 31349 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.267402 27982 catalog_manager.cc:5697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 4, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 4 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:26.186676 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 113844us
I20260709 13:12:26.268920 31231 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:26.145871 31518 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:12:26.269335 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:26.208297 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.210812 31214 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:26.270967 31214 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.188496 28227 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:26.273296 31214 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:26.273922 31242 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:26.274493 31242 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:26.275496 31242 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.169607 31464 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:26.181726 31385 consensus_queue.cc:1175] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "611ec64ec58244fe82058d6130e4e6f3" responder_term: 7 status { last_received { term: 7 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 7 index: 7 } }
I20260709 13:12:26.277319 30804 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:26.277938 30804 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.198175 31370 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:26.280332 30804 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:26.196471 31255 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:26.217257 28228 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
W20260709 13:12:26.281984 27984 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 259) took 18360 ms (client timeout 15000 ms). Trace:
I20260709 13:12:26.221980 28078 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:26.126659 28198 leader_election.cc:400] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
W20260709 13:12:26.283974 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 157520us
I20260709 13:12:26.182690 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:26.212627 31276 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.207729 31499 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:26.293716 28198 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:26.305299 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:26.307744 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:26.309459 27982 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 5 to 7, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 7 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:26.329602 27982 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.148206 31503 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.331878 27982 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 8, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.334362 27982 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 5 to 7, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 7 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.208458 31504 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.336704 28198 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:26.282554 27984 rpcz_store.cc:269] 0709 13:12:07.954797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:07.957284 (+  2487us) service_pool.cc:224] Handling call
0709 13:12:26.281859 (+18324575us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:19.217836 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:19.218453 (+   617us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:19.218504 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:12:19.218521 (+    17us) tablet.cc:662] Decoding operations
0709 13:12:19.220136 (+  1615us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:19.220221 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:19.220247 (+    26us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:12:19.229147 (+  8900us) tablet.cc:701] Row locks acquired
0709 13:12:19.229186 (+    39us) write_op.cc:260] PREPARE: finished
0709 13:12:19.229373 (+   187us) write_op.cc:270] Start()
0709 13:12:19.229529 (+   156us) write_op.cc:276] Timestamp: P: 1783602739229311 usec, L: 0
0709 13:12:19.229567 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:12:19.230348 (+   781us) log.cc:844] Serialized 6775 byte log entry
0709 13:12:19.657271 (+426923us) op_driver.cc:464] REPLICATION: finished
0709 13:12:20.071017 (+413746us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59cb56d0ac 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59d20adef0
0709 13:12:20.181141 (+110124us) write_op.cc:301] APPLY: starting
0709 13:12:20.181233 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:20.181659 (+   426us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:20.181698 (+    39us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:20.190038 (+  8340us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:20.190426 (+   388us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:12:20.190488 (+    62us) write_op.cc:312] APPLY: finished
0709 13:12:20.190934 (+   446us) log.cc:844] Serialized 119 byte log entry
0709 13:12:20.271564 (+ 80630us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:20.296959 (+ 25395us) write_op.cc:454] Released schema lock
0709 13:12:20.351487 (+ 54528us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:20.408663 (+ 57176us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:20.410939 (+  2276us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":881897,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":519553,"apply.run_cpu_time_us":5417,"apply.run_wall_time_us":210186,"num_ops":17,"prepare.queue_time_us":83702,"prepare.run_cpu_time_us":14080,"prepare.run_wall_time_us":45836,"raft.queue_time_us":184381,"raft.run_cpu_time_us":29211,"raft.run_wall_time_us":486968,"replication_time_us":427501,"spinlock_wait_cycles":4226432,"thread_start_us":679366,"threads_started":4,"wal-append.queue_time_us":117753,"wal-append.run_cpu_time_us":8899,"wal-append.run_wall_time_us":1838429}]]}
I20260709 13:12:26.346731 31245 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:26.370740 31276 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:26.370458 31245 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.371604 31276 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:26.346539 31416 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.343364 31074 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:26.286381 31367 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
W20260709 13:12:26.314369 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 203910us
I20260709 13:12:26.378530 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:26.320254 28235 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:26.339906 31370 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:26.333410 31499 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.398351 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2082) took 1648 ms. Trace:
I20260709 13:12:26.399067 28152 rpcz_store.cc:276] 0709 13:12:24.750491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.279340 (+528849us) service_pool.cc:224] Handling call
0709 13:12:25.279670 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:25.280201 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:25.280230 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:25.367259 (+ 87029us) log.cc:844] Serialized 52 byte log entry
0709 13:12:25.513434 (+146175us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:25.871422 (+357988us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.066116 (+194694us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b180032e660. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:12:26.074879 (+  8763us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.381840 (+306961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.381958 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.398111 (+ 16153us) raft_consensus.cc:1685] finished
0709 13:12:26.398135 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.398275 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":552453,"prepare.queue_time_us":692600,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":81,"raft.queue_time_us":78459,"raft.run_cpu_time_us":667,"raft.run_wall_time_us":133516,"spinlock_wait_cycles":12633856,"thread_start_us":318997,"threads_started":2,"wal-append.queue_time_us":156142}
I20260709 13:12:26.402053 28235 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:26.335160 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2086) took 1444 ms. Trace:
I20260709 13:12:26.383436 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:26.350142 27982 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 7, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 7 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.349623 31503 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:26.350759 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:26.407699 28157 rpcz_store.cc:276] 0709 13:12:24.890840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.565067 (+674227us) service_pool.cc:224] Handling call
0709 13:12:25.565415 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:25.565793 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:25.565824 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:25.566268 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:12:25.836018 (+269750us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:12:26.334397 (+498379us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.334825 (+   428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.334901 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.334944 (+    43us) raft_consensus.cc:1685] finished
0709 13:12:26.334958 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.335100 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123166,"prepare.queue_time_us":301059,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":2577,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":13228,"thread_start_us":630089,"threads_started":2,"wal-append.queue_time_us":269669}
I20260709 13:12:26.357995 31214 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:12:26.411710 27982 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 8, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.349118 31528 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:12:26.396970 30804 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:26.405462 31416 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:26.434938 28230 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:26.394487 31349 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.449707 27982 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 8, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 8 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:26.452018 27982 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 3 to 5, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 5 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.468027 27982 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.486049 27982 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 3 to 7, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 7 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:26.436698 30804 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:26.493985 31074 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:26.494439 31548 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:26.496215 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 223473us
W20260709 13:12:26.497810 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 147196us
I20260709 13:12:26.508566 31074 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.508551 30804 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:26.523651 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:26.436033 31380 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.006s
I20260709 13:12:26.404479 31237 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:12:26.525334 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 422991us
I20260709 13:12:26.349638 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2085) took 1468 ms. Trace:
I20260709 13:12:26.408589 31053 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:12:26.408869 30892 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:12:26.408979 31231 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:12:26.533444 28155 rpcz_store.cc:276] 0709 13:12:24.881416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.512515 (+631099us) service_pool.cc:224] Handling call
0709 13:12:25.512922 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:25.515703 (+  2781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:25.515739 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:25.516237 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:12:25.807557 (+291320us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:26.344044 (+536487us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.349182 (+  5138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.349302 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.349364 (+    62us) raft_consensus.cc:1685] finished
0709 13:12:26.349383 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.349565 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120986,"prepare.queue_time_us":319999,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":148,"raft.queue_time_us":7703,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":1631,"thread_start_us":674972,"threads_started":2,"wal-append.queue_time_us":383303}
I20260709 13:12:26.432303 28760 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:26.547739 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:26.552012 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:26.747721 28230 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:26.560971 28762 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:26.526834 31370 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:26.634174 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69429.5us
I20260709 13:12:26.752202 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:26.754576 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 588) took 1012 ms. Trace:
W20260709 13:12:26.638900 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 115348us
I20260709 13:12:26.404278 31276 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.655925 31263 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:26.651067 31242 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:26.661584 31527 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:12:26.670629 31504 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:26.681843 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:26.770866 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.771270 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:26.638733 31394 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:12:26.637089 31539 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:12:26.750932 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:26.757846 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:26.758051 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:26.758603 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:26.758713 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:26.830356 28758 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:26.759171 31273 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:26.733726 31345 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.569s	user 0.012s	sys 0.000s
I20260709 13:12:26.774030 31530 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.774825 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:26.766381 31263 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:26.783726 31456 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.517s	user 0.005s	sys 0.005s
I20260709 13:12:26.784843 31499 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:26.834829 31273 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.838266 31461 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:26.786639 31511 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.004s
I20260709 13:12:26.643021 31552 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.130s
W20260709 13:12:26.800941 31210 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.000s
I20260709 13:12:26.768580 31370 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:26.803545 30804 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.852689 30804 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.812600 31346 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.817543 31377 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.820679 31245 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:26.853839 31346 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:26.856861 31210 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.859251 31210 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.843695 31530 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.834524 31345 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.836501 31380 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
W20260709 13:12:26.849793 31517 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.755s	user 0.002s	sys 0.026s
W20260709 13:12:26.850790 31513 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.730s	user 0.008s	sys 0.003s
I20260709 13:12:26.839478 31217 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:26.866585 31345 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.869539 31187 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.844126 31511 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:26.872370 31511 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:26.863153 31273 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.877286 31370 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.839488 31456 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.859757 31263 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.882962 31263 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:26.884713 31456 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:26.897115 31530 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:26.506567 27982 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 8, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.928581 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:26.941982 28049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.005s	sys 0.005s
W20260709 13:12:26.942785 28049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.006s	sys 0.005s
W20260709 13:12:26.642177 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 237398us
I20260709 13:12:26.955893 28081 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:26.959967 28080 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:26.743733 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 360365us
I20260709 13:12:26.755117 28226 rpcz_store.cc:276] 0709 13:12:25.742410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.743685 (+  1275us) service_pool.cc:224] Handling call
0709 13:12:25.743965 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:25.744257 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:25.744281 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:25.744695 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 13:12:25.984453 (+239758us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:26.330493 (+346040us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.754202 (+423709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.754302 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.754351 (+    49us) raft_consensus.cc:1685] finished
0709 13:12:26.754369 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.754513 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16866,"prepare.queue_time_us":163540,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":212311,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":203,"thread_start_us":807383,"threads_started":3,"wal-append.queue_time_us":221155}
I20260709 13:12:26.964655 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:26.966670 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2069) took 2905 ms. Trace:
I20260709 13:12:26.760352 28153 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:26.763729 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:26.969792 28157 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:26.970435 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 422569us
W20260709 13:12:26.974030 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 210246us
I20260709 13:12:26.974669 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:26.967294 28231 rpcz_store.cc:276] 0709 13:12:24.060757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.172374 (+111617us) service_pool.cc:224] Handling call
0709 13:12:24.172839 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:25.636204 (+1463365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:25.636233 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:25.636701 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:12:25.894896 (+258195us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:25.894953 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:25.895045 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.850260 (+955215us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:26.965726 (+115466us) raft_consensus.cc:1685] finished
0709 13:12:26.965779 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.966321 (+   542us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1503766,"raft.queue_time_us":1476303,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":13305,"spinlock_wait_cycles":6819584,"thread_start_us":250779,"threads_started":1,"wal-append.queue_time_us":405653}
I20260709 13:12:26.979225 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2080) took 2231 ms. Trace:
W20260709 13:12:26.983268 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:26.977294 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:26.979907 28158 rpcz_store.cc:276] 0709 13:12:24.747186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.221301 (+474115us) service_pool.cc:224] Handling call
0709 13:12:25.221657 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:25.282891 (+ 61234us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:12:25.788801 (+505910us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b18002bfe40. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:12:25.838711 (+ 49910us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.380105 (+541394us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:26.389080 (+  8975us) log.cc:844] Serialized 52 byte log entry
0709 13:12:26.389232 (+   152us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:26.389498 (+   266us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.389936 (+   438us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.390178 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:26.409642 (+ 19464us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:26.887115 (+477473us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b18002bfe40. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:12:26.978450 (+ 91335us) raft_consensus.cc:1685] finished
0709 13:12:26.978514 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:26.978956 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155680,"prepare.queue_time_us":510992,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":6908,"raft.run_cpu_time_us":627,"raft.run_wall_time_us":112457,"spinlock_wait_cycles":92445184,"thread_start_us":495228,"threads_started":2,"wal-append.queue_time_us":599545}
I20260709 13:12:26.842751 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:26.832717 28756 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:26.992609 30804 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:26.992938 31273 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:12:26.994385 30884 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:26.995604 31276 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:26.828519 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:26.996557 31155 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.997694 31412 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:27.002159 31210 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.005388 31503 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:26.975581 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:27.004037 31501 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:12:27.022269 31377 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:27.013998 31276 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:27.025518 31155 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:27.026320 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51926.4us
I20260709 13:12:27.026821 31385 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:27.027699 31385 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.017720 31503 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.030527 31552 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:27.030993 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.022619 31210 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:27.032140 28081 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:27.021588 30804 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:27.031765 31501 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:27.035745 30884 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.036350 31380 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:12:27.036453 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 193741us
I20260709 13:12:27.036931 31380 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:27.037590 31380 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:27.037469 31377 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.038064 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:27.030043 31503 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.041326 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.025514 31419 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:27.034365 31501 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:27.042207 28085 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:27.042603 31419 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.043604 31501 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.041517 31074 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:27.044976 31074 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:27.045643 31074 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:27.045063 30804 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.041352 31266 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:27.054188 28758 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:27.061077 31266 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:27.061764 31266 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:27.062258 31210 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.062786 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:27.063784 31419 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:27.064062 28762 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 823) took 1455 ms (client timeout 1609 ms). Trace:
I20260709 13:12:27.063768 28153 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.029214 31276 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:27.067207 28152 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:27.064460 28762 rpcz_store.cc:269] 0709 13:12:25.608573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.612475 (+  3902us) service_pool.cc:224] Handling call
0709 13:12:25.782700 (+170225us) spinlock_profiling.cc:243] Waited 8.61 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:27.063629 (+1280929us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b54007008d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:27.063873 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1756266112}
I20260709 13:12:27.090143 31458 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:27.090785 31458 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:27.097252 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 59421.5us
I20260709 13:12:27.101135 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:27.104955 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.108896 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:27.111204 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 575) took 1349 ms. Trace:
I20260709 13:12:27.112119 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:27.113648 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.112888 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.095117 31155 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.123916 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 825) took 1061 ms. Trace:
I20260709 13:12:27.092327 31458 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.130335 28158 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:27.132184 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:27.129581 28760 rpcz_store.cc:276] 0709 13:12:26.062332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.062668 (+   336us) service_pool.cc:224] Handling call
0709 13:12:27.123486 (+1060818us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b5400439ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:27.123703 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":800064384}
I20260709 13:12:27.136461 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.141424 31385 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:27.126157 28081 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
W20260709 13:12:27.127422 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.128047 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:27.144877 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 169581us
I20260709 13:12:27.149540 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:27.153847 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.153697 31445 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:27.161679 31506 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.129995 31458 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.111761 28154 rpcz_store.cc:276] 0709 13:12:25.762071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.773698 (+ 11627us) service_pool.cc:224] Handling call
0709 13:12:25.774062 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:25.800178 (+ 26116us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:26.077984 (+277806us) log.cc:844] Serialized 14 byte log entry
0709 13:12:26.338408 (+260424us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:26.338463 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:26.338509 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:27.110685 (+772176us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b5400255654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:27.110759 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:27.110989 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126834,"prepare.queue_time_us":312736,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":292,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":6431,"spinlock_wait_cycles":59557760,"thread_start_us":407302,"threads_started":2,"wal-append.queue_time_us":343066}
I20260709 13:12:27.163529 31506 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.165299 31377 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:27.176808 31501 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.181789 28158 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.189742 28125 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:27.194000 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 66172.7us
I20260709 13:12:27.195515 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:27.197563 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:27.199745 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:27.194167 30892 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:27.190307 31377 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:27.201380 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:27.201503 31377 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.202487 28758 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:27.201192 30892 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.204228 31377 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.205212 30892 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:27.205981 28762 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:27.194062 31214 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:27.190714 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:27.211491 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 83268.1us
I20260709 13:12:27.223887 28762 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
W20260709 13:12:27.233650 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:27.234344 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:27.240976 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 65432.2us
I20260709 13:12:27.241971 31377 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:27.242491 31377 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.242906 31377 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:27.241087 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 656) took 1027 ms. Trace:
W20260709 13:12:27.249703 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72336.1us
I20260709 13:12:27.249127 28084 rpcz_store.cc:276] 0709 13:12:26.213910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.214031 (+   121us) service_pool.cc:224] Handling call
0709 13:12:26.214396 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:26.955786 (+741390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:26.955871 (+    85us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:26.956983 (+  1112us) log.cc:844] Serialized 26 byte log entry
0709 13:12:27.011834 (+ 54851us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:27.011860 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:27.164517 (+152657us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b540027c9d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:27.167505 (+  2988us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:27.206213 (+ 38708us) raft_consensus.cc:1685] finished
0709 13:12:27.206233 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:27.240869 (+ 34636us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43664,"raft.run_cpu_time_us":405,"raft.run_wall_time_us":406,"spinlock_wait_cycles":1702041472,"thread_start_us":54206,"threads_started":1,"wal-append.queue_time_us":113150}
I20260709 13:12:27.250586 31503 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.257313 31506 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.257925 31210 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:27.259776 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:27.260972 28153 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
W20260709 13:12:27.267921 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 79898.1us
W20260709 13:12:27.270867 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 101936us
I20260709 13:12:27.268616 30892 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:12:27.273713 30892 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:27.274183 30892 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:27.274601 31367 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:27.275861 28758 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:27.263013 31214 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:27.278435 30892 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.285480 30892 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.292116 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:27.279927 31503 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.302457 28158 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:27.308853 31511 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:27.314749 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 62174.6us
I20260709 13:12:27.315284 31419 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.315889 31419 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.317103 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:27.318463 28157 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:27.315918 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:27.321717 31377 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:27.322747 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:27.324095 31242 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:27.324837 31321 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:27.325364 31321 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:27.325028 31242 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:27.325774 31321 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.327169 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:27.330036 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:27.321151 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:27.330451 31214 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.331310 28153 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:27.292112 31155 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:12:27.332978 31511 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:27.324683 31377 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.343067 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:27.344027 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:27.346258 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 43836.3us
I20260709 13:12:27.346596 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.350605 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:27.351333 28083 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:27.355822 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.355794 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:27.353545 28157 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:27.362557 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.364580 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.365867 28081 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:27.342913 31210 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.336009 30804 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:12:27.353958 31419 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.339423 31266 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.383951 28085 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.390110 31511 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:27.391010 31445 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.390837 31074 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.391625 31445 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.392210 31445 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:27.393213 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57615.1us
I20260709 13:12:27.395170 31464 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:12:27.396443 31464 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:27.396845 31464 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:27.396886 31210 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.399518 31210 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.390102 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:27.401177 28759 raft_consensus.cc:2376] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5. Re-sending same reply.
I20260709 13:12:27.396064 31276 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.398043 31464 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.420423 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:27.383955 31501 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.424973 28230 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.390883 31511 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:27.427188 28764 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:27.430358 28158 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:27.430168 31501 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:27.432531 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 100363us
I20260709 13:12:27.434324 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:27.434404 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:27.434827 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:27.435670 28155 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:27.435792 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:27.437171 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:27.437117 31187 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.441185 31187 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:27.441401 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:27.442420 28161 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:27.442901 31266 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.439838 31367 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:27.443825 31367 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:27.445118 31367 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.446966 31367 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:27.451009 28078 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:27.457250 31464 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.429234 31511 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:27.466346 28083 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:27.432614 31501 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:27.472088 31380 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:27.469662 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 74903.9us
I20260709 13:12:27.441646 31187 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.467213 31242 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:27.484477 28230 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:27.473280 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:27.488083 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:27.489533 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:27.491199 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:27.492471 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:27.493681 28757 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:27.493815 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:27.495105 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:27.496346 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:27.497367 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:27.498828 28761 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:27.499743 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.500633 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:27.509462 28758 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:27.509577 28755 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:27.503350 28198 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:27.507089 28231 raft_consensus.cc:2376] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 8. Re-sending same reply.
I20260709 13:12:27.515784 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:27.522013 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 86369.4us
I20260709 13:12:27.524283 28085 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:27.526010 28083 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:27.526955 28757 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:27.527061 28756 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:27.528308 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:27.528748 28757 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:27.529136 28232 raft_consensus.cc:2376] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 8. Re-sending same reply.
I20260709 13:12:27.530462 28757 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:27.533468 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:27.536453 31380 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.536762 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:27.544070 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:27.545382 28050 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:27.545523 31263 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:27.546557 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:27.547359 30804 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:27.548403 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:27.550112 28761 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:27.551146 28125 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:27.551499 28761 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:27.552078 30804 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:27.552358 28125 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:27.504343 31074 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:27.554436 31074 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:27.552567 30804 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.554903 31074 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:27.555393 30804 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.555527 31074 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.525286 31242 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:27.557114 31503 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:27.557382 31242 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.557598 31503 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:27.557860 31242 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:27.548486 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.566629 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:27.567427 28230 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:27.580687 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.581996 31210 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:27.582537 31210 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.582962 31210 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:27.585177 28127 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.632s (SENT)
I20260709 13:12:27.586164 28079 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:12:27.586376 28127 leader_election.cc:336] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.652s (SENT)
I20260709 13:12:27.586848 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:27.587706 31511 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.588419 31321 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.590704 31506 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:27.590864 31530 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.591415 31530 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:27.592166 28764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 826) took 1526 ms (client timeout 1664 ms). Trace:
W20260709 13:12:27.592576 28764 rpcz_store.cc:269] 0709 13:12:26.066204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.066368 (+   164us) service_pool.cc:224] Handling call
0709 13:12:27.591973 (+1525605us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400512cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:27.592107 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2724912512}
I20260709 13:12:27.593918 31530 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:27.588835 31458 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:27.594913 28760 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:27.595353 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:27.566298 31501 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:27.597379 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:27.601552 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:27.603472 28049 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:27.600500 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:27.590725 31321 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:27.649763 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.623991 31501 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:27.629820 31458 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:27.659657 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:27.648339 31263 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:27.648643 31385 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:12:27.668913 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 135708us
W20260709 13:12:27.673614 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 74535.7us
I20260709 13:12:27.654773 31276 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:27.713380 31276 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:27.714126 31276 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.720501 30804 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.680096 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:27.723367 28233 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:27.658105 31321 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:27.681063 28234 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:27.681950 31570 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:27.725497 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 102070us
W20260709 13:12:27.683562 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 86261.2us
I20260709 13:12:27.682410 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:27.685127 31385 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:27.685609 31506 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.686107 31155 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:27.688710 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:27.689498 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:27.695258 31456 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:27.701067 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:27.702170 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:27.733311 28228 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:27.630275 31503 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.707139 31187 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.709069 31511 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:27.709452 31458 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:27.710402 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:27.711763 31445 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:27.726193 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.753634 28232 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:27.754077 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:27.667382 31530 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:27.729696 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:27.749008 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:27.758318 28231 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:27.761162 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:27.725697 31570 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:27.767303 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 180969us
I20260709 13:12:27.731328 31456 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:27.777021 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.779682 28763 raft_consensus.cc:2368] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:27.780148 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:27.783612 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 65455.7us
I20260709 13:12:27.784132 31338 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:27.786477 31338 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:27.787115 31338 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:27.760035 31380 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.791630 28232 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:27.796775 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 72133.7us
I20260709 13:12:27.779038 31263 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:27.778053 31503 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:27.767724 31276 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:27.769671 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:27.779542 31552 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:12:27.822029 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61004.7us
W20260709 13:12:27.821009 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 110744us
I20260709 13:12:27.777166 31377 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:27.754077 31506 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:27.836143 28127 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.734s (SENT)
W20260709 13:12:27.836318 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:27.770191 31570 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:27.856016 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:27.862931 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 129162us
I20260709 13:12:27.875589 30892 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.876307 28227 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:27.779788 31530 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.866288 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:27.838272 31574 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:27.889571 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:27.882546 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 62532.8us
I20260709 13:12:27.881059 28226 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:27.892154 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:27.890691 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 60238.1us
I20260709 13:12:27.838181 30804 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.894845 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:27.824402 31511 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:12:27.821682 31380 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:27.926184 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 137018us
I20260709 13:12:27.888818 31276 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:27.890009 31552 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:27.942342 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 60813.8us
I20260709 13:12:27.891260 31574 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:27.889842 30884 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:12:27.876364 30892 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:27.948246 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 81053.6us
W20260709 13:12:27.948720 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 91410.5us
I20260709 13:12:27.836048 31338 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.958822 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:27.938659 31377 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:27.956058 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 267475us
I20260709 13:12:27.880903 31385 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.856132 31501 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:12:27.977824 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 83150.8us
I20260709 13:12:27.947589 31574 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:27.970567 30804 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:27.984812 31370 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:27.987285 31380 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:27.987566 31547 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:27.988886 31346 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:12:27.990177 31346 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:27.990631 31346 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:27.991746 31346 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.993006 31187 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:27.986937 31242 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:27.992795 28235 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:27.952716 31419 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:27.947527 31276 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:28.004885 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 176347us
I20260709 13:12:27.992794 31581 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.199s
I20260709 13:12:27.973634 31321 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:28.013103 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:28.019798 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.018389 31591 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:28.027909 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49248us
I20260709 13:12:27.997524 31370 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:28.034009 28124 leader_election.cc:400] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:28.037364 28124 leader_election.cc:403] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:28.009557 30884 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:28.042222 31589 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:12:28.045644 31321 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.046523 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:28.047083 31461 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:12:28.047712 31578 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.152s
I20260709 13:12:28.048673 31276 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:28.048903 31538 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 13:12:28.053501 31576 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:12:28.053831 31242 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.055751 28226 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:27.970702 31263 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:28.094728 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48294.7us
I20260709 13:12:28.095711 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:28.097438 28226 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:28.102324 31547 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:28.105547 28199 leader_election.cc:400] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:28.105881 28199 leader_election.cc:403] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:12:28.106542 31511 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:28.019316 31187 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:12:28.069092 28228 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:28.070254 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:27.997262 31210 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:28.024367 31445 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:28.091833 28231 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:28.105180 31581 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:28.063894 30884 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:28.109715 30804 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:28.120957 28085 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
W20260709 13:12:28.130291 31579 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:12:28.130492 31594 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:28.131608 31598 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:12:28.133170 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:28.133941 28231 raft_consensus.cc:2368] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8.
I20260709 13:12:28.134037 30819 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:28.134473 30819 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:28.135432 30819 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.137020 31599 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:28.084668 31370 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:28.139611 31503 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:12:28.140873 31577 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
W20260709 13:12:28.143669 31585 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:12:28.146284 31530 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.149591 30892 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:28.150931 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:28.127624 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 236763us
W20260709 13:12:28.093852 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 123507us
I20260709 13:12:28.020262 31377 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
W20260709 13:12:28.163952 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 68313.2us
I20260709 13:12:28.164083 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:28.158200 30819 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:28.182715 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 122806us
W20260709 13:12:28.183713 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 177341us
I20260709 13:12:28.184665 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:28.186542 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:27.988598 31552 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:28.157737 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 181398us
I20260709 13:12:28.190708 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.186754 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:28.128279 30884 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:28.187915 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.061406 28233 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:28.037631 31591 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.093266 31276 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:12:28.128973 31506 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.136304 31570 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.191094 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:28.201231 31594 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:28.202188 31187 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:12:28.205797 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.172466 31412 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:28.214315 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:28.215229 28226 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:28.214069 28233 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:28.193931 31547 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:28.218824 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 54866.9us
I20260709 13:12:28.220510 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.234023 28235 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:28.232767 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.239581 28226 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:28.239063 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:28.240823 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:28.225548 31187 raft_consensus.cc:1068] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:12:28.225171 31530 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:28.248063 28049 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:28.231053 31321 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.282123 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.288019 28231 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:28.287281 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 67597.4us
I20260709 13:12:28.249739 31530 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:28.257617 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:28.311939 28756 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:12:28.312311 28127 leader_election.cc:336] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.936s (SENT)
W20260709 13:12:28.313866 28127 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.052s (SENT)
I20260709 13:12:28.314117 31602 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:28.314311 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:28.314692 31602 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.315117 31602 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:28.316270 31605 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:28.316972 31605 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:28.320617 31601 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:12:28.322743 28232 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:28.329111 31591 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.291738 31604 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:28.256656 31370 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:12:28.296003 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57023.8us
I20260709 13:12:28.246505 31266 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:12:28.281585 31464 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:28.292258 31595 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.304953 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.306051 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.261965 31321 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:28.346124 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.352161 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:28.350339 31155 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:28.365245 31155 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:28.366322 31155 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.366930 31574 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:28.312163 31530 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.360939 31377 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:28.369026 31604 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:28.369603 31530 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:28.351281 31605 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.306959 31501 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:28.392174 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 156879us
W20260709 13:12:28.393658 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.334344 31578 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:28.350672 31370 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:28.360905 31210 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:28.364470 31591 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:28.392628 31074 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.816s	user 0.015s	sys 0.000s
I20260709 13:12:28.350826 31506 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.401058 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:28.404564 28049 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:28.405977 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.411365 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.415207 28231 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:28.398816 31464 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:28.435400 31551 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:28.435909 31551 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:28.436105 31464 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:28.438129 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.439409 31377 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:28.440917 31605 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:28.441574 31605 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.445060 31380 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:28.445649 31380 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:28.446028 31456 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.446630 31380 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:28.449271 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 52526.4us
I20260709 13:12:28.449877 31412 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:12:28.452576 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 138758us
I20260709 13:12:28.460070 28231 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:28.460357 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:28.456272 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:28.440092 31377 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:28.464174 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.452703 31412 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:28.485003 28157 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:28.500656 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:28.509327 28154 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:28.505555 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:28.476263 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.486622 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:28.473654 31570 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:28.363206 31595 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.464120 31605 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.468920 31266 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:28.522392 28235 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:28.522696 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.416028 31074 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:28.529429 28231 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:28.525386 28155 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:28.532056 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 179979us
I20260709 13:12:28.414698 31591 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:28.535066 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 74810.9us
I20260709 13:12:28.370357 31604 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.464529 31506 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:28.412000 31501 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.549232 28232 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:28.552345 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:28.527513 31266 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:28.537547 28085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:12:28.613514 28085 raft_consensus.cc:606] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:28.607905 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:28.607368 31412 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.534382 31605 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.535074 31338 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.428774 31578 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.630574 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:28.630942 31614 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:12:28.631625 28235 raft_consensus.cc:2410] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:28.631748 31578 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.617136 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:28.633064 31618 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:12:28.541952 31574 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
W20260709 13:12:28.624976 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 274471us
W20260709 13:12:28.627053 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.524813 30884 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:28.636190 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:28.659947 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.528227 31370 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:28.674659 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:28.504750 31551 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.686395 28235 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:28.689363 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:28.635164 31605 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.699543 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:28.693097 31511 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.007s
W20260709 13:12:28.705603 28127 leader_election.cc:336] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.350s (SENT)
W20260709 13:12:28.698283 31419 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.674s	user 0.012s	sys 0.000s
I20260709 13:12:28.649081 31338 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.692878 31321 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:28.711508 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:28.713831 28085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
W20260709 13:12:28.711385 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 175446us
I20260709 13:12:28.714298 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.715169 28231 raft_consensus.cc:2410] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:12:28.714191 31380 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.702863 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:28.708511 31511 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:28.726145 28758 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:28.726195 28226 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:28.728529 28755 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:28.730877 31608 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:28.731546 31608 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:28.732092 28230 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:28.733064 28233 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:28.733659 31628 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:12:28.748355 28231 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:28.732532 31608 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:28.712707 30884 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:28.713481 31419 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:12:28.754206 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53573.3us
I20260709 13:12:28.718083 31578 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:28.786720 28231 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:12:28.733747 31380 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.745251 31377 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.707378 31266 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.772477 31595 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:28.804586 31626 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:28.805912 31623 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:12:28.806612 31621 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:12:28.817587 31591 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.819016 31501 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.821528 31506 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:28.827842 31456 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:28.829972 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:28.868307 28155 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:28.802877 31629 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.796491 28231 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:28.840638 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:28.820451 31338 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:28.795228 31074 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:28.849716 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.854723 28758 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:12:28.839026 31503 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.001s
W20260709 13:12:28.857959 31620 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:12:28.858812 31380 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:28.859473 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:28.865738 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:28.838517 31242 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:28.916940 31242 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:28.917798 31346 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:28.866596 28230 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:28.919836 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:28.854140 30819 consensus_meta.cc:287] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.685s	user 0.005s	sys 0.009s
I20260709 13:12:28.888356 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:28.894912 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:28.866173 31551 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:28.897835 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:28.860350 31582 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:12:28.841313 31625 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:12:28.910334 28156 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:28.913040 31615 raft_consensus.cc:2764] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:12:28.912909 28756 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:28.886694 31629 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.877820 31626 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:28.910267 31547 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.001s
I20260709 13:12:28.926595 28085 raft_consensus.cc:606] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:28.947000 28085 raft_consensus.cc:614] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:12:28.926019 28127 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.606s (SENT)
I20260709 13:12:28.947721 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:28.897274 31503 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
W20260709 13:12:28.933843 31602 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.618s	user 0.014s	sys 0.000s
W20260709 13:12:28.934373 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 397469us
I20260709 13:12:28.936003 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.936870 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.921501 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.963331 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.964226 28230 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:28.965161 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.965997 28230 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:12:28.966786 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:28.967933 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.918318 31242 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:28.969184 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.969590 31242 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:28.970078 28231 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:28.971016 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.974720 31629 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:28.975610 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:28.976593 28235 raft_consensus.cc:2410] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:28.977665 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.979569 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:28.981536 31377 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:28.983939 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:28.985010 28153 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:28.972155 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:28.968565 28230 raft_consensus.cc:2368] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:28.988284 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:28.991385 28155 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:28.946517 31506 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:28.996654 31643 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:28.997107 31506 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.997450 31643 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:28.951753 31547 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:29.002156 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:29.003104 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.004415 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.005412 28235 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:29.007619 28231 raft_consensus.cc:2368] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10.
I20260709 13:12:29.016424 28084 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:28.987073 31602 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:29.052807 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:29.135447 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.017652 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:29.029165 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.052879 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:29.145521 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.053709 28760 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:29.087179 28159 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:29.142836 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:29.149696 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.153015 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.155393 28231 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:29.156937 28230 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
W20260709 13:12:29.021644 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 351611us
I20260709 13:12:29.161262 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.169256 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.170181 28228 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:29.173353 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:29.177652 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:29.186728 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.185642 28080 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 4.
I20260709 13:12:29.150521 31506 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:29.095644 31640 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:29.418196 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.569084 28077 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:29.593240 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.594174 28230 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:29.205626 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.461938 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2124) took 1261 ms. Trace:
I20260709 13:12:29.599233 28232 rpcz_store.cc:276] 0709 13:12:28.200722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.324548 (+123826us) service_pool.cc:224] Handling call
0709 13:12:28.335473 (+ 10925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:28.550333 (+214860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:29.461566 (+911233us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54000dedd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:29.461817 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488582400}
I20260709 13:12:29.601519 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.603191 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:29.461514 31530 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.091s	user 0.015s	sys 0.000s
I20260709 13:12:29.610751 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.304356 28227 raft_consensus.cc:2410] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:29.618100 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:29.639194 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:29.666489 28226 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:29.605530 28127 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.053s (SENT)
I20260709 13:12:29.199002 31616 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:29.620555 31610 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 13:12:29.655962 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.666980 28231 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:29.711995 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.669888 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:29.573015 31626 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:29.571954 31338 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.746263 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.753785 28230 raft_consensus.cc:2410] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:12:29.674548 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:29.194942 31412 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.470s
I20260709 13:12:29.722584 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.723835 31276 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:29.658588 31643 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:29.736141 31616 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:29.653731 31506 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.751500 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:29.673659 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:29.766613 28230 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:29.784555 28230 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:29.770241 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:29.789399 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 183937us
I20260709 13:12:29.793303 28226 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:29.790491 28127 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.625s (SENT) [suppressed 26 similar messages]
I20260709 13:12:29.762341 31530 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:29.767315 31608 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:29.817988 28728 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.015s	sys 0.065s
I20260709 13:12:29.817811 28228 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:12:29.819828 28728 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.015s	sys 0.065s
I20260709 13:12:29.782152 31643 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.826296 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:29.832294 28161 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:29.811655 28230 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:12:29.835713 28727 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.008s	sys 0.017s
W20260709 13:12:29.836184 28727 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.009s	sys 0.017s
W20260709 13:12:29.836781 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:29.826387 28127 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.677s (SENT)
I20260709 13:12:29.840690 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:29.768836 31338 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:29.851449 28158 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:29.788084 31276 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:29.795497 31506 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:29.840154 31346 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.031s	sys 0.062s
I20260709 13:12:29.861852 31530 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:29.880422 28160 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:12:29.881125 28160 pending_rounds.cc:85] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 4
I20260709 13:12:29.913980 28160 pending_rounds.cc:107] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:12:29.914424 28160 raft_consensus.cc:2889] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:29.917529 31276 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:29.918460 31276 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.919129 31276 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:29.926812 31506 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:29.839470 31643 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:29.938019 28156 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:29.870479 31602 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:29.827113 31608 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:29.947141 28161 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:29.950788 31616 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:29.952201 31338 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:29.954751 31602 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.955392 31602 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:29.959678 31608 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:29.957762 31602 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:29.964550 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 138299us
I20260709 13:12:29.977022 28230 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:29.982218 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:29.983707 28127 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.873s (SENT)
I20260709 13:12:29.994541 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:29.997893 31551 consensus_meta.cc:287] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.066s	user 0.010s	sys 0.004s
W20260709 13:12:30.016894 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:30.009605 31651 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:12:30.009809 31608 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.030771 31530 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:12:30.032488 31643 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:30.026923 31551 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:30.059937 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 76282.2us
I20260709 13:12:30.061267 31661 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.061977 31661 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.064395 31661 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:30.041922 31608 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.065871 31276 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:30.049845 31551 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.067013 31665 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:30.085631 28127 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.928s (SENT)
I20260709 13:12:30.086407 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:30.092639 28230 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:12:30.093155 28230 pending_rounds.cc:85] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 3
I20260709 13:12:30.093498 28230 pending_rounds.cc:107] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:12:30.093779 28230 raft_consensus.cc:2889] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:30.103055 31412 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:30.103509 31551 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:30.103839 31412 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.105917 31608 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:30.107141 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 129897us
I20260709 13:12:30.094064 31666 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:30.113052 31632 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.308s	user 0.008s	sys 0.011s
I20260709 13:12:30.088958 31665 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.126605 31412 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:30.129630 31665 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:30.136025 31461 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.136657 31461 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.139361 31461 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:30.142304 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 594) took 1479 ms. Trace:
I20260709 13:12:30.142783 28755 rpcz_store.cc:276] 0709 13:12:28.662627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.671185 (+  8558us) service_pool.cc:224] Handling call
0709 13:12:30.142059 (+1470874us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b5400439ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:30.142229 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72274176}
I20260709 13:12:30.145118 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43764 (request call id 28) took 1434 ms. Trace:
I20260709 13:12:30.145514 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.147598 28083 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:30.154095 28161 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:30.145519 28085 rpcz_store.cc:276] 0709 13:12:28.710949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.713691 (+  2742us) service_pool.cc:224] Handling call
0709 13:12:30.144914 (+1431223us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54002add54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce671853 00007f59d363f60a 00007f59ccbc57d4 00007f59ccbc5762 00007f59ccbc5691 00007f59ccbc560c 00007f59ccbc48c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:30.145031 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487820032}
I20260709 13:12:30.146028 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.157051 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.157819 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.159053 28083 raft_consensus.cc:2410] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:12:30.160125 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:30.161033 28160 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:30.164845 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.165373 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:30.168160 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:30.170778 28233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1736) took 1779 ms (client timeout 1577 ms). Trace:
I20260709 13:12:30.171238 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:30.173341 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.177364 28232 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:30.180577 28161 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:30.171386 28233 rpcz_store.cc:269] 0709 13:12:28.391205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.399233 (+  8028us) service_pool.cc:224] Handling call
0709 13:12:30.170474 (+1771241us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b5400306e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:30.170638 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684993792}
I20260709 13:12:30.180325 28086 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
W20260709 13:12:30.181819 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.157869 28078 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:30.197154 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:30.199038 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 63361.3us
I20260709 13:12:30.198424 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.201280 28085 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:30.201583 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.200635 28082 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:30.198424 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.207442 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:30.208627 28757 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:30.210057 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.200855 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:30.214056 28755 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:30.218569 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.221956 28763 raft_consensus.cc:2410] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:12:30.210878 28077 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:30.205505 31506 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.245014 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.245410 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:30.246469 28161 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 4.
I20260709 13:12:30.247115 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:30.221171 28050 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:30.207574 31530 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.207168 31608 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.248940 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.251266 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:30.255839 28081 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:30.255203 31647 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:30.258373 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.259078 31461 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:30.259806 28757 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:12:30.251076 31666 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
W20260709 13:12:30.263453 28127 leader_election.cc:336] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.563s (SENT)
I20260709 13:12:30.263994 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:30.224491 31636 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.010s	sys 0.052s
W20260709 13:12:30.268500 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 61676.9us
W20260709 13:12:30.270447 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 67533.5us
I20260709 13:12:30.272081 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:12:30.271440 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:30.272845 28078 raft_consensus.cc:606] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:30.273869 28078 raft_consensus.cc:614] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:12:30.268983 31636 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.010s	sys 0.053s
I20260709 13:12:30.274194 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:30.274370 28127 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.676s (SENT)
I20260709 13:12:30.276033 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:30.276055 28152 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 5.
W20260709 13:12:30.278605 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.278896 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:30.268846 31672 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:30.270153 31631 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:30.281148 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 76327.3us
I20260709 13:12:30.272413 31263 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:30.282253 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:30.251189 31506 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.273764 31665 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:30.283059 31677 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:30.283838 31677 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:30.283524 31665 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.283063 31464 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:30.284617 31464 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.285136 31464 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:30.286226 31665 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:30.287000 31263 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.266278 31530 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.289796 31263 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:30.291106 31530 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:30.291821 31530 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:30.292269 31530 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.293087 28156 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:30.294569 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:30.295173 28156 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:30.296062 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:30.265559 31608 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.284600 31506 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:30.298026 31506 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:30.298576 31608 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:30.299386 31631 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:30.300439 31631 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.301378 28230 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:30.302685 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:30.303191 28233 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:30.304366 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:30.298676 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:30.306708 28160 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:30.306787 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:30.307595 28230 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:30.307998 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:30.296604 28156 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 5.
I20260709 13:12:30.317324 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:30.318539 28230 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:30.321681 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:30.444070 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:30.321297 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:30.292658 31530 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:30.381338 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:30.335053 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.340102 28152 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:30.450572 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.452508 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.453169 28761 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:30.454321 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:30.322561 28159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 601) took 1850 ms (client timeout 1538 ms). Trace:
I20260709 13:12:30.401383 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.444585 28232 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:30.445508 31506 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:30.445755 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44122.2us
I20260709 13:12:30.450086 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
W20260709 13:12:30.432775 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.450500 28160 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:30.460932 28078 raft_consensus.cc:606] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:30.461413 28078 raft_consensus.cc:614] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:30.445537 28228 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
W20260709 13:12:30.455511 28052 connection.cc:256] RPC call timeout handler was delayed by -1.4419724659999247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
I20260709 13:12:30.284648 31677 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.182s
I20260709 13:12:30.311684 30892 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.156s
I20260709 13:12:30.401578 28762 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 5 LEARNER]: Advancing to term 6
I20260709 13:12:30.319873 31570 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:30.444132 28234 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:30.405956 31647 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:30.455917 28159 rpcz_store.cc:269] 0709 13:12:28.471960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.472472 (+   512us) service_pool.cc:224] Handling call
0709 13:12:30.322182 (+1849710us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b540036d154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:30.322310 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849695488}
W20260709 13:12:30.462927 28052 connection.cc:256] RPC call timeout handler was delayed by -1.2763724659998843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
I20260709 13:12:30.473181 28763 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:30.495096 28229 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:30.475205 28234 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:30.504031 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:30.581513 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:30.495405 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 221065us
W20260709 13:12:30.504616 28227 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1763) took 1638 ms (client timeout 1606 ms). Trace:
I20260709 13:12:30.507230 30804 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:30.583473 30804 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:30.518118 28762 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 6 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:12:30.529040 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.584910 28762 pending_rounds.cc:85] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 3
I20260709 13:12:30.585173 28762 pending_rounds.cc:107] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:12:30.539109 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.544683 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.585510 28762 raft_consensus.cc:2889] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 6 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:30.477594 28156 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 13:12:30.477311 28158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 597) took 2021 ms (client timeout 2048 ms). Trace:
I20260709 13:12:30.481737 28159 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:12:30.588281 28158 rpcz_store.cc:269] 0709 13:12:28.455806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.455928 (+   122us) service_pool.cc:224] Handling call
0709 13:12:30.477107 (+2021179us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400317754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:30.477249 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2959242112}
I20260709 13:12:30.580036 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1735) took 2270 ms. Trace:
I20260709 13:12:30.507436 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1739) took 1974 ms. Trace:
I20260709 13:12:30.528584 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.590801 28084 rpcz_store.cc:276] 0709 13:12:28.539235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.539393 (+   158us) service_pool.cc:224] Handling call
0709 13:12:28.612749 (+ 73356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:29.017170 (+404421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:30.507101 (+1489931us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54002ace54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:30.507334 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":927684864}
I20260709 13:12:30.463610 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:30.487253 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:12:30.592900 28078 raft_consensus.cc:606] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:30.593072 28127 leader_election.cc:336] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.569s (SENT)
I20260709 13:12:30.593391 28078 raft_consensus.cc:614] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:30.463131 28761 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:30.589705 28229 rpcz_store.cc:276] 0709 13:12:28.310034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.329955 (+ 19921us) service_pool.cc:224] Handling call
0709 13:12:30.494539 (+2164584us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b58001c0598. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:12:30.494834 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:30.579750 (+ 84916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:30.579972 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12543232}
W20260709 13:12:30.583277 28227 rpcz_store.cc:269] 0709 13:12:28.866329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:28.897440 (+ 31111us) service_pool.cc:224] Handling call
0709 13:12:30.504386 (+1606946us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54000e8654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:30.504543 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682632064}
W20260709 13:12:30.600062 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:30.650115 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73542.8us
I20260709 13:12:30.538388 31605 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:30.657243 28234 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.658380 28234 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.658252 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:30.660326 28235 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:30.661353 28079 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:30.662804 31242 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.663348 31242 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:30.655711 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 62629.3us
I20260709 13:12:30.665743 31242 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:30.467559 31677 consensus_queue.cc:1243] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:12:30.667295 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:30.668514 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:30.670291 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 206751us
I20260709 13:12:30.673767 28052 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:30.678534 28158 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:30.463004 31586 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.512902 31672 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.669914 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:30.705266 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.689079 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.580681 31570 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.708029 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:30.594885 31321 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.715175 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.717077 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:30.716207 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:30.539713 31266 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:30.462404 31678 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.718151 28158 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:30.718391 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:30.517092 31370 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:30.524748 31263 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:12:30.757028 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 83368.2us
I20260709 13:12:30.713673 31672 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:30.733536 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:30.733615 28081 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:30.717597 31684 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.737993 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.680447 31668 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:30.504040 31580 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.224s
I20260709 13:12:30.715051 31570 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:30.515220 31530 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.503002 31464 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.712970 31445 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:12:30.527514 31412 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.815331 31412 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.817986 31412 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:30.736930 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 465520us
I20260709 13:12:30.818775 28152 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:30.819244 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:30.723680 31587 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:30.467743 31506 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.819981 31587 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:30.820420 31587 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:30.821900 31506 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:30.678534 31155 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:12:30.733675 31625 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:30.823068 31625 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.823500 31625 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:30.824867 31631 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:30.757877 31678 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.768908 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:12:30.784345 28127 leader_election.cc:336] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.881s (SENT)
I20260709 13:12:30.827486 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:30.763217 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:30.828141 31678 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:30.828462 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 65309.6us
I20260709 13:12:30.829151 31412 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:30.829707 31412 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:30.830523 31412 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.713351 31671 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:12:30.832433 31671 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:30.832787 31445 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:30.833199 31445 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.833529 31445 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:30.832811 31671 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:30.812366 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:30.481381 31647 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.835181 28081 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:30.835328 31671 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.792213 31684 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.837524 31671 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:30.837761 31587 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.688452 31242 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:12:30.831625 31412 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:30.834172 31530 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:30.712908 31586 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.839066 31684 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:30.814044 31263 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:30.846505 31464 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:30.792685 30804 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.852545 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.846386 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:30.765499 31346 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:30.847509 31586 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:30.775215 31602 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:30.865437 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.866132 28761 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:30.737804 31456 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 13:12:30.774470 31668 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.869295 28081 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:12:30.730208 31321 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:30.872280 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:30.873453 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.874401 31587 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:30.868690 31602 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:30.923122 28229 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:12:30.874040 31600 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:30.921793 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 250541us
I20260709 13:12:30.873641 31346 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:30.924934 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.924129 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:30.875280 31668 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:30.965548 28234 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:12:30.967151 28234 pending_rounds.cc:85] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 4
I20260709 13:12:30.967444 28234 pending_rounds.cc:107] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:12:30.967726 28234 raft_consensus.cc:2889] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:30.899467 31263 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:30.897579 31580 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:30.973835 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:30.924674 31602 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:30.975715 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:30.979390 28124 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.008s	sys 0.034s
W20260709 13:12:30.980610 28124 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.008s	sys 0.034s
W20260709 13:12:30.979664 28154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.005s	sys 0.036s
W20260709 13:12:30.981182 28154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.005s	sys 0.037s
I20260709 13:12:30.926342 31600 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:30.981596 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 711593us
I20260709 13:12:30.984161 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:30.986106 28079 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:30.918237 31380 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:12:30.973402 31580 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:30.995442 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 490364us
I20260709 13:12:30.993297 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:30.985244 31600 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:12:30.971504 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 161823us
I20260709 13:12:30.999889 31580 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:31.020112 28079 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:31.023471 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.014482 31637 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
W20260709 13:12:31.024703 31680 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:12:31.029640 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:31.037465 31681 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:12:31.038653 31692 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.039211 31692 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:31.039613 31321 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:31.041764 31692 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:31.041821 31667 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.042423 31667 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.043022 31692 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:31.043138 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:31.043550 31692 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.043957 31692 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:31.047171 28078 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:31.048928 28086 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:12:31.049357 28086 pending_rounds.cc:85] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 8
I20260709 13:12:31.049595 28086 pending_rounds.cc:107] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:12:31.049853 28086 raft_consensus.cc:2889] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:31.026398 31346 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.062104 31691 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.062861 31691 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.066090 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.073011 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.073938 28083 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:31.081015 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.083839 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.081223 31691 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:31.085343 28080 raft_consensus.cc:2410] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:12:31.094264 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 78505.7us
W20260709 13:12:31.097589 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.099972 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:31.101114 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.035579 31693 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:31.109668 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.119445 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.120414 28227 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:31.127725 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 308499us
I20260709 13:12:31.131731 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:31.134370 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.067684 31445 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.135484 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.135900 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.130501 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:31.137451 28232 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:31.140569 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:31.141337 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 357024us
W20260709 13:12:31.142156 28127 leader_election.cc:336] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.933s (SENT)
I20260709 13:12:31.142630 28127 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:31.137984 28232 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:31.141580 28227 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:31.147316 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.149334 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.146533 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.150673 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:31.155200 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.156226 28233 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:31.157131 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.152073 28226 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:31.156144 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:12:31.169494 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.169807 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.170408 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.170782 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.170994 28227 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:31.131326 31551 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:31.173601 28231 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 5: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:31.173889 28228 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 6: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:31.178320 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.143378 31666 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:31.176091 28235 raft_consensus.cc:2368] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:31.146116 31456 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:31.138684 31506 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:31.173310 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 607) took 1024 ms. Trace:
I20260709 13:12:31.177106 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.176542 28232 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:31.171284 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:31.193037 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:31.130839 31696 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:12:31.168471 31671 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:31.205302 28232 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:31.204913 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 280816us
I20260709 13:12:31.190923 28226 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:31.138779 31670 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:12:31.209465 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:31.143635 31646 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:12:31.135776 31603 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
W20260709 13:12:31.215569 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.215402 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 231310us
I20260709 13:12:31.114482 31693 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:31.181227 31640 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:12:31.189627 31506 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.201565 31613 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:31.243537 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:31.153477 31597 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
W20260709 13:12:31.201653 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 66493.3us
I20260709 13:12:31.255265 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.249338 28760 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:12:31.247730 28124 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:31.251782 28228 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:12:31.212534 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 70419.3us
I20260709 13:12:31.269305 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:31.251334 31678 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.213547 31671 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:31.277118 31671 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:31.241583 31608 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:31.282696 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.236517 31634 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:31.182372 31665 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:31.292826 28234 raft_consensus.cc:2410] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:12:31.302790 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.251013 31506 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:31.300387 28077 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
W20260709 13:12:31.292922 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 99966.4us
I20260709 13:12:31.322140 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.323572 28081 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:31.324690 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.325752 28231 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:31.327139 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.328150 28077 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:31.329478 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.332643 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.304316 31456 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:31.319870 31689 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 13:12:31.340583 28077 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:12:31.246006 31625 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.345369 28235 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:31.303409 31665 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:31.339206 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.302627 31634 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:31.302500 31666 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:31.346807 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.346835 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:31.361241 31697 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:12:31.361740 31594 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:31.370805 28080 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:31.337858 31605 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:31.362188 31594 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.341457 31530 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:31.360018 31370 consensus_meta.cc:287] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.595s	user 0.006s	sys 0.006s
I20260709 13:12:31.292801 31690 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
W20260709 13:12:31.384964 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 89603.2us
I20260709 13:12:31.357256 31634 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.357375 31665 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:31.244233 31685 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:12:31.344547 31608 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:31.325539 31662 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:12:31.331434 31683 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:12:31.337568 31696 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:12:31.406893 28232 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:12:31.460317 28232 pending_rounds.cc:85] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 3
I20260709 13:12:31.394126 31594 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:31.344815 31700 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
W20260709 13:12:31.505885 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 43778.7us
I20260709 13:12:31.460598 28232 pending_rounds.cc:107] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:12:31.507124 28232 raft_consensus.cc:2889] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:31.386737 31647 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:12:31.446534 31445 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:31.435920 31530 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.522399 31719 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.458415 31605 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:31.461283 31608 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:31.542980 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.463364 31692 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:31.533159 31699 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.296s	user 0.001s	sys 0.001s
I20260709 13:12:31.561417 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:31.545789 31703 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:12:31.572373 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.574324 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.575220 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.576462 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.589378 28158 raft_consensus.cc:2393] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 8: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:31.589644 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.589653 28157 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:31.577083 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.577924 31608 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:31.552829 31704 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:12:31.597029 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.598165 28155 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:31.598404 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.599888 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.600073 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.600322 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.600646 28155 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:31.601632 28160 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:31.602974 28160 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:12:31.603379 31412 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.749s	user 0.007s	sys 0.002s
I20260709 13:12:31.600081 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.605548 28159 raft_consensus.cc:2410] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:12:31.605491 31693 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:31.588111 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.588660 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:31.590389 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.617297 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.617437 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.591509 28156 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:31.582404 31692 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:31.597008 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.596930 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.620013 28157 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.620121 28161 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:31.552603 31700 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:31.588060 31702 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:31.609719 31705 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
W20260709 13:12:31.610642 31709 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:12:31.648000 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:31.611922 31720 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:12:31.651881 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.612913 28155 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:31.613564 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:31.586408 31672 consensus_meta.cc:287] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.810s	user 0.010s	sys 0.000s
I20260709 13:12:31.600642 28153 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:31.637632 31580 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.002s
I20260709 13:12:31.638100 31412 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:31.639914 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.654767 28157 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:31.655144 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.639117 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:31.644883 31702 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:31.532286 31719 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:31.656450 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.663082 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.666977 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.668562 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.662781 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:31.674233 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.676551 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.685019 28079 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:31.658771 31625 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:31.689589 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.692998 28158 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:31.620560 31608 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:12:31.661559 31603 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:31.695482 28079 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:31.699900 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:31.663462 31711 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:31.699993 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:31.649610 31700 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.705786 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.710592 28084 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:31.707511 28077 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:31.681555 31696 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:12:31.713042 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 252443us
I20260709 13:12:31.713989 28763 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:31.712666 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:31.715657 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:31.715895 28760 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:31.716343 28756 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:31.719305 31710 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:31.719749 31710 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:31.660861 31412 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:31.724642 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.728787 28157 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:31.729650 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.725394 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.725812 28152 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 8. Ops: [6.5-6.5]
I20260709 13:12:31.730651 28155 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:31.726863 28159 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:31.727257 31713 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:12:31.702826 31602 consensus_meta.cc:287] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.714s	user 0.008s	sys 0.003s
I20260709 13:12:31.732697 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:31.733647 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:31.660811 31672 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.746943 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.697306 31603 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:31.750885 28078 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:31.751467 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:31.661881 31580 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.737367 31678 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:31.756283 28156 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:31.752223 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.710250 31530 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:31.765901 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.766597 28152 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.767232 31580 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:31.768808 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:31.769538 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 45891.6us
I20260709 13:12:31.762004 28160 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:12:31.760936 31603 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:31.764396 31678 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:31.764670 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:31.721463 31696 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:31.705009 31683 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 13:12:31.763022 31570 consensus_meta.cc:287] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.922s	user 0.010s	sys 0.003s
W20260709 13:12:31.783023 31600 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.762s	user 0.012s	sys 0.000s
I20260709 13:12:31.789674 28081 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:31.785979 28756 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:31.795311 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:31.785209 28158 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:31.796723 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.797051 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.798513 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.799710 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.799767 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.800258 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.801148 28077 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:31.800381 28153 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:31.801833 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.802596 28077 raft_consensus.cc:2410] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:31.189877 28154 rpcz_store.cc:276] 0709 13:12:30.149138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.162150 (+ 13012us) service_pool.cc:224] Handling call
0709 13:12:30.162550 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:30.162823 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:30.277382 (+114559us) log.cc:844] Serialized 14 byte log entry
0709 13:12:31.172897 (+895515us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:31.172963 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:31.173021 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.173100 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:31.173246 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":191181,"prepare.queue_time_us":116523,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":39,"raft.queue_time_us":191780,"spinlock_wait_cycles":4333312,"thread_start_us":817597,"threads_started":2,"wal-append.queue_time_us":714598}
I20260709 13:12:31.813062 31672 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:31.818552 31530 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:31.819041 31412 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:31.819233 31530 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:31.820186 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.820304 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.820914 28080 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:31.822001 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.823231 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.824767 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.825399 31702 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:31.826490 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.827234 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.827622 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.827904 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.828457 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.829008 28158 raft_consensus.cc:2410] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:12:31.829185 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.829555 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.831589 31456 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:31.832100 31456 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:31.793952 31600 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:31.834383 31506 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 13:12:31.834831 31506 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.720609 31710 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:31.788678 31696 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:31.826507 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:31.713343 31711 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:31.841831 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51764us
I20260709 13:12:31.850426 28226 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:31.828676 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.843250 28077 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:31.753931 31602 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:31.856302 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.848711 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.849026 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.853889 31722 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:31.854079 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.847445 28152 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:31.860085 28082 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:31.862380 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.865091 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.867132 28153 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:31.868904 28156 raft_consensus.cc:2410] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:12:31.869699 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 58388.6us
I20260709 13:12:31.791064 31670 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:12:31.841450 31605 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:31.871824 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:31.835280 31600 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:31.878742 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:31.882992 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.795558 31570 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.881762 31608 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:12:31.820425 31530 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.884157 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:31.891433 28758 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:31.884721 28156 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:31.860226 31722 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:31.883066 31715 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
I20260709 13:12:31.881640 31678 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 13:12:31.883877 31668 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.910s	user 0.013s	sys 0.000s
I20260709 13:12:31.881583 31719 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:31.907498 31678 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:31.907846 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:31.891397 31605 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:31.909358 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:31.911126 31640 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:31.911684 31640 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:31.912935 31608 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:31.913437 31608 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:31.913895 31608 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:31.908034 31586 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:31.911556 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:31.919916 31678 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:31.921067 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:31.921986 28756 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:31.926118 31711 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:31.928004 30804 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:31.930263 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.930974 31634 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.931378 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:31.932354 28152 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:31.931007 28159 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 5.
I20260709 13:12:31.933928 31602 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:31.901504 31700 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:31.894814 31722 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:31.938210 31690 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:31.938726 31690 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:31.939742 31690 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:31.883296 31321 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:31.913715 31506 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:31.879588 31677 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:12:31.943260 31690 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:31.943604 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.944283 28156 raft_consensus.cc:2376] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8. Re-sending same reply.
I20260709 13:12:31.945297 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:31.918195 31668 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.948145 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.945221 31586 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:31.951728 31570 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:31.951814 31586 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:31.953270 31586 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:31.954793 31724 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:31.955647 28154 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:31.956451 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:31.959587 31210 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:31.960105 31210 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:31.961432 31678 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:31.941100 31321 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:31.964416 31321 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:31.966050 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:31.967546 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:31.968067 28762 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:12:31.968241 31370 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:31.969413 31370 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.969787 31370 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:31.975183 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 63674.7us
I20260709 13:12:31.976097 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:31.976372 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 632) took 1321 ms. Trace:
I20260709 13:12:31.976785 28229 rpcz_store.cc:276] 0709 13:12:30.655355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.673561 (+ 18206us) service_pool.cc:224] Handling call
0709 13:12:30.707247 (+ 33686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:30.923530 (+216283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.976108 (+1052578us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54001f5ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:31.976311 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493518720}
I20260709 13:12:31.977396 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:31.978637 31677 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:31.979084 31677 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:31.979568 31677 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:31.981137 31506 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:31.982064 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:31.982702 28085 raft_consensus.cc:2376] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7. Re-sending same reply.
W20260709 13:12:31.986198 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 115533us
I20260709 13:12:31.988427 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:31.989966 28127 leader_election.cc:400] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:31.994056 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:31.998162 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:32.000253 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.001052 28079 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:12:31.996486 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:32.003976 28761 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:32.006650 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:31.948498 31668 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:32.013482 28762 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:32.014384 28155 raft_consensus.cc:2410] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:12:32.014557 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:32.016616 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:32.017584 28078 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:31.992090 31665 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.002s
I20260709 13:12:32.028115 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:32.029804 28228 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:32.031282 28052 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:31.991252 31678 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:32.037914 31719 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:12:32.047394 31719 raft_consensus.cc:1068] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:12:32.050431 31713 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:32.006552 31456 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.051542 31506 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:32.051882 31263 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:12:32.052454 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:31.987448 31667 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:32.007308 31696 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:32.039670 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.053977 31242 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:32.054487 31242 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:32.054783 31696 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.055500 31242 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.057543 31702 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:32.058123 31640 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.059871 28231 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:32.060338 31710 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:32.052368 31263 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:32.063792 31696 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:32.064400 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:32.031875 31155 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:32.001709 31602 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:12:32.086302 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:32.043716 31711 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:32.006284 31594 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.087728 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:32.088927 28758 raft_consensus.cc:2368] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:32.045197 28228 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:12:32.060333 28231 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:32.037521 28127 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.600s (SENT)
I20260709 13:12:32.097438 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:32.026449 31724 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:32.064601 31242 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:32.070273 31640 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.038316 31665 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.097981 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:32.100369 28761 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:12:32.015869 31670 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:12:32.137104 28085 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:32.137982 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 100510us
W20260709 13:12:32.104120 31671 consensus_meta.cc:287] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.826s	user 0.012s	sys 0.002s
W20260709 13:12:32.102008 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.026145 31684 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:12:32.075579 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:32.125061 28084 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
W20260709 13:12:32.144990 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:32.141011 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.083555 31702 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:12:32.073134 31321 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
W20260709 13:12:32.146418 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.139781 31456 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.147861 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:32.148605 28152 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:32.107975 31724 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.150926 28158 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:32.104957 31155 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:32.154320 28152 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:32.155704 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:32.110904 31678 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:32.158723 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:32.160004 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:32.109762 31711 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:32.164045 28159 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:32.160686 28153 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:32.109515 31464 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:12:32.145565 31640 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:32.174072 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 98576.8us
I20260709 13:12:32.107723 31683 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:12:32.152495 31530 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.152638 31700 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:12:32.207705 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.162935 31321 consensus_queue.cc:1243] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:12:32.209127 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:32.209908 28081 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:32.157476 31724 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:32.211697 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.212368 28078 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:32.179644 31530 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:32.180310 31731 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:12:32.179641 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.181239 31646 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:32.221153 28160 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:32.221230 31646 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:32.222806 31646 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.181869 31602 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:32.182392 31682 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:32.224967 31602 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:32.225288 31682 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:32.226256 31602 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.226620 31682 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.228317 31602 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:32.186612 31594 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:32.241695 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 94325.3us
W20260709 13:12:32.188083 31708 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.919s	user 0.003s	sys 0.011s
I20260709 13:12:32.188979 31684 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:32.188771 31711 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:32.191280 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 194057us
I20260709 13:12:31.854549 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:32.192808 31631 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:32.162093 31263 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.201947 31640 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.156620 31634 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:32.204933 31665 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.205016 28077 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:32.210938 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.271309 28081 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:32.218827 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:32.080845 31210 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.224622 31646 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:32.228488 31682 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:32.271538 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.276086 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.277123 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.277618 28077 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:32.150686 31671 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.245476 28160 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:32.245991 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:32.243145 31702 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:12:32.209775 31321 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:32.249607 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.249538 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:32.245249 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:32.252445 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:32.209165 31700 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:32.198503 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2116) took 1151 ms. Trace:
I20260709 13:12:32.257854 28161 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:12:32.259833 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.261993 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:32.265409 28078 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:32.272226 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 642) took 1031 ms. Trace:
W20260709 13:12:32.289620 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 70848.3us
I20260709 13:12:32.281414 28081 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:32.274019 31263 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:32.281740 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.308804 28081 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:32.309074 28160 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:32.282346 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:32.258931 31631 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:32.290992 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:32.299680 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.287569 28728 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:32.290542 28152 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 5: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:32.270815 31678 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:32.313537 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:32.316536 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:32.314356 28198 leader_election.cc:400] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:32.317668 28198 leader_election.cc:403] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:12:32.307147 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54836.2us
I20260709 13:12:32.308884 28084 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:32.320437 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:32.246515 31693 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:12:32.310310 28226 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:32.283883 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:32.258621 31684 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.289923 28082 rpcz_store.cc:276] 0709 13:12:31.046815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.048081 (+  1266us) service_pool.cc:224] Handling call
0709 13:12:31.048473 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.860634 (+812161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.198137 (+337503us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54000c8fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:32.198406 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1855051392}
I20260709 13:12:32.314265 28234 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:32.306798 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.321026 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:32.325146 31597 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:32.291219 28227 rpcz_store.cc:276] 0709 13:12:31.240403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.240510 (+   107us) service_pool.cc:224] Handling call
0709 13:12:31.243279 (+  2769us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.267934 (+ 24655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.267965 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.936186 (+668221us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b2c0004c898. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:31.936660 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:12:32.126269 (+189609us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:32.158997 (+ 32728us) log.cc:844] Serialized 14 byte log entry
0709 13:12:32.264181 (+105184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.264304 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.264370 (+    66us) raft_consensus.cc:1685] finished
0709 13:12:32.272034 (+  7664us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.272167 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7759,"prepare.queue_time_us":30297,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":1786,"raft.queue_time_us":77521,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":190,"spinlock_wait_cycles":44292352,"thread_start_us":290599,"threads_started":2,"wal-append.queue_time_us":220377}
W20260709 13:12:32.310953 31666 consensus_meta.cc:287] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.945s	user 0.004s	sys 0.009s
I20260709 13:12:32.333665 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:32.293435 31210 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:32.286594 31747 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:32.341024 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:32.348872 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.350332 30804 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:32.357610 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:32.361711 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:32.359174 31640 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:32.365252 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49837.3us
I20260709 13:12:32.312157 31743 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:32.340029 31464 raft_consensus.cc:2764] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:12:32.346387 31685 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:32.371440 28077 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:32.278996 31155 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.373901 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.317938 31748 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:32.291818 31321 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.334275 31634 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:32.376309 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:32.430899 31634 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:32.431464 31634 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:32.431731 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64749us
I20260709 13:12:32.433923 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:32.378926 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:32.436059 28227 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:32.381510 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 631) took 1732 ms. Trace:
I20260709 13:12:32.409487 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.353881 31666 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:32.437124 28159 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:32.364650 31700 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:32.437994 31700 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:32.438897 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.329787 31678 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.345594 31506 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:12:32.368049 31668 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:32.423115 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:32.384346 31747 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:12:32.425851 31739 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:12:32.430356 31321 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.434954 31756 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:32.462918 31756 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:32.463663 31756 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:32.382119 31370 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.467811 31370 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:32.377220 28079 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:32.468557 28086 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 11. Ops: []
W20260709 13:12:32.411031 31603 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.620s	user 0.012s	sys 0.000s
I20260709 13:12:32.473568 31603 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.476081 31603 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.441529 28161 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:12:32.476994 28758 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:12:32.477603 31530 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.379981 31685 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.478612 31685 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:32.373775 31662 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:12:32.436936 28230 rpcz_store.cc:276] 0709 13:12:30.648958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.649117 (+   159us) service_pool.cc:224] Handling call
0709 13:12:30.649534 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.000522 (+350988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.005727 (+  5205us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.015047 (+  9320us) log.cc:844] Serialized 26 byte log entry
0709 13:12:31.175515 (+160468us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:31.175553 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.175623 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.187422 (+1011799us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:32.374741 (+187319us) raft_consensus.cc:1685] finished
0709 13:12:32.374761 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.381447 (+  6686us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26333,"raft.queue_time_us":331400,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":15029,"spinlock_wait_cycles":40241408,"thread_start_us":434214,"threads_started":2,"wal-append.queue_time_us":238754}
I20260709 13:12:32.447763 28159 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:32.444802 31668 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:32.449345 31753 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:32.451396 31684 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:32.420372 31605 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.512s	user 0.006s	sys 0.007s
W20260709 13:12:32.437256 31745 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:12:32.493433 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:32.456010 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:32.456616 31666 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:32.506377 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 645) took 1234 ms. Trace:
I20260709 13:12:32.506814 28233 rpcz_store.cc:276] 0709 13:12:31.271854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.282062 (+ 10208us) service_pool.cc:224] Handling call
0709 13:12:31.282577 (+   515us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:31.283134 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.283174 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:31.934459 (+651285us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b2c0004c898. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:31.985405 (+ 50946us) log.cc:844] Serialized 52 byte log entry
0709 13:12:32.134493 (+149088us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:32.134536 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.134605 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.506031 (+371426us) raft_consensus.cc:1685] finished
0709 13:12:32.506053 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.506313 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":978,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"spinlock_wait_cycles":8969984,"thread_start_us":90117,"threads_started":1,"wal-append.queue_time_us":160062}
W20260709 13:12:32.440006 31608 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.526s	user 0.013s	sys 0.000s
I20260709 13:12:32.509027 31608 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.511487 31608 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.459023 28228 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:32.425173 31738 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:12:32.512923 31753 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:32.513626 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2145) took 1067 ms. Trace:
W20260709 13:12:32.514169 31677 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.000s
I20260709 13:12:32.513985 31753 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.514855 31677 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.514030 28232 rpcz_store.cc:276] 0709 13:12:31.446276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.792474 (+346198us) service_pool.cc:224] Handling call
0709 13:12:31.846996 (+ 54522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:31.859659 (+ 12663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:31.859709 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:31.974997 (+115288us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b1800412000. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a
0709 13:12:31.975461 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:12:32.128995 (+153534us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:32.129047 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.129145 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.513302 (+384157us) raft_consensus.cc:1685] finished
0709 13:12:32.513323 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.513564 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12062,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":6839,"spinlock_wait_cycles":3446912,"thread_start_us":153095,"threads_started":1,"wal-append.queue_time_us":187474}
I20260709 13:12:32.479730 28078 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:32.518637 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:32.486771 31668 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:32.521834 31678 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.522799 28228 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:32.526104 28234 raft_consensus.cc:2410] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:12:32.526371 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1750) took 1263 ms. Trace:
I20260709 13:12:32.481334 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:32.486662 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:32.489925 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:32.491067 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:32.491200 28152 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:32.528680 28228 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:32.499943 28159 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:32.501083 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:32.503212 31605 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:32.503825 31742 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
W20260709 13:12:32.539465 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46124us
W20260709 13:12:32.540321 28728 leader_election.cc:336] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.538s (SENT)
I20260709 13:12:32.540820 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:32.541000 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.541618 28159 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:32.541330 31724 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.542078 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:32.543751 31724 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:32.544471 31722 consensus_meta.cc:287] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.606s	user 0.007s	sys 0.003s
I20260709 13:12:32.544916 31722 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.546283 31749 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:32.546679 31749 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:32.547143 31722 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.547923 28157 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:32.549448 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.550177 28161 raft_consensus.cc:2376] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 7. Re-sending same reply.
I20260709 13:12:32.551141 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:32.551549 31747 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:32.551813 28161 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:32.504312 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:32.553005 28160 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:32.553364 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.554016 28161 raft_consensus.cc:2376] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 10. Re-sending same reply.
I20260709 13:12:32.554102 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.554214 31634 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.554952 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.556444 31634 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:32.378577 31631 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.477272 31671 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:32.548548 31668 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.505148 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:32.538045 31605 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:32.491564 31665 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:12:32.600586 28082 raft_consensus.cc:2368] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:32.599645 31605 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:32.602090 31605 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.603794 31685 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.604349 28233 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
W20260709 13:12:32.538014 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56797.8us
I20260709 13:12:32.593422 31445 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:32.564716 28085 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:32.593227 31580 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:32.604681 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:32.549727 31751 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:12:32.609848 28152 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
W20260709 13:12:32.595223 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:32.610430 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 116229us
I20260709 13:12:32.595743 28080 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:32.526707 28083 rpcz_store.cc:276] 0709 13:12:31.263003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.275103 (+ 12100us) service_pool.cc:224] Handling call
0709 13:12:31.275596 (+   493us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:31.301266 (+ 25670us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:31.337211 (+ 35945us) log.cc:844] Serialized 14 byte log entry
0709 13:12:31.704143 (+366932us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:31.714981 (+ 10838us) log.cc:844] Serialized 52 byte log entry
0709 13:12:31.715114 (+   133us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:31.732071 (+ 16957us) log.cc:844] Serialized 14 byte log entry
0709 13:12:31.765385 (+ 33314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:31.765476 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.061365 (+295889us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b1800289180. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:12:32.169613 (+108248us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b1800289180. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1
0709 13:12:32.523642 (+354029us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b1800289180. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:12:32.526156 (+  2514us) raft_consensus.cc:1685] finished
0709 13:12:32.526179 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.526315 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74661,"prepare.queue_time_us":84400,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":76,"raft.queue_time_us":81292,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":10897,"spinlock_wait_cycles":53346688,"thread_start_us":331590,"threads_started":2,"wal-append.queue_time_us":378393}
I20260709 13:12:32.502882 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:32.587714 28159 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:32.611802 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:32.611871 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 52430.4us
I20260709 13:12:32.610975 28226 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:32.618677 28083 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 6: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:32.619632 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.620488 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.621477 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.622555 28161 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:32.731283 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 117142us
I20260709 13:12:32.608352 31662 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:32.607509 31570 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:32.606740 31580 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:32.704932 31690 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.761s	user 0.012s	sys 0.000s
I20260709 13:12:32.732287 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.528337 31700 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.749150 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:32.749888 28226 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:32.750002 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:32.750677 28228 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:32.750963 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:32.751574 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:32.752072 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:32.752195 28228 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:32.753155 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:32.753535 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:32.754411 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:32.754748 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:32.759663 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.761828 31580 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.762812 31747 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.763408 31747 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.763942 31668 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:32.704582 31666 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:32.767120 31666 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.769510 31666 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:32.772437 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:32.773718 28083 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:32.761749 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.775456 28154 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:32.776530 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:32.777500 28228 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:32.777331 28159 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:32.779021 28158 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:32.781656 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:32.745291 28231 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.610059 31600 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.145s
I20260709 13:12:32.783952 28231 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:32.739871 31684 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.790094 28226 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:32.704594 31602 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.153s
I20260709 13:12:32.813370 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:32.817667 28226 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:32.799365 28228 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:12:32.832580 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 234087us
W20260709 13:12:32.806432 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.827459 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:32.834659 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.834975 28083 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:32.547513 31749 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:32.833750 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:32.778960 31677 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:32.844905 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:32.847656 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.779554 31662 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:32.845638 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:32.852957 28083 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:32.854239 31716 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:32.854921 31752 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.428s	user 0.012s	sys 0.000s
I20260709 13:12:32.852301 31662 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.854871 31716 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:32.858784 28083 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:32.861106 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:32.862424 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:32.863117 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:32.864398 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:32.865513 28083 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:32.867820 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:32.868716 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.865514 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:32.871784 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:32.877027 28153 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:32.877344 31716 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.878156 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:32.786123 31570 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:32.920962 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:32.920572 28159 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:32.932443 28158 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:32.858659 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:32.941695 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 83110us
I20260709 13:12:32.882915 31600 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:32.832258 31684 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:32.936081 31530 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.982589 28231 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:33.026742 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2153) took 1209 ms. Trace:
W20260709 13:12:33.030041 31761 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:12:32.989063 31685 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:33.033778 31766 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:12:33.034843 31600 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:33.037272 31684 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:33.032701 31530 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.058772 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:33.041822 31756 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.578s	user 0.012s	sys 0.000s
I20260709 13:12:32.988299 31210 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:33.043435 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2143) took 1707 ms. Trace:
I20260709 13:12:33.045111 28231 rpcz_store.cc:276] 0709 13:12:31.817751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.059430 (+241679us) service_pool.cc:224] Handling call
0709 13:12:32.059743 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:32.623826 (+564083us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54000b6ba0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f506f 00007f59ce66df60 00007f59ce66f7de 00007f59ce679b49 00007f59ce67a416 00007f59ce676d8f 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62
0709 13:12:33.026453 (+402627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:33.026680 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257608448}
I20260709 13:12:33.071105 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:33.053063 28160 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:33.067555 28154 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
W20260709 13:12:33.061281 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.041183 31747 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:33.048291 28226 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:33.048919 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2146) took 1244 ms. Trace:
I20260709 13:12:33.076244 31570 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:33.070489 28235 rpcz_store.cc:276] 0709 13:12:31.335744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.367876 (+ 32132us) service_pool.cc:224] Handling call
0709 13:12:31.817667 (+449791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:31.817895 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:31.853746 (+ 35851us) log.cc:844] Serialized 14 byte log entry
0709 13:12:32.285399 (+431653us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:32.285453 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:32.285496 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.285559 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:33.043247 (+757688us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54003566d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:33.043359 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33401,"prepare.queue_time_us":29339,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":17228,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":18758,"spinlock_wait_cycles":962867712,"thread_start_us":452387,"threads_started":2,"wal-append.queue_time_us":498705}
I20260709 13:12:33.082036 28080 rpcz_store.cc:276] 0709 13:12:31.801721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:31.823004 (+ 21283us) service_pool.cc:224] Handling call
0709 13:12:33.046263 (+1223259us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b5400207e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:33.046378 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1707971840}
I20260709 13:12:33.084682 28154 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:33.099952 28154 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:33.104800 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:33.087118 31210 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:33.111245 28160 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:33.106737 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:33.124171 28226 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:33.130970 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:33.073451 31756 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.132637 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.099376 31603 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:33.136610 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 115316us
W20260709 13:12:33.137071 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 205094us
I20260709 13:12:33.144716 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:33.145285 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:33.107234 31735 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:33.146761 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:33.140789 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.150146 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:33.163183 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.158648 28232 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:33.159221 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:33.156358 28229 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:33.168467 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:33.169003 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:33.176137 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:33.166832 31242 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:33.198217 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:33.199085 28756 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:33.203502 28226 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:33.151270 31757 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:12:33.146358 31603 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.132243 31600 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.214119 28728 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:33.221230 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:33.223035 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59952.9us
I20260709 13:12:33.226438 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.231760 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:33.202127 31776 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:33.209443 31747 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:33.201200 31735 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:33.234794 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:33.237358 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 75713.6us
I20260709 13:12:33.276639 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:33.229127 31760 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:33.281915 31760 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.282636 31760 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:12:33.058185 31683 consensus_queue.cc:1175] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "11f10c12c03e47dcac88b050d4c1431b" responder_term: 5 status { last_received { term: 4 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:33.206180 31665 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:12:33.281515 31608 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:33.216004 31210 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.286319 31608 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.236282 31242 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:33.239310 31634 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:33.237658 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.296023 31789 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:33.220652 31662 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
W20260709 13:12:33.290200 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55479.5us
I20260709 13:12:33.301218 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:33.291435 31774 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:12:33.299350 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 61805.4us
I20260709 13:12:33.304539 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.308349 28756 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:12:33.314391 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:33.315127 28082 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:33.315447 31757 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:33.317178 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:33.318131 31719 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:33.318554 31719 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:33.318792 31756 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:33.319026 31719 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:33.319725 31770 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:12:33.288548 31608 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:33.339124 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.340536 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:33.343196 28124 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:33.347956 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:33.348686 28080 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:33.350618 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:33.303735 31678 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:33.361510 28082 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:33.342480 31787 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:12:33.347201 31665 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.364663 31665 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.305083 31786 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:33.337208 31789 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:33.346776 31608 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:33.343793 31605 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:33.386832 31789 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.377723 31603 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:33.379565 31678 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:33.391096 31678 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:33.384456 31777 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:12:33.391610 31665 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:33.365705 31756 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:33.385427 31760 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:33.398051 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47546.8us
I20260709 13:12:33.399160 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.406057 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:33.407322 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:33.411182 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.406200 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.427362 28758 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:33.428078 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:33.429234 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:33.391590 31678 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:33.430115 28763 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:33.434751 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:33.400465 31756 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:33.470958 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:33.486150 31794 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:33.487399 31794 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:33.488571 28229 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:33.429679 31753 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:12:33.438966 31750 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.020s	user 0.003s	sys 0.000s
I20260709 13:12:33.456395 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:33.502513 28082 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:33.461741 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:33.493278 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 151246us
I20260709 13:12:33.435369 31700 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:33.453235 31530 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:33.457252 28077 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:33.487756 31794 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:33.479631 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:33.506637 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:33.502812 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 68110us
I20260709 13:12:33.469486 31749 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:33.426964 31786 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:33.490207 31756 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.509258 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:33.434446 31662 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:12:33.441435 31768 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:12:33.441612 31580 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:12:33.508039 31551 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.463980 31668 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:12:33.506223 31693 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:33.526671 31693 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:33.526261 31551 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.527155 31693 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:33.530720 31760 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:33.533313 31753 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:33.534045 31753 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:33.535156 31753 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.516429 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:33.537096 31530 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:33.506590 31700 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:33.516027 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.515175 31756 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:33.513731 31713 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:33.525309 31792 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.537142 31803 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:33.544190 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:33.547186 31797 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.545526 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:33.537567 31530 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.540658 31700 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:33.557138 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:33.537446 31685 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:12:33.541172 31713 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:33.568171 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:33.542625 31735 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:33.575274 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:33.580379 28153 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:33.544932 31794 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.583042 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:33.583657 28160 raft_consensus.cc:2376] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10. Re-sending same reply.
I20260709 13:12:33.584807 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:33.586254 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:33.586942 28160 raft_consensus.cc:2376] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6. Re-sending same reply.
I20260709 13:12:33.587953 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:33.580507 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:33.559731 31692 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:33.560932 31600 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:33.560141 31570 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:33.605051 31570 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:33.605302 31806 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:33.606045 31570 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.558506 31530 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:33.596110 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:33.609181 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:33.609802 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:33.610522 28153 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:33.610435 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:33.611300 28160 raft_consensus.cc:2410] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:12:33.611534 28153 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:33.575294 31608 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:33.613250 31608 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:33.613734 31608 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:33.614435 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46390.9us
I20260709 13:12:33.615231 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:33.561935 31551 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:33.553319 31792 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.564926 31797 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.580119 31786 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:33.601290 31713 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.595359 31787 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:12:33.604264 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:33.621719 31713 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:33.629755 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:33.642259 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:33.642501 28235 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:33.643765 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:33.625376 31600 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:33.645951 31600 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.647199 31600 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:33.648653 31747 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.649228 31747 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.651513 31792 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:33.652107 31735 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:33.652516 31735 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:33.653488 31797 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:33.653407 31735 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.653911 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:33.661409 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 104353us
I20260709 13:12:33.595130 31692 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:33.673200 31692 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:33.664775 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:33.670449 31608 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.671468 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:33.672086 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:33.663880 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 184359us
I20260709 13:12:33.682327 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:33.628669 31786 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:33.628985 31794 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:33.689880 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:33.614475 31719 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.703131 31809 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:33.707167 28153 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:12:33.704175 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 119963us
I20260709 13:12:33.718431 28761 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:33.718995 28761 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:33.724216 28080 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:33.727140 28156 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:33.730314 28761 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.730984 31631 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:33.731720 28761 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:33.732259 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.733606 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:33.735914 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:33.738176 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 84314us
I20260709 13:12:33.709319 31786 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:33.709976 31807 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:33.717020 31810 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:12:33.742084 31807 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.742504 31807 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:33.739449 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:33.711112 31768 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:12:33.745419 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.756690 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:33.746030 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:33.692359 31693 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.748411 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:33.759713 28226 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:33.760097 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:33.761679 28235 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:12:33.762019 28235 pending_rounds.cc:85] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 8
I20260709 13:12:33.762281 28235 pending_rounds.cc:107] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:12:33.762501 28235 raft_consensus.cc:2889] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:33.749027 28198 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.753258 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:33.755582 28080 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:33.760530 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.766606 31696 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:33.708707 31678 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.766402 31816 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:33.770246 31817 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:33.770025 31816 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.781405 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:33.740556 31672 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:33.797605 28756 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:12:33.798914 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 61097.2us
I20260709 13:12:33.763887 31412 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:33.756101 31768 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:33.739544 31603 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:12:33.813547 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2159) took 1665 ms. Trace:
I20260709 13:12:33.816498 28226 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:33.817755 31811 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:33.817881 31793 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:33.814054 28157 rpcz_store.cc:276] 0709 13:12:32.148310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.148716 (+   406us) service_pool.cc:224] Handling call
0709 13:12:32.149105 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:32.548349 (+399244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:33.813266 (+1264917us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54001978d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:33.813480 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":916727552}
I20260709 13:12:33.772038 31757 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 13:12:33.822311 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65726.7us
I20260709 13:12:33.829701 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.802948 31788 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:33.814054 31768 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:33.838101 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 155898us
I20260709 13:12:33.839916 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:33.798444 31756 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.801971 31530 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.855435 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.820962 31803 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:12:33.825131 31700 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.849678 31788 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:33.877235 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:33.881876 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 52273.3us
I20260709 13:12:33.824877 31810 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:12:33.884549 31810 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:33.884866 31815 raft_consensus.cc:2764] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:12:33.885057 31810 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:33.872999 31412 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:33.887943 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.889735 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:33.885829 31815 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:33.880718 31608 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:33.882812 31719 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:33.907287 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 147585us
I20260709 13:12:33.904960 31818 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:33.903658 31684 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:33.919071 31700 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:33.922684 31530 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:33.923746 31692 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.914731 31608 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.924626 31788 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.909607 31820 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:33.896968 31805 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 13:12:33.916911 31818 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:33.929867 31608 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:33.955307 31608 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:33.957890 31773 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.022s	user 0.003s	sys 0.000s
W20260709 13:12:33.949328 28085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 708) took 1693 ms (client timeout 1611 ms). Trace:
I20260709 13:12:33.925817 31719 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:33.951697 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 62063.5us
I20260709 13:12:33.927948 31811 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:33.927418 31816 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:33.959643 28085 rpcz_store.cc:269] 0709 13:12:32.248749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.259590 (+ 10841us) service_pool.cc:224] Handling call
0709 13:12:33.942113 (+1682523us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54002daad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:33.942269 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582641408}
I20260709 13:12:33.936455 31678 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:33.935734 31693 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:33.965389 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:33.942440 31828 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:33.965893 31812 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:12:33.948833 31700 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:33.951391 31594 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.960711 31818 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:33.980544 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 231592us
I20260709 13:12:33.960785 31809 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:33.985445 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:33.940565 31808 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.147s
I20260709 13:12:33.941450 31412 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.003319 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:34.003597 28199 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.659s (SENT)
I20260709 13:12:34.004127 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:34.004263 28157 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:34.004861 28158 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:34.004703 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:34.000991 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:33.974433 31719 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:33.986873 31756 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:34.015127 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:34.016836 28154 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:33.999805 31828 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:33.986760 31829 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:34.028714 31811 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:34.033298 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 162807us
I20260709 13:12:33.990077 31594 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.028510 31816 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.937156 31810 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.044070 31827 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:12:34.044077 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 260914us
I20260709 13:12:34.050828 28199 leader_election.cc:400] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:34.004851 31693 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:12:33.990207 31819 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:33.975407 31822 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.163s
I20260709 13:12:34.067407 28080 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:34.064548 28157 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:34.069638 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:34.070289 31799 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:12:34.070825 31825 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:34.071604 31825 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:34.072078 31825 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:34.072985 31825 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.075613 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:34.076510 28084 raft_consensus.cc:2410] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:33.999156 31815 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
W20260709 13:12:34.079317 28156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2156) took 1932 ms (client timeout 2411 ms). Trace:
I20260709 13:12:34.079401 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:34.080822 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:34.080869 28153 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:34.079952 28156 rpcz_store.cc:269] 0709 13:12:32.147339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.147542 (+   203us) service_pool.cc:224] Handling call
0709 13:12:33.514191 (+1366649us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b54002559a8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d4454924 00007f59d363d227 00007f59d363cab8 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:34.079079 (+564888us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54003935d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:34.079258 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486030976}
I20260709 13:12:34.037575 31820 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:34.084148 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:34.084156 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:34.085919 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:34.088673 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:34.090714 28083 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:34.100153 31719 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.104863 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:34.106988 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:34.107546 28761 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:34.110093 28158 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:34.112203 31833 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:34.079459 31603 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.111379 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:34.117882 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 242347us
I20260709 13:12:34.118580 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.075531 31824 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.086843 31757 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.053231 31816 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.119753 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:34.123315 28158 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:34.125070 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:34.095705 31829 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:34.093444 31753 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:34.119769 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.097355 31810 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:34.129556 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:34.138523 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:34.121264 31749 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.607s	user 0.013s	sys 0.001s
I20260709 13:12:34.142302 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.121654 31692 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:34.128172 31719 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:34.127977 31815 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:34.156112 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.132500 31829 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:34.157286 28154 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:34.138080 31608 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:34.184451 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:34.159857 31613 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.194020 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:34.194700 28761 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
W20260709 13:12:34.197369 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72475.9us
I20260709 13:12:34.149987 31819 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:12:34.204880 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 165263us
I20260709 13:12:34.126681 31603 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.211887 28154 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:34.210335 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:34.216411 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:34.184105 31530 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.097016 31820 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.186964 31684 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:34.193174 31841 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:34.130090 31757 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.227195 31845 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:34.228080 31845 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:34.231258 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.232439 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.143486 31824 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.224985 31753 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:34.219182 31678 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.212707 31815 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:34.239589 31678 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.240754 31815 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.242085 31820 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:34.245340 31771 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.311s	user 0.009s	sys 0.000s
I20260709 13:12:34.247962 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:34.248450 31757 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.235242 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.208444 31613 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.289450 31846 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.293994 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1817) took 1476 ms. Trace:
W20260709 13:12:34.287814 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 52643.3us
I20260709 13:12:34.289186 31594 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.294466 28234 rpcz_store.cc:276] 0709 13:12:32.818094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.874036 (+ 55942us) service_pool.cc:224] Handling call
0709 13:12:32.874352 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:32.874707 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:32.874737 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:32.875225 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:12:32.935768 (+ 60543us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:32.935806 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.936381 (+   575us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.244662 (+1308281us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:34.248804 (+  4142us) raft_consensus.cc:1685] finished
0709 13:12:34.248825 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.293928 (+ 45103us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42730,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":39993,"spinlock_wait_cycles":4776832,"thread_start_us":59950,"threads_started":1,"wal-append.queue_time_us":58906}
I20260709 13:12:34.296501 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:34.297636 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:34.249045 31794 consensus_meta.cc:287] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.002s
I20260709 13:12:34.237042 31826 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.238128 31719 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:34.305675 28084 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:34.306092 31827 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:34.308760 31827 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.309487 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2153) took 1815 ms. Trace:
I20260709 13:12:34.313654 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:34.226370 31749 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:12:34.222415 31530 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.317512 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:34.318436 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:34.318966 28077 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:34.320803 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:34.311288 28081 rpcz_store.cc:276] 0709 13:12:32.493913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.589206 (+ 95293us) service_pool.cc:224] Handling call
0709 13:12:32.589608 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:32.617921 (+ 28313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:32.617949 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:32.618380 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:12:34.228046 (+1609666us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b18003336a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:12:34.230798 (+  2752us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:34.245546 (+ 14748us) log.cc:844] Serialized 14 byte log entry
0709 13:12:34.248019 (+  2473us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:34.309034 (+ 61015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.309151 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.309213 (+    62us) raft_consensus.cc:1685] finished
0709 13:12:34.309234 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.309408 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43596,"prepare.queue_time_us":14798,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":89054,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":115108,"spinlock_wait_cycles":8295808,"thread_start_us":18515,"threads_started":2,"wal-append.queue_time_us":1525146}
I20260709 13:12:34.290462 31753 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:34.218672 31807 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.304476 31696 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:12:34.293006 31684 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:34.329063 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:34.313588 31841 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:34.345075 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:34.346408 31613 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:34.312963 31631 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:34.357715 28078 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
W20260709 13:12:34.377759 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 67378.4us
I20260709 13:12:34.287344 31829 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.379601 31829 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:34.292383 31846 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.364171 31841 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:34.391779 28085 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:34.401288 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 80578.4us
I20260709 13:12:34.402581 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:34.390991 31815 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:34.362967 31631 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.390090 31696 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:34.402136 31768 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.554s	user 0.005s	sys 0.007s
I20260709 13:12:34.397459 31820 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:34.398386 31816 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:34.411144 31815 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.405514 31684 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:34.325135 31826 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:34.423015 31839 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:12:34.423244 31853 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:34.415580 31749 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:34.426718 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.421650 31696 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.428241 31678 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:34.445966 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.436952 31684 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.447292 31684 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:34.444325 31749 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.447345 31855 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.459290 31851 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:12:34.403622 31753 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.467952 28124 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:34.432152 31768 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.472362 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:34.471378 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:34.478207 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:34.479126 28077 raft_consensus.cc:2376] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 6. Re-sending same reply.
I20260709 13:12:34.476794 28763 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:34.482100 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:34.481948 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:34.466334 31696 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.471084 31849 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:34.489661 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:34.490692 28762 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:12:34.494328 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 177950us
I20260709 13:12:34.480569 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:34.497211 28077 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:34.499985 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:34.500599 28125 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:34.505164 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:34.508167 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:34.502683 31799 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:12:34.509050 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:34.502272 31412 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:34.509915 28229 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:34.507577 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:34.509929 31816 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:34.510742 28234 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:34.510990 31816 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.511593 31816 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:34.504280 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:34.512802 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:34.512571 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:34.513659 28235 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:34.514254 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:34.514513 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:34.495626 31803 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:34.487262 31631 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:34.519774 31852 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:12:34.514724 31719 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:34.525481 31719 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.527668 31719 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.466886 31841 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.498081 31263 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:34.514910 28228 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:34.503815 31849 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:34.505396 31846 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.538267 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:34.540508 28234 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:34.544586 31826 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:34.536608 31829 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.544109 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:34.503862 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:34.545542 31829 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.529094 31859 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:34.532996 31825 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:34.470927 31855 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.537745 31803 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:34.537525 31263 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.550756 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:34.561998 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:34.549206 31716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.666s	user 0.003s	sys 0.014s
W20260709 13:12:34.571607 31716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.691s	user 0.008s	sys 0.014s
I20260709 13:12:34.567396 31263 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:34.587313 28761 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:34.587885 28761 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:34.559427 31859 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.593349 28158 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:12:34.594270 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:34.594915 28160 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:34.594415 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:34.595902 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:34.597147 28082 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:34.595029 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:34.563264 31825 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.580448 31815 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.605964 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:34.598621 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:34.618741 28158 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:34.602632 28761 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.619684 28156 raft_consensus.cc:2376] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 9. Re-sending same reply.
I20260709 13:12:34.607163 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:34.643922 28761 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:34.591485 31700 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.599s	user 0.013s	sys 0.000s
I20260709 13:12:34.653647 28234 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:34.652854 31816 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:34.654598 31860 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:12:34.659086 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:34.660122 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.658147 31816 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:34.656297 28157 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 4. Current term is 6. Ops: []
I20260709 13:12:34.669744 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:34.618640 31826 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:34.665653 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:34.670850 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.671307 31696 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:12:34.673484 31811 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:34.673861 31594 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:34.678812 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.681994 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:34.656659 31700 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:34.590673 31749 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.683513 28235 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:34.683859 31678 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:34.685134 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:34.600962 31855 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:34.683676 31866 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:12:34.677460 31768 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:34.686003 31816 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:34.694833 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:34.693120 28235 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:12:34.701223 28235 pending_rounds.cc:85] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 4
I20260709 13:12:34.701539 28235 pending_rounds.cc:107] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:12:34.701803 28235 raft_consensus.cc:2889] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:34.678483 31792 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:34.683094 31594 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.708029 28235 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:34.690644 31811 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:34.678428 31826 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.696265 31846 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:34.703751 31792 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.706735 31594 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:34.726943 31870 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:34.617120 31803 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.735193 28081 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:34.588462 31849 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.737279 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:34.726413 31792 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:34.728024 31746 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:34.725620 31829 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:34.745496 28077 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:34.745772 28761 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:34.741163 28081 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:34.763490 31858 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:34.765939 31799 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:34.780555 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:34.781250 28158 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:34.739089 31825 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:34.740651 31768 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:34.783401 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:34.792132 31811 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.758688 31816 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.801317 31816 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.801925 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 657) took 2332 ms. Trace:
I20260709 13:12:34.788767 31758 raft_consensus.cc:3055] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:34.803364 31758 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:34.802369 28230 rpcz_store.cc:276] 0709 13:12:32.469067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.531716 (+ 62649us) service_pool.cc:224] Handling call
0709 13:12:32.532026 (+   310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:32.593558 (+ 61532us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:32.601764 (+  8206us) log.cc:844] Serialized 14 byte log entry
0709 13:12:34.519778 (+1918014us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b18002f72e0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a 00007f59ce681a2a
0709 13:12:34.569299 (+ 49521us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:34.581852 (+ 12553us) log.cc:844] Serialized 52 byte log entry
0709 13:12:34.582046 (+   194us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:34.682389 (+100343us) log.cc:844] Serialized 14 byte log entry
0709 13:12:34.682633 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:34.682721 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:34.801693 (+118972us) raft_consensus.cc:1685] finished
0709 13:12:34.801716 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:34.801865 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40217,"prepare.queue_time_us":146512,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":1698,"raft.queue_time_us":45314,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":20596,"spinlock_wait_cycles":16880384,"thread_start_us":146336,"threads_started":2,"wal-append.queue_time_us":1984105}
I20260709 13:12:34.803944 31816 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.759034 31827 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:12:34.823447 28158 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:34.807365 31859 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.781829 31613 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 13:12:34.836925 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.813805 31412 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.812536 31768 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:34.845418 28226 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:12:34.848944 31684 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.851290 31684 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:34.853350 31826 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:34.855451 31810 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.712s	user 0.009s	sys 0.003s
W20260709 13:12:34.844038 31818 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.846s	user 0.010s	sys 0.004s
I20260709 13:12:34.856657 31818 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.859007 31818 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:34.861668 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:34.865070 31464 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:12:34.866350 28160 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:34.867252 31867 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 13:12:34.871857 28233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 655) took 2376 ms (client timeout 2135 ms). Trace:
I20260709 13:12:34.872447 28081 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
W20260709 13:12:34.872319 28233 rpcz_store.cc:269] 0709 13:12:32.468312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.508564 (+ 40252us) service_pool.cc:224] Handling call
0709 13:12:34.844492 (+2335928us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b54003b3654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:34.844692 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104780544}
I20260709 13:12:34.873055 28081 pending_rounds.cc:85] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 6
I20260709 13:12:34.873334 28081 pending_rounds.cc:107] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:12:34.873603 28081 raft_consensus.cc:2889] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:34.888685 31464 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:12:34.897598 28158 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:12:34.902495 28161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 632) took 2356 ms (client timeout 2603 ms). Trace:
I20260709 13:12:34.905759 31850 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:34.902953 28161 rpcz_store.cc:269] 0709 13:12:32.540618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.554731 (+ 14113us) service_pool.cc:224] Handling call
0709 13:12:34.897129 (+2342398us) spinlock_profiling.cc:243] Waited 66.8 ms on lock 0x7b540033e854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:34.897311 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153659776}
I20260709 13:12:34.908635 28161 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 4. Current term is 5. Ops: []
I20260709 13:12:34.870085 31412 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.931097 31412 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:34.937456 31880 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:12:34.942385 31263 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.893610 31811 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:34.906579 31850 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.957271 31747 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:34.963254 30804 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:12:34.963475 31841 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:12:34.967080 28077 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:34.967783 31792 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:34.970709 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:34.980754 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:34.952476 31783 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:34.986794 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:34.987568 28230 raft_consensus.cc:2376] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 8. Re-sending same reply.
I20260709 13:12:34.997282 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:34.981838 31753 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:34.999655 31753 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.002504 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.003139 28761 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:34.949067 31768 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:34.946215 31594 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.002034 31586 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:35.024552 31586 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:35.025458 31586 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.026875 31586 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:34.998399 31825 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.031050 31825 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.032610 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:35.033277 31825 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.033654 28235 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:34.963382 31850 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:35.006111 31753 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.012385 31792 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:34.997867 31783 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:34.984968 31747 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.982069 31851 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:12:35.022688 31594 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:35.038800 28235 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:35.039674 31747 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:35.049994 28152 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2172) took 2701 ms (client timeout 1906 ms). Trace:
W20260709 13:12:35.050524 28152 rpcz_store.cc:269] 0709 13:12:32.348500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.348624 (+   124us) service_pool.cc:224] Handling call
0709 13:12:35.049772 (+2701148us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b5400254c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:35.049917 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314119424}
I20260709 13:12:35.054162 28155 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:35.055583 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 628) took 2516 ms. Trace:
I20260709 13:12:35.055948 28155 rpcz_store.cc:276] 0709 13:12:32.539196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.555074 (+ 15878us) service_pool.cc:224] Handling call
0709 13:12:35.053958 (+2498884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:35.054671 (+   713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.055349 (+   678us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b5400254c50. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:35.055490 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5745708800}
I20260709 13:12:35.061203 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:35.061539 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.062450 28762 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:34.960026 31811 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.071965 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.083199 31827 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
W20260709 13:12:35.087324 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.088966 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.089479 31263 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:35.089862 28227 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1822) took 2020 ms (client timeout 2086 ms). Trace:
I20260709 13:12:35.035208 31783 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:35.090255 28227 rpcz_store.cc:269] 0709 13:12:33.069654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.071854 (+  2200us) service_pool.cc:224] Handling call
0709 13:12:35.088701 (+2016847us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b58001c0598. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552eee 0000000000375939 00007f59d35c61e0 00007f59d368f0c4 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:35.089783 (+  1082us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4421376}
I20260709 13:12:35.091363 31783 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:35.097725 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2158) took 1022 ms. Trace:
I20260709 13:12:35.098238 28084 rpcz_store.cc:276] 0709 13:12:34.077571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.077681 (+   110us) service_pool.cc:224] Handling call
0709 13:12:34.078014 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.307372 (+229358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.097472 (+790100us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54001ae0d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:35.097662 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522706304}
W20260709 13:12:35.103690 31875 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.381s	user 0.001s	sys 0.000s
I20260709 13:12:35.104372 31825 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:35.104976 31825 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.067571 31753 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:35.107160 31825 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:35.110661 31796 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.554s	user 0.006s	sys 0.001s
W20260709 13:12:35.117591 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 409768us
I20260709 13:12:35.119076 31735 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:35.138705 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.131861 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 723) took 1381 ms. Trace:
I20260709 13:12:35.142931 28078 rpcz_store.cc:276] 0709 13:12:33.750285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.751159 (+   874us) service_pool.cc:224] Handling call
0709 13:12:33.767757 (+ 16598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.358424 (+590667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.131551 (+773127us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b540027c9d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:35.131792 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1356125056}
I20260709 13:12:35.144248 31788 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:35.087358 31747 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:35.117580 31825 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.145051 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 726) took 1106 ms. Trace:
I20260709 13:12:35.107174 31753 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:35.155350 31753 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:35.144698 31788 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:35.137274 31530 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:35.147972 28085 rpcz_store.cc:276] 0709 13:12:34.038496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.058062 (+ 19566us) service_pool.cc:224] Handling call
0709 13:12:34.058405 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.399355 (+340950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.144612 (+745257us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54000c80d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:35.144969 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766114688}
I20260709 13:12:35.147537 31634 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:35.152366 31825 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.167270 31811 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:35.167856 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.130057 31758 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.173916 31758 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.184444 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:35.177294 28152 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:12:35.199426 28154 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:12:35.162757 31605 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.203456 31815 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.204042 31815 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.206494 31815 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.208105 31605 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.210988 31605 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:35.212985 31760 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 13:12:35.174839 31850 consensus_queue.cc:1175] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "611ec64ec58244fe82058d6130e4e6f3" responder_term: 6 status { last_received { term: 5 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:35.213056 31825 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.214430 31851 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.174867 31662 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:12:35.177225 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:35.199450 28158 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 7: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:35.203291 31811 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:35.176860 28157 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:35.223289 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:35.226444 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:35.229930 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.235457 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.218158 31851 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.237354 31803 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:35.239427 31803 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:35.239801 31803 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:35.228359 31833 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:35.234126 31580 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:35.227824 31760 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:12:35.169278 31810 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:12:35.236585 31827 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:35.245595 31886 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:12:35.222637 31788 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.267630 31788 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:35.273921 31851 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:35.248776 31833 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.286612 31833 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:35.287844 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.288693 28084 raft_consensus.cc:2368] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:35.289100 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:35.293865 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.294503 28763 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:35.278584 31807 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:35.287286 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.300655 28085 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
W20260709 13:12:35.284192 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.299317 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.303366 31682 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:35.303828 31682 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:35.304597 31682 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:35.305382 31862 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.771s	user 0.004s	sys 0.000s
I20260709 13:12:35.307430 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:35.308434 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.309177 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:35.309610 28227 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:35.310047 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.310627 28233 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:35.303391 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:35.311919 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:35.312472 28231 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.313742 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:35.314316 28085 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:35.316078 28081 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:35.316197 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.316883 28233 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:35.245415 31803 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.325548 28231 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:35.326315 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.315146 31787 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:35.331131 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.331157 31787 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.327607 28081 raft_consensus.cc:2410] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:12:35.329632 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:35.333818 28084 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:35.301687 31851 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.307005 31677 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.330435 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:35.337167 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.337702 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.349375 28085 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:35.339282 31787 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:35.338047 31851 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.350904 28231 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:35.352185 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.354043 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.357134 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:35.359009 28233 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:35.360453 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.354949 28229 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:35.342644 31722 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:35.376170 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.377430 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.382296 28084 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:35.352829 31747 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.364967 31851 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:35.372740 28125 leader_election.cc:400] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:35.392449 28125 leader_election.cc:403] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:12:35.374099 31842 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.067s	user 0.002s	sys 0.000s
I20260709 13:12:35.394670 31608 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:12:35.388971 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 728) took 1277 ms. Trace:
I20260709 13:12:35.399533 28080 rpcz_store.cc:276] 0709 13:12:34.111458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.195205 (+ 83747us) service_pool.cc:224] Handling call
0709 13:12:34.219217 (+ 24012us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:34.224060 (+  4843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:34.224093 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:34.224625 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:12:34.537136 (+312511us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:34.537173 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.388491 (+851318us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b54002ad854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:35.388549 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:35.388603 (+    54us) raft_consensus.cc:1685] finished
0709 13:12:35.388617 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:35.388901 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9398,"raft.queue_time_us":8725,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":51438,"spinlock_wait_cycles":10248960,"thread_start_us":311505,"threads_started":1,"wal-append.queue_time_us":326055}
I20260709 13:12:35.340660 31677 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.371577 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:35.409462 28231 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:35.415865 31793 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:35.419466 31793 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.420007 31793 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:35.402943 31580 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:35.438992 28227 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:35.403103 31753 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.403801 31747 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.452106 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 649) took 3096 ms. Trace:
I20260709 13:12:35.452589 28232 rpcz_store.cc:276] 0709 13:12:32.355293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:32.516421 (+161128us) service_pool.cc:224] Handling call
0709 13:12:32.516763 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:33.164122 (+647359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.451846 (+2287724us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54000951d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:35.452043 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1475596160}
I20260709 13:12:35.445900 31793 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.461944 31580 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
W20260709 13:12:35.464622 31889 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.465s	user 0.001s	sys 0.000s
I20260709 13:12:35.468044 31753 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.464653 31580 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:35.469007 31580 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:35.470314 31677 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.472529 31580 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:35.474090 31898 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:12:35.474268 31608 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7.
I20260709 13:12:35.475736 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:35.482564 28078 raft_consensus.cc:2410] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:12:35.473935 31580 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:35.485918 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2181) took 1482 ms. Trace:
I20260709 13:12:35.486248 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 721) took 1767 ms. Trace:
I20260709 13:12:35.486347 28153 rpcz_store.cc:276] 0709 13:12:34.003431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.004120 (+   689us) service_pool.cc:224] Handling call
0709 13:12:34.080668 (+ 76548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.082158 (+  1490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.485707 (+1403549us) spinlock_profiling.cc:243] Waited 76.2 ms on lock 0x7b54003935d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:35.485852 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175363456}
I20260709 13:12:35.486821 28082 rpcz_store.cc:276] 0709 13:12:33.718876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.719002 (+   126us) service_pool.cc:224] Handling call
0709 13:12:33.738991 (+ 19989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.597618 (+858627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.485993 (+888375us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b540017c854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:35.486183 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1973101312}
I20260709 13:12:35.491909 31608 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:35.506529 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.529790 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:35.534965 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.500491 31608 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.535935 28082 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 8: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:35.536885 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.537556 31608 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.538162 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.538969 28080 raft_consensus.cc:2368] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
W20260709 13:12:35.544107 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.540933 31845 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:35.548775 28080 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:35.550047 28080 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:35.553416 28761 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:35.559213 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.559320 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.559849 28081 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:35.557127 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:35.562523 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.563872 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.564637 28078 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:35.565565 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:35.566989 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:35.570236 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:35.574221 28228 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:35.575932 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:35.577395 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.578960 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:35.581107 28078 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:35.581648 28078 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:35.583623 28078 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.584654 28078 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:35.582731 28127 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:35.591156 31793 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.592437 28080 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.593974 28080 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:35.595393 31833 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.595960 31833 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.577885 31677 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:35.597272 31677 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:35.597874 31677 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:35.598155 31833 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.600135 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.607425 28078 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:12:35.607941 28078 pending_rounds.cc:85] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 4
I20260709 13:12:35.608220 28078 pending_rounds.cc:107] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:12:35.608525 28078 raft_consensus.cc:2889] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:35.609838 31853 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:35.610455 31853 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:35.611375 31853 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.618254 28228 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:35.574847 31845 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:35.576989 31506 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:35.577009 31608 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:35.602182 31807 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:35.616195 31846 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:35.631047 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:35.635030 28080 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:35.635264 28761 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:35.627271 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.621542 31600 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:35.612865 31870 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:12:35.636852 31608 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:35.626148 31820 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.627173 31605 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:12:35.639621 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.640295 28082 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
W20260709 13:12:35.641113 31882 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.639s	user 0.000s	sys 0.002s
I20260709 13:12:35.634747 31807 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.645141 31807 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:35.646377 31682 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:35.648195 28198 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:35.649510 31846 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:35.650530 31846 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:35.651353 31836 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.451s	user 0.004s	sys 0.001s
I20260709 13:12:35.653321 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:35.653702 31837 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.439s	user 0.002s	sys 0.000s
I20260709 13:12:35.653996 28235 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:35.655009 31608 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.658484 31820 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.659600 31809 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:35.660864 31820 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.661731 31833 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.662240 31833 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:35.663555 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.664228 31668 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:35.664342 31833 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:35.664577 31803 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:35.664712 31668 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:35.627413 31845 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:35.666877 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.671064 28235 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:35.669288 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:35.675074 28199 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.643s (SENT) [suppressed 58 similar messages]
I20260709 13:12:35.689599 28234 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:35.679776 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:35.705839 28083 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:12:35.706208 28083 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:35.675254 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.686491 28230 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:12:35.686806 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.709059 28234 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:35.709681 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:35.687574 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.666069 28233 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:35.693929 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.712723 28231 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:35.692234 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.711934 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:35.638110 31600 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:35.721948 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:35.660200 31809 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:35.705134 31815 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.705539 31753 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:35.691592 31845 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:35.726945 28083 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.735802 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:35.743831 28079 raft_consensus.cc:2368] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:35.705551 31155 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:35.748953 31671 raft_consensus.cc:2764] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:12:35.749073 31155 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:35.750361 31671 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:35.751719 31155 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:35.751855 31671 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:35.752393 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.759256 28234 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:35.761916 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:35.746582 31809 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:35.758610 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:35.719587 31833 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:35.776647 28154 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:12:35.740362 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.768308 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.777667 28082 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:35.773366 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:35.778715 28155 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:35.775461 28152 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:35.777204 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.765560 28227 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:12:35.718924 31672 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:35.786057 28156 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:35.724660 31370 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:35.800330 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.791610 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:35.792393 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:35.788954 28082 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:12:35.802210 28082 pending_rounds.cc:85] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 7
I20260709 13:12:35.802582 28082 pending_rounds.cc:107] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:12:35.802874 28082 raft_consensus.cc:2889] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:35.803063 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:35.802271 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:35.803710 28229 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:35.806310 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.742434 31753 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.817322 31753 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:35.760861 31815 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.823511 31815 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.827267 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:35.752689 31671 consensus_queue.cc:1243] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:12:35.830473 31600 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.831957 31600 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:35.758236 31603 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:35.833331 31833 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:35.834527 31833 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.841164 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.841787 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.842542 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.842825 28755 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:35.843648 31808 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:35.843818 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:35.844260 31808 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.844923 31808 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:35.845144 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 669) took 1851 ms. Trace:
I20260709 13:12:35.845553 28762 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:35.845912 31824 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:35.846016 31445 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:35.845597 28226 rpcz_store.cc:276] 0709 13:12:33.993375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.993522 (+   147us) service_pool.cc:224] Handling call
0709 13:12:34.844869 (+851347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.846037 (+  1168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.844957 (+998920us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54003b3650. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:35.845086 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1956900096}
I20260709 13:12:35.848956 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.850340 31668 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.851763 31668 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:35.853701 31850 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.854462 31850 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.858367 31809 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.859025 31809 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:35.860055 28226 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:35.866189 31807 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:35.871037 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 78729.1us
I20260709 13:12:35.872104 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:35.844165 28125 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:35.805353 31672 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:35.977254 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.901703 31603 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:35.947892 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:35.909089 28226 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:12:35.904269 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.942862 31445 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:12:35.928782 28155 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 5. Current term is 6. Ops: []
I20260709 13:12:35.864136 30804 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 3, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:35.918301 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:36.071982 28229 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:35.928067 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:35.941720 31793 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:35.920390 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:35.922930 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:35.903594 28079 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:35.958115 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:36.082437 28755 raft_consensus.cc:2410] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 8 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:12:35.868532 31807 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:35.895853 31834 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:35.927863 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.089068 28156 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:36.089691 31793 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:36.090152 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:36.090809 28156 raft_consensus.cc:2376] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6. Re-sending same reply.
I20260709 13:12:36.091740 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.092073 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:36.092998 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 172698us
I20260709 13:12:35.975585 27981 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 5 to 7, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 7 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
W20260709 13:12:35.919179 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.095041 28156 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:35.989503 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 648) took 1337 ms. Trace:
W20260709 13:12:35.990129 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 315081us
I20260709 13:12:35.967460 28154 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:35.936393 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:35.901955 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:36.104575 28762 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:35.929021 28761 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:35.912495 31370 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:35.974570 28227 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:35.987531 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:36.009874 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 165774us
I20260709 13:12:35.869856 31729 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 13:12:35.900990 31850 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:35.902611 31696 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:12:36.069898 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:36.085646 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 137861us
I20260709 13:12:36.093978 28152 raft_consensus.cc:2410] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:36.106863 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:36.107909 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:36.057255 31605 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:36.001567 31817 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:36.097827 28160 rpcz_store.cc:276] 0709 13:12:34.652389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.868385 (+215996us) service_pool.cc:224] Handling call
0709 13:12:34.868717 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:34.869630 (+   913us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:12:34.885468 (+ 15838us) log.cc:844] Serialized 14 byte log entry
0709 13:12:35.662770 (+777302us) spinlock_profiling.cc:243] Waited 980 us on lock 0x7b18004559e0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a 00007f59ce681a2a
0709 13:12:35.989131 (+326361us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:35.989203 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:35.989248 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.989317 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:35.989445 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15596,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":6825,"raft.queue_time_us":90,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":41535,"spinlock_wait_cycles":2255104,"thread_start_us":326046,"threads_started":1}
I20260709 13:12:36.110720 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:36.089066 31746 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:36.116518 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:36.116858 28156 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:36.121407 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 133956us
I20260709 13:12:36.151882 28080 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:36.040956 31815 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:36.056267 31603 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:36.132944 28155 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:36.111768 31631 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:36.133260 28084 raft_consensus.cc:2410] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:12:36.139014 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.098898 31758 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:36.136092 28152 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:36.113068 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:36.111332 31580 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.000s
I20260709 13:12:36.145646 28086 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 6.
I20260709 13:12:36.147904 28079 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:36.090180 31793 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:36.152465 28124 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:36.155807 28160 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: [6.9-6.9]
W20260709 13:12:36.165351 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 52664.9us
I20260709 13:12:36.171850 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:36.172125 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:36.172901 28081 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:36.176311 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:36.178011 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.178236 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:36.178857 28160 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:36.138175 31846 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:36.175283 27981 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 5 to 6, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 6 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:36.161093 31603 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:36.122062 31370 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.121173 31713 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:36.160166 31631 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.214248 28156 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:36.164400 31758 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.211781 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:36.219261 28152 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:36.145156 31850 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:12:36.148527 31665 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:12:36.175992 31809 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:36.175635 31801 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:36.162348 31746 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.222543 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.238795 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:36.225302 27981 catalog_manager.cc:5697] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 6 to 7, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 7 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:36.165904 31815 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.154763 31817 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:36.233935 27982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 190) took 18487 ms (client timeout 15000 ms). Trace:
I20260709 13:12:36.180627 31608 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:36.240830 27981 catalog_manager.cc:5697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 2 to 7, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 7 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:36.180825 31822 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:12:36.220571 31631 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:36.247763 31746 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:36.237928 31809 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:36.222467 31713 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.258942 31370 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:36.262426 31682 consensus_meta.cc:287] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.616s	user 0.012s	sys 0.001s
I20260709 13:12:36.265080 31667 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:36.273440 28152 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
W20260709 13:12:36.274338 31907 log.cc:927] Time spent T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:12:36.274983 31793 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.180488 31722 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:12:36.277455 28156 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:36.332978 28156 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:36.177287 31753 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.226953 31758 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:36.306507 31677 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.708s	user 0.014s	sys 0.000s
I20260709 13:12:36.290953 27981 catalog_manager.cc:5697] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 4 to 5, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 5 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:36.348981 27981 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 3 to 4, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:36.302158 28160 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:36.298681 31746 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.313681 28086 raft_consensus.cc:2410] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 8 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:12:36.353365 31746 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.353852 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.355237 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.355703 31746 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:36.356956 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.306923 31667 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:36.302565 31809 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.351962 27981 catalog_manager.cc:5697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 3 to 4, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 4 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:36.334551 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 409391us
I20260709 13:12:36.341885 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.337219 31608 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:36.384007 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.384889 28156 raft_consensus.cc:2393] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:36.280197 28158 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:36.331521 31797 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:36.385162 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.385820 28160 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:36.385988 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.386637 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.387290 28160 raft_consensus.cc:2368] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:36.390561 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.392225 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:36.397810 28156 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:36.388949 27981 catalog_manager.cc:5697] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
W20260709 13:12:36.403115 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.378619 31793 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:36.370349 31672 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:36.359865 31845 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.571s	user 0.003s	sys 0.008s
W20260709 13:12:36.400245 31808 consensus_meta.cc:287] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.555s	user 0.004s	sys 0.007s
I20260709 13:12:36.361994 31677 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:36.422782 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:36.423250 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.423095 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:36.428719 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 300097us
I20260709 13:12:36.403590 31746 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:36.278111 27982 rpcz_store.cc:269] 0709 13:12:17.843369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:17.869773 (+ 26404us) service_pool.cc:224] Handling call
0709 13:12:36.165288 (+18295515us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:27.381371 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:27.416271 (+ 34900us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:27.416352 (+    81us) write_op.cc:435] Acquired schema lock
0709 13:12:27.416398 (+    46us) tablet.cc:662] Decoding operations
0709 13:12:27.418692 (+  2294us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:27.418804 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:27.418841 (+    37us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:12:27.469656 (+ 50815us) tablet.cc:701] Row locks acquired
0709 13:12:27.486267 (+ 16611us) write_op.cc:260] PREPARE: finished
0709 13:12:27.552943 (+ 66676us) write_op.cc:270] Start()
0709 13:12:27.553120 (+   177us) write_op.cc:276] Timestamp: P: 1783602747525908 usec, L: 0
0709 13:12:27.553172 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 13:12:27.625089 (+ 71917us) log.cc:844] Serialized 8580 byte log entry
0709 13:12:28.551651 (+926562us) op_driver.cc:464] REPLICATION: finished
0709 13:12:28.924244 (+372593us) spinlock_profiling.cc:243] Waited 7.06 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce672e28 00007f59d20ad581 00007f59d20acd58 00007f59d20b19f1 00007f59d20b19a9 00007f59d20b1939 00007f59d20b1901 00007f59d20b0b2d 000000000038f354 000000000038f199
0709 13:12:29.098549 (+174305us) write_op.cc:301] APPLY: starting
0709 13:12:29.098648 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:29.099136 (+   488us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:29.099179 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:29.569690 (+470511us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:29.570386 (+   696us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 13:12:29.570487 (+   101us) write_op.cc:312] APPLY: finished
0709 13:12:29.571228 (+   741us) log.cc:844] Serialized 152 byte log entry
0709 13:12:30.141265 (+570037us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59cb56d0ac 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b
0709 13:12:30.294150 (+152885us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:30.294413 (+   263us) write_op.cc:454] Released schema lock
0709 13:12:30.294524 (+   111us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:30.295060 (+   536us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:30.295091 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1748625,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":421681,"apply.run_cpu_time_us":71799,"apply.run_wall_time_us":1196491,"num_ops":22,"prepare.queue_time_us":132465,"prepare.run_cpu_time_us":32819,"prepare.run_wall_time_us":1543137,"raft.queue_time_us":257711,"raft.run_cpu_time_us":32476,"raft.run_wall_time_us":456469,"replication_time_us":978238,"spinlock_wait_cycles":31080576,"thread_start_us":1304762,"threads_started":5,"wal-append.queue_time_us":1230760,"wal-append.run_cpu_time_us":16206,"wal-append.run_wall_time_us":2434965}]]}
I20260709 13:12:36.434163 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:36.434006 27981 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:36.437122 28199 leader_election.cc:400] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:36.404191 31580 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:12:36.414112 31822 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:36.415798 31807 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.439556 28199 leader_election.cc:403] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:36.416049 31870 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:12:36.436141 31908 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:12:36.396413 31608 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.400931 31797 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.444818 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:36.427563 31722 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.428349 31672 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.441390 28161 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:36.449329 31911 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:12:36.417126 31808 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.417935 31845 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.461766 27981 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 7 to 8, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 8 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:36.441083 31747 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:36.454919 28050 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:36.466598 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:36.470537 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:36.435973 31746 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.509492 27981 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 5 to 8, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 8 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:36.463375 31801 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:36.428275 31667 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.525440 28155 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:36.533942 31608 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:36.534279 31922 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:36.721307 31608 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:36.720468 31603 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.721923 31608 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.535571 31808 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:36.536454 31677 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:36.724216 31603 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:36.537992 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.540007 31833 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:36.542145 31915 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 13:12:36.543871 31912 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:12:36.545946 31845 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:36.558636 31913 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:12:36.548596 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.453689 31822 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.468281 31722 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.456490 31807 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:36.584719 31803 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.879s	user 0.005s	sys 0.005s
I20260709 13:12:36.620230 28234 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
W20260709 13:12:36.637046 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.455785 31870 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.523447 31713 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:36.459874 31682 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.291s
W20260709 13:12:36.644718 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 178224us
I20260709 13:12:36.708842 31824 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:36.709982 31667 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.746161 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:36.725247 31677 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:36.617453 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 162592us
I20260709 13:12:36.751477 27981 catalog_manager.cc:5697] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:36.560118 31817 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:36.469099 31815 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:36.645973 31801 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.768131 28155 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:36.643780 31788 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:36.645916 31747 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:36.775658 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:36.724534 31608 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:36.733203 31809 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:36.735416 31746 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:36.824769 31631 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.001s
I20260709 13:12:36.825412 31631 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.791196 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:36.826192 27981 catalog_manager.cc:5697] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 4 to 7, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 7 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:36.827476 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:36.777256 31824 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:36.828609 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:36.829136 31824 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.829676 31822 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:36.830164 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:36.750272 31807 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:36.804845 28159 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:36.777616 31600 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.908s	user 0.005s	sys 0.005s
I20260709 13:12:36.842252 31870 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:36.842952 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:36.794620 31713 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.843638 28234 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:36.843861 31713 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.845937 31713 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:36.846220 31925 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:36.846823 31925 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:36.847625 31605 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.705s	user 0.006s	sys 0.006s
I20260709 13:12:36.848062 31605 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:36.848868 31797 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:36.849041 28160 raft_consensus.cc:2410] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:36.803944 31817 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.850713 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:36.851367 28235 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:36.798625 31677 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.854254 31677 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:36.828401 27981 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 6 to 7, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 7 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:36.859094 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:36.813897 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:36.818243 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.818778 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 658) took 1049 ms. Trace:
I20260709 13:12:36.864173 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 659) took 1093 ms. Trace:
I20260709 13:12:36.820014 31608 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.822093 31722 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:36.815681 31918 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:36.863204 28153 rpcz_store.cc:276] 0709 13:12:35.769407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.769559 (+   152us) service_pool.cc:224] Handling call
0709 13:12:35.769922 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.770388 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.770423 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:35.801963 (+ 31540us) log.cc:844] Serialized 26 byte log entry
0709 13:12:36.234599 (+432636us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:36.234639 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.234708 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:36.818534 (+583826us) raft_consensus.cc:1685] finished
0709 13:12:36.818556 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.818706 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32665,"raft.queue_time_us":514,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":21359,"thread_start_us":376772,"threads_started":1,"wal-append.queue_time_us":521427}
I20260709 13:12:36.862183 28763 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 5.
I20260709 13:12:36.881639 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:36.859447 28757 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:36.864655 28157 rpcz_store.cc:276] 0709 13:12:35.771123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.771571 (+   448us) service_pool.cc:224] Handling call
0709 13:12:35.771895 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.772252 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.772282 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:35.788401 (+ 16119us) log.cc:844] Serialized 26 byte log entry
0709 13:12:35.868146 (+ 79745us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b480024c258. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:12:36.161789 (+293643us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:36.464272 (+302483us) log.cc:844] Serialized 14 byte log entry
0709 13:12:36.862124 (+397852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.863792 (+  1668us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:36.863850 (+    58us) raft_consensus.cc:1685] finished
0709 13:12:36.863941 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:36.864110 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":238153,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":256798,"raft.run_cpu_time_us":2746,"raft.run_wall_time_us":91252,"spinlock_wait_cycles":26122496,"thread_start_us":998272,"threads_started":3,"wal-append.queue_time_us":533863}
I20260709 13:12:36.866144 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:36.907927 28078 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:36.869503 28050 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:36.808816 31801 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:36.828289 31631 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:36.922616 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.923892 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:36.468420 31672 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:36.937107 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:36.942739 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.915805 31677 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.915299 31603 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:36.940486 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:36.958631 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:36.935487 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.870369 31846 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:36.960047 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:36.914716 31794 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.960413 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:36.915189 31713 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:36.964532 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:36.956954 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 87542.9us
I20260709 13:12:36.931147 31716 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:36.935878 31665 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:36.960183 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:36.961218 28199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.024s	sys 0.048s
I20260709 13:12:36.930477 31918 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:36.921566 31920 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:36.921509 31608 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.870332 31817 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.961205 28077 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:36.915402 31682 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:36.949740 31805 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:12:36.968845 28199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.025s	sys 0.048s
I20260709 13:12:36.969720 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:36.961374 28078 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:36.951028 31826 raft_consensus.cc:1027] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: attempt to promote peer 70996712c3c942468a6006602de6477f: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:12:36.964015 28762 raft_consensus.cc:2368] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:12:36.960050 31677 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:36.921288 31808 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.977188 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:36.977079 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:36.967525 31668 consensus_meta.cc:287] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 1.115s	user 0.016s	sys 0.000s
I20260709 13:12:36.971334 31665 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:36.978386 31925 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:36.984469 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:37.014001 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 576951us
I20260709 13:12:37.030890 31665 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:37.037406 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:37.038834 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:37.014642 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 45063.6us
I20260709 13:12:36.972635 31788 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.042153 31788 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:37.045897 31758 consensus_meta.cc:287] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.683s	user 0.006s	sys 0.005s
W20260709 13:12:37.046736 31917 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.627s	user 0.004s	sys 0.013s
I20260709 13:12:37.046604 31758 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.048234 28761 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:37.036489 28161 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:37.048849 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2200) took 1240 ms. Trace:
I20260709 13:12:37.027194 31672 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.050379 31672 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:37.049590 28228 rpcz_store.cc:276] 0709 13:12:35.808370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.921098 (+112728us) service_pool.cc:224] Handling call
0709 13:12:35.921564 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.922098 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.922137 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:35.922706 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:12:36.309735 (+387029us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:36.309807 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:36.309905 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:37.045939 (+736034us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:37.048486 (+  2547us) raft_consensus.cc:1685] finished
0709 13:12:37.048507 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.048750 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":13449,"spinlock_wait_cycles":2540928,"thread_start_us":309014,"threads_started":1,"wal-append.queue_time_us":463304}
I20260709 13:12:37.050915 31672 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:37.051909 31672 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.053073 31925 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:37.053606 31927 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:37.044463 31788 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.054903 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:37.061683 28083 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:36.980399 31920 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.085506 31925 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:37.085363 31824 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:37.085968 31758 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:37.086061 31925 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:37.087244 31925 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.087745 31824 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.033362 31808 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:36.967844 31794 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.013836 31805 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.067592 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 629) took 1227 ms. Trace:
W20260709 13:12:37.069288 31906 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.904s	user 0.000s	sys 0.017s
W20260709 13:12:37.072024 31841 raft_consensus.cc:1068] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
  Committed config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }.
  Pending config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.016206 31930 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.089795 31815 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:37.089247 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:37.100596 28235 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
W20260709 13:12:37.097667 30804 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.016s	sys 0.024s
I20260709 13:12:37.110448 28763 rpcz_store.cc:276] 0709 13:12:35.840350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.842705 (+  2355us) service_pool.cc:224] Handling call
0709 13:12:36.392148 (+549443us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b5c00382a0c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d1fe1161 00007f59d2182a79 00007f59d363c9b6 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:37.067334 (+675186us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400654354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:37.067511 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149901440}
I20260709 13:12:37.088511 31925 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:37.016306 31716 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
W20260709 13:12:37.112074 31793 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.002s
I20260709 13:12:37.092072 28077 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:37.092389 31668 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:37.114455 31754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.019s	sys 0.019s
W20260709 13:12:37.123437 28124 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.021s	sys 0.067s
I20260709 13:12:37.126080 28083 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:12:37.112200 31805 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:37.087247 31920 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:37.141566 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 52388.3us
I20260709 13:12:37.145999 31747 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:37.088970 31665 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:37.142808 30804 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.017s	sys 0.024s
I20260709 13:12:37.090119 31746 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
W20260709 13:12:37.147953 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 203576us
I20260709 13:12:37.103768 31927 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 13:12:37.151237 28124 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.021s	sys 0.067s
I20260709 13:12:37.157472 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2180) took 1468 ms. Trace:
I20260709 13:12:37.159731 28199 leader_election.cc:400] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:37.088542 31824 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:37.181723 28083 rpcz_store.cc:276] 0709 13:12:35.689527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.689819 (+   292us) service_pool.cc:224] Handling call
0709 13:12:35.690094 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:37.061060 (+1370966us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5400387d20. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5eca1d 00007f59ce5d31a4 00007f59ce655447 00007f59ce66dfe7 00007f59ce66f7de 00007f59ce679b49 00007f59ce67a416 00007f59ce676d8f 00007f59ce67683e 00007f59d363b419
0709 13:12:37.156611 (+ 95551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.157386 (+   775us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250163584}
I20260709 13:12:37.183832 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 682) took 1537 ms. Trace:
W20260709 13:12:37.160496 31754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.999s	user 0.020s	sys 0.020s
I20260709 13:12:37.185019 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:37.184401 28232 rpcz_store.cc:276] 0709 13:12:35.646280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.648819 (+  2539us) service_pool.cc:224] Handling call
0709 13:12:35.649140 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:35.649492 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:35.649519 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:35.650008 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:12:35.924381 (+274373us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:35.924422 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:35.924503 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:37.068559 (+1144056us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:37.183526 (+114967us) raft_consensus.cc:1685] finished
0709 13:12:37.183552 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.183762 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4738,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":84,"spinlock_wait_cycles":24865664,"thread_start_us":273995,"threads_started":1,"wal-append.queue_time_us":413658}
I20260709 13:12:37.187877 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:37.158213 31155 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.186142 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:37.152336 31667 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.192073 28083 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:37.177479 31747 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:37.203308 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:37.235877 28226 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:37.206032 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.239378 28083 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:37.241464 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:37.242760 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:37.244565 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.245443 28233 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:37.246585 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.247363 28233 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:37.257093 28230 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:12:37.258869 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:37.262559 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 448749us
I20260709 13:12:37.260504 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.248575 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.270381 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:37.271459 28157 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:37.272117 28154 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:37.270411 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:37.273955 28156 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:37.287156 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:37.288931 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:37.290056 28235 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:12:37.186838 31929 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:37.187566 31753 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.152441 31668 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.321798 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:37.237442 31850 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:37.194977 31833 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:37.190733 31824 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:37.261195 31747 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:37.239179 31600 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.235348 31786 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:37.372200 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:37.292199 31608 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.193652 31155 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.255030 31667 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.355979 31753 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.363685 31833 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:37.373144 31921 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:37.371555 31682 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:37.375507 31809 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:37.375893 31817 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.138161 31930 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.393241 31677 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.030287 31918 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.396554 31833 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:37.348526 31929 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.401412 31808 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:37.409401 31905 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.404s	user 0.004s	sys 0.000s
W20260709 13:12:37.410055 31931 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:12:37.411435 31954 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.435063 31929 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.425588 31954 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:37.436983 31924 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.615s	user 0.004s	sys 0.000s
I20260709 13:12:37.438328 31954 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:37.440096 31909 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.146s	user 0.004s	sys 0.016s
W20260709 13:12:37.440120 31919 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.597s	user 0.008s	sys 0.000s
I20260709 13:12:37.402297 31921 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.443120 31921 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.430430 31930 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.355834 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:37.482676 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 753) took 1385 ms. Trace:
W20260709 13:12:37.482465 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 441743us
I20260709 13:12:37.489707 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:37.488574 28049 leader_election.cc:400] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:12:37.494629 28049 leader_election.cc:403] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:12:37.485616 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2198) took 1144 ms. Trace:
I20260709 13:12:37.498202 28161 rpcz_store.cc:276] 0709 13:12:36.340607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.841992 (+501385us) service_pool.cc:224] Handling call
0709 13:12:36.842454 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:37.049289 (+206835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.484826 (+435537us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54000807d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:37.485329 (+   503us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434370944}
I20260709 13:12:37.488075 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:37.494423 28158 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:12:37.492334 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 507785us
I20260709 13:12:37.486361 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2200) took 1096 ms. Trace:
I20260709 13:12:37.500972 28160 rpcz_store.cc:276] 0709 13:12:36.389776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.390350 (+   574us) service_pool.cc:224] Handling call
0709 13:12:37.485715 (+1095365us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54001969d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:37.486069 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051353472}
I20260709 13:12:37.501519 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:37.483666 28082 rpcz_store.cc:276] 0709 13:12:36.096823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.099277 (+  2454us) service_pool.cc:224] Handling call
0709 13:12:36.109779 (+ 10502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:36.126267 (+ 16488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:36.126302 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:37.055560 (+929258us) spinlock_profiling.cc:243] Waited 9.42 ms on lock 0x7b1800432700. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a
0709 13:12:37.056119 (+   559us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b2c000014c8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:37.056670 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:12:37.481602 (+424932us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:37.481729 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.481905 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:37.482023 (+   118us) raft_consensus.cc:1685] finished
0709 13:12:37.482068 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.482411 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7822,"raft.queue_time_us":16509,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":11418,"spinlock_wait_cycles":35380352,"thread_start_us":400760,"threads_started":1,"wal-append.queue_time_us":351368}
I20260709 13:12:37.482201 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:37.480489 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.504544 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:37.546111 31788 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:12:37.422675 31850 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:37.547070 31794 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.547585 31788 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:37.408030 31824 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.505853 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.507102 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.505110 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2204) took 1035 ms. Trace:
W20260709 13:12:37.508198 28730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.019s	sys 0.058s
W20260709 13:12:37.508944 28231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 2195) took 1751 ms (client timeout 2049 ms). Trace:
I20260709 13:12:37.374994 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.511763 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.552413 28234 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:37.553802 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.511775 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:37.556324 31921 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:37.557399 31921 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:37.551249 28231 rpcz_store.cc:269] 0709 13:12:35.757026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.765984 (+  8958us) service_pool.cc:224] Handling call
0709 13:12:37.508392 (+1742408us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54000e8b54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:37.508656 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1587611392}
I20260709 13:12:37.558317 31921 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:37.559800 28730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.019s	sys 0.058s
I20260709 13:12:37.560341 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.513979 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1812) took 1339 ms. Trace:
I20260709 13:12:37.517256 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2212) took 1304 ms. Trace:
I20260709 13:12:37.379594 31600 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.524993 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2197) took 1188 ms. Trace:
I20260709 13:12:37.526849 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2195) took 1190 ms. Trace:
I20260709 13:12:37.507170 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:37.528793 31729 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:37.531418 31716 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.562098 28152 rpcz_store.cc:276] 0709 13:12:36.212558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.307532 (+ 94974us) service_pool.cc:224] Handling call
0709 13:12:36.307924 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:36.308202 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:36.784715 (+476513us) log.cc:844] Serialized 14 byte log entry
0709 13:12:37.104861 (+320146us) log.cc:844] Serialized 14 byte log entry
0709 13:12:37.515896 (+411035us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b180033cd00. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:12:37.516362 (+   466us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:37.516484 (+   122us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:37.516590 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.516773 (+   183us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.517040 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":411416,"prepare.queue_time_us":124133,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":1829,"raft.queue_time_us":419681,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":1863,"spinlock_wait_cycles":2664704,"thread_start_us":763521,"threads_started":2,"wal-append.queue_time_us":536422}
I20260709 13:12:37.567485 31716 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.568907 31927 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:37.569056 31929 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:37.569947 31927 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.533254 31668 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:37.533775 31789 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:37.534602 31754 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.398126 31753 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.537308 31746 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:37.536314 31786 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:37.560421 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 635163us
I20260709 13:12:37.515712 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2216) took 1206 ms. Trace:
I20260709 13:12:37.562922 28159 rpcz_store.cc:276] 0709 13:12:36.336591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.836314 (+499723us) service_pool.cc:224] Handling call
0709 13:12:36.836625 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:36.836848 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:36.837153 (+   305us) log.cc:844] Serialized 14 byte log entry
0709 13:12:37.523899 (+686746us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:37.524042 (+   143us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:37.524189 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.524350 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.524717 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":380120,"prepare.queue_time_us":4072,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":382764,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":207,"thread_start_us":305815,"threads_started":1,"wal-append.queue_time_us":564180}
I20260709 13:12:37.571928 31729 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.548547 31788 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:37.561950 28086 rpcz_store.cc:276] 0709 13:12:36.174838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.176071 (+  1233us) service_pool.cc:224] Handling call
0709 13:12:37.513445 (+1337374us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400209c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:37.513736 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312947584}
I20260709 13:12:37.586436 28235 raft_consensus.cc:2368] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:12:37.580950 28158 rpcz_store.cc:276] 0709 13:12:36.309263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.395630 (+ 86367us) service_pool.cc:224] Handling call
0709 13:12:37.493318 (+1097688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:37.500446 (+  7128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.515238 (+ 14792us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001969d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:37.515589 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2523111808}
W20260709 13:12:37.598896 31853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.378s	user 0.014s	sys 0.017s
W20260709 13:12:37.596709 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 110260us
W20260709 13:12:37.594518 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 106516us
I20260709 13:12:37.551051 28227 rpcz_store.cc:276] 0709 13:12:36.469176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.470664 (+  1488us) service_pool.cc:224] Handling call
0709 13:12:36.471202 (+   538us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:36.915944 (+444742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:36.915996 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:36.918962 (+  2966us) log.cc:844] Serialized 52 byte log entry
0709 13:12:37.275674 (+356712us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:37.275755 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.275879 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:37.504337 (+228458us) raft_consensus.cc:1685] finished
0709 13:12:37.504387 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.504820 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":250182,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":800224,"threads_started":2,"wal-append.queue_time_us":488849}
I20260709 13:12:37.563081 28155 rpcz_store.cc:276] 0709 13:12:36.336011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.804282 (+468271us) service_pool.cc:224] Handling call
0709 13:12:36.841333 (+ 37051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:36.841658 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:36.842146 (+   488us) log.cc:844] Serialized 14 byte log entry
0709 13:12:37.525974 (+683828us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:37.526117 (+   143us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:37.526270 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.526382 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:37.526645 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":337619,"prepare.queue_time_us":1021,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":338491,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":165,"thread_start_us":345501,"threads_started":1,"wal-append.queue_time_us":555731}
W20260709 13:12:37.602502 31853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.384s	user 0.014s	sys 0.017s
I20260709 13:12:37.617017 28231 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
W20260709 13:12:37.623138 28757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 633) took 1739 ms (client timeout 1507 ms). Trace:
W20260709 13:12:37.623723 28757 rpcz_store.cc:269] 0709 13:12:35.841705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.843556 (+  1851us) service_pool.cc:224] Handling call
0709 13:12:37.550450 (+1706894us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54007e0254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:37.581319 (+ 30869us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2109323008}
I20260709 13:12:37.599728 31786 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.580415 31754 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.631918 31716 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.632284 28227 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:37.633178 31934 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:37.634111 31934 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.636935 31665 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.644361 31803 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:37.644984 31933 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:37.645869 31933 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.651211 31682 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.652633 31754 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.648396 31155 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.658259 28233 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:37.661679 31753 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.662725 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:37.659246 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.627072 31788 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.657702 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:37.666570 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 170006us
I20260709 13:12:37.640959 31729 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.670364 31786 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:37.670671 31921 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.671437 31826 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:12:37.697048 31826 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.697654 31826 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:37.645332 31803 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:37.703451 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45675.4us
I20260709 13:12:37.704901 28052 leader_election.cc:400] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:12:37.668236 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:37.707234 31747 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.643091 31927 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.649446 31933 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.676864 31600 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.656402 31716 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:37.720819 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 58427.1us
I20260709 13:12:37.681169 31930 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.677042 31789 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:37.685454 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 184178us
I20260709 13:12:37.686578 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:37.689518 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:37.688097 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:37.692003 31603 consensus_meta.cc:287] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.730s	user 0.008s	sys 0.007s
I20260709 13:12:37.693800 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:37.705207 31803 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.674220 31667 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:37.710567 31801 consensus_meta.cc:287] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.779s	user 0.015s	sys 0.012s
I20260709 13:12:37.717903 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:37.731377 31936 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:37.746881 31930 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.749210 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:37.733013 28759 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:37.733218 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:37.733464 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:37.735338 31957 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:12:37.735986 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 886) took 1331 ms. Trace:
W20260709 13:12:37.728998 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 70316.5us
W20260709 13:12:37.754889 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.755729 31933 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:37.757834 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:37.759377 31921 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:37.760389 31927 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:37.726709 31789 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:37.751724 28086 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:37.765344 31603 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:37.765568 31932 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:37.766671 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:37.753860 28761 rpcz_store.cc:276] 0709 13:12:36.403997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.404898 (+   901us) service_pool.cc:224] Handling call
0709 13:12:36.405220 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:37.048655 (+643435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.735120 (+686465us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b5400436554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:37.735658 (+   538us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1478290688}
I20260709 13:12:37.772323 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:37.781107 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:37.781333 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:37.782589 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44872us
I20260709 13:12:37.777307 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:37.787580 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:37.724329 31850 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.789124 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.789386 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:37.789773 31960 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.118s	user 0.005s	sys 0.000s
I20260709 13:12:37.790069 28763 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:37.791524 31722 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:37.791751 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:37.737345 31934 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.792552 28762 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:37.787645 28233 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:37.764144 31927 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:37.794509 31927 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:37.795982 28762 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:12:37.796953 28760 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:37.792550 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:37.756544 31933 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:37.797523 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:37.803241 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 98608.6us
I20260709 13:12:37.807207 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:37.814646 28757 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:12:37.817011 28081 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
W20260709 13:12:37.818030 28226 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.582s	user 0.005s	sys 0.013s
I20260709 13:12:37.819484 28759 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 6 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:12:37.820989 28226 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:37.814065 31933 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.823479 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:37.824323 28080 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
W20260709 13:12:37.825271 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 67736us
I20260709 13:12:37.821527 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:37.826402 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:37.827656 31921 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:37.828487 28199 leader_election.cc:400] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:37.755278 31716 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.829730 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:37.836443 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:37.842092 31934 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:37.842509 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:37.844630 28763 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:37.845888 28764 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:12:37.848701 28082 raft_consensus.cc:2410] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:12:37.849453 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47694.5us
I20260709 13:12:37.851938 31930 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:37.854770 31815 consensus_meta.cc:287] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.712s	user 0.016s	sys 0.000s
I20260709 13:12:37.855384 31815 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:37.856354 31918 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:37.856900 31918 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.850914 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:37.857308 31918 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:37.857385 28227 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:37.851819 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:37.865239 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:37.875499 31964 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:12:37.875753 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:37.875385 31789 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.879101 31754 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:37.879647 31754 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.880023 31789 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:37.877398 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:37.884057 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:37.832871 31853 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:37.887485 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:37.891801 28081 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:37.776015 31801 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.880041 31754 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:37.902717 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 66411.6us
I20260709 13:12:37.910665 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:37.913830 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 62159.1us
I20260709 13:12:37.895249 31747 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:37.891459 31682 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:37.921151 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:37.922705 31805 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.743s	user 0.016s	sys 0.001s
I20260709 13:12:37.923682 28079 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:37.925665 31716 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.927520 31551 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:37.928233 31551 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:37.933815 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.935292 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:37.936158 28235 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:37.936241 31853 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.937870 31925 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:37.938496 31925 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:37.940371 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:37.938613 31853 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.862859 31826 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.951902 28079 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:12:37.953310 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47787.8us
W20260709 13:12:37.921936 31965 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
W20260709 13:12:37.955574 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 127349us
W20260709 13:12:37.923254 31963 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.325s	user 0.000s	sys 0.002s
I20260709 13:12:37.923254 31805 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.959820 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:37.965540 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 163738us
I20260709 13:12:37.967162 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:37.974128 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:37.976138 28755 raft_consensus.cc:2368] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10.
I20260709 13:12:37.976136 31551 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.979277 31551 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:37.965148 31850 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:12:37.981405 31667 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:37.982203 31957 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.982378 31667 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.983069 31957 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.965471 31803 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.985050 31803 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:37.986035 31667 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:37.987075 31803 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:37.992411 31600 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:37.993043 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:37.993793 28079 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:37.972091 31853 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:37.995061 31853 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.995805 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:37.998107 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:37.998713 31927 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.956249 31789 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:37.951398 31925 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:37.979313 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:38.003777 28233 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:38.004251 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48122us
I20260709 13:12:38.000808 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:38.020860 31833 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.004s
I20260709 13:12:38.021478 31833 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:38.022927 28229 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:38.023947 31833 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:38.025714 31974 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:12:38.028093 31801 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:38.030207 31826 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:38.031034 31933 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:38.031589 31933 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:38.032006 31933 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:38.033749 31957 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:38.035385 31925 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.042462 31957 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:12:38.047713 31966 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.365s	user 0.000s	sys 0.004s
I20260709 13:12:38.049077 28230 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:38.058617 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 54938.8us
I20260709 13:12:38.074811 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:38.075727 28226 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:38.076575 31918 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:38.077646 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 110783us
I20260709 13:12:38.079126 31918 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:38.079583 28198 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:38.079980 31671 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.080437 28158 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.080991 31788 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:38.081866 31713 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:38.080670 31961 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:38.080858 31789 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:38.081072 31603 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:12:38.085569 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:38.085800 31826 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:38.084692 31789 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.087283 31603 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:38.087703 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:38.057575 31747 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:12:38.049749 31600 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:38.089335 31682 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:38.089630 31600 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.089823 31682 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.089964 31608 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:38.090530 31608 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:38.090991 31608 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:38.091188 31853 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.005587 31580 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:38.091789 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:38.092167 31580 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.092432 28229 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:38.092607 31580 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:38.082181 31788 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:38.093488 31788 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:38.090103 31600 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.094515 31788 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.095986 31788 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.080698 31671 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:38.097939 31973 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:12:38.091792 31853 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:12:38.101083 31671 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:38.104451 31962 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.423s	user 0.002s	sys 0.000s
I20260709 13:12:38.105260 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:38.109951 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 889) took 1167 ms. Trace:
I20260709 13:12:38.110399 28755 rpcz_store.cc:276] 0709 13:12:36.942878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.943431 (+   553us) service_pool.cc:224] Handling call
0709 13:12:38.109733 (+1166302us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54004d21d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:38.109888 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2317243776}
I20260709 13:12:38.111274 31826 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:38.112286 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:38.112458 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:38.112972 28230 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:38.113119 31793 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:38.113626 31793 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.114022 31793 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:38.087812 31603 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:38.114806 31682 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:38.105173 31927 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:38.137532 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.138309 28079 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:38.138458 31155 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:38.140367 31370 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:38.140874 31370 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:38.141794 31370 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.143198 31370 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:38.136195 31747 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:38.145366 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.145809 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.146059 28159 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:38.146432 28152 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:38.146624 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:38.148262 31671 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:38.148698 31671 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.149052 31671 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.152807 31936 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:38.152640 31826 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.153445 31936 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.164708 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.156471 31716 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.172791 28158 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:38.115895 31603 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:38.175170 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 44942.6us
I20260709 13:12:38.134421 31833 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:38.145479 31747 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.183610 28079 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:38.185864 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.189396 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.190137 28084 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:38.190930 31716 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.192811 31803 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.193271 31716 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:38.193359 31803 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.195706 28085 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:12:38.195834 31803 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.172441 31754 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.197278 28080 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:38.197314 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.198244 31754 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.198765 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.179816 31805 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.189599 31603 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.201756 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.193872 31826 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:38.207585 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.222645 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:38.223011 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:38.183177 31833 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.225728 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:38.226536 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:38.226629 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:38.230381 28234 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:38.231066 28229 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:38.200153 31747 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:38.241772 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:38.241847 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 262750us
I20260709 13:12:38.246212 28157 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.249645 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:38.208477 31801 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:38.251737 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49202.1us
I20260709 13:12:38.251560 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.261469 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:38.234840 31833 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.285451 28161 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:38.293819 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.294448 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.294643 28153 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:38.294751 31793 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.295617 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:38.296065 31722 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:38.296304 28756 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:38.295976 31801 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.296939 31722 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.298806 31801 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.299508 31722 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:38.300719 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.304939 28198 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:38.305853 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:38.306432 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:38.306699 28755 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:38.307267 31464 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:38.307682 31464 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:38.311650 28049 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:38.313247 31930 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:38.314792 28078 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:38.299458 28154 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:12:38.316912 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:38.318327 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.319736 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:38.315894 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.289570 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.319782 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.286412 28160 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:38.203905 31853 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:12:38.337294 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:38.337965 31815 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:38.315979 31927 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.339834 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:38.340890 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.345495 31155 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:38.366292 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.367467 28084 raft_consensus.cc:2410] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:12:38.350235 31464 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.288407 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:38.351590 31682 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.344208 28049 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:38.349051 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.337195 31936 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:38.333390 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.333238 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:38.335963 31925 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:38.382428 31925 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:38.331000 28082 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:38.245059 31921 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:12:38.384136 31921 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:38.384680 31921 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.385421 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.386183 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.386641 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:38.362378 28079 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:38.387327 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:38.388195 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:38.366262 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.389442 31682 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.352766 31793 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.394625 28081 raft_consensus.cc:2368] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12.
I20260709 13:12:38.364521 28124 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:38.378444 31464 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:38.362639 31608 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.380375 28049 leader_election.cc:400] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10.
I20260709 13:12:38.380251 31551 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:38.380800 31754 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:38.386860 31921 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:38.360767 28755 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:38.387260 31927 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.387355 28080 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:12:38.393679 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.262781 28156 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:38.404444 28152 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:38.281703 31933 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.410383 28084 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:38.412181 28085 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:38.413516 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:38.375569 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.393265 31815 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 12 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:38.436682 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.437050 28086 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:38.422542 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:38.449479 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.449815 28082 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:38.451098 31551 consensus_queue.cc:1243] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:38.452298 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.452937 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:38.453117 31608 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.432883 31754 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:38.455789 31754 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:38.438840 31722 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:38.462296 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:38.463037 28227 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:38.468825 28756 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:38.469709 31667 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.469478 31815 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.470345 31667 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:38.474777 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.475618 31667 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:38.481390 28730 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:38.483242 31662 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:38.483687 31671 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.253710 31826 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:38.486898 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64457.4us
I20260709 13:12:38.487860 28730 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:38.494655 28158 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:38.256426 31918 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:38.494184 28228 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:38.497274 28080 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:38.497675 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:38.501513 28159 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:38.498823 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:38.509033 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 50973us
W20260709 13:12:38.520550 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 190590us
I20260709 13:12:38.519884 28077 raft_consensus.cc:2393] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:38.524641 28080 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:12:38.499455 31662 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:38.525669 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.532051 28157 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:38.514719 31921 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:38.439353 31668 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
W20260709 13:12:38.534523 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 154324us
I20260709 13:12:38.537132 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.515482 31957 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:38.511570 31918 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:38.554306 31671 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.520294 31682 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:38.557152 31682 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.542254 28082 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:38.547286 28160 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:12:38.548090 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.550009 31927 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:38.536446 31921 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:38.559610 31682 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:38.552146 31918 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.560545 31921 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.561189 31933 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.562152 31921 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.563045 31853 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:38.565311 31853 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:38.565750 31853 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:38.569650 31853 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.571313 31853 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:38.536046 31662 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:38.573078 31662 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.583611 31662 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.585150 31916 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:38.625737 31916 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.626194 31916 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:38.626474 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 53920.1us
I20260709 13:12:38.486421 31600 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.564455 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:38.630347 31803 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:12:38.630824 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 66446.3us
I20260709 13:12:38.596932 31580 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.631484 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:38.632318 31801 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:12:38.598536 28227 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:38.511518 31667 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:12:38.632833 31801 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:38.606912 31608 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:38.635820 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:38.630383 31600 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.523761 31826 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.609921 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:38.613075 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.618198 31603 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:38.606240 28125 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:38.632898 31580 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:38.640816 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:38.646638 28077 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:38.607066 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.647696 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:38.648909 28234 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:38.646019 28231 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:38.681214 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:38.681905 28227 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.647590 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:38.684230 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 78160us
I20260709 13:12:38.644232 31793 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:38.688174 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:38.689420 28153 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:38.690553 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:38.692296 31608 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.694880 31608 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:38.694960 28077 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:38.695562 31608 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:38.695948 31608 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.696324 31608 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:38.697700 28231 pending_rounds.cc:85] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 5
I20260709 13:12:38.698122 28231 pending_rounds.cc:107] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:12:38.698486 28231 raft_consensus.cc:2889] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:38.699206 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:38.700963 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:38.702505 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:38.706640 31793 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:38.708036 31793 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.637442 31667 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:38.726416 28228 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: [9.8-10.9]
I20260709 13:12:38.738209 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:38.738873 28234 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:38.739809 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:38.741164 31805 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:38.745661 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:38.637872 31930 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:38.612383 31754 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.757009 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:38.773465 31803 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:38.774246 31803 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:38.775455 31803 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.776618 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:38.681730 31603 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:38.776886 31805 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.777309 28154 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:38.778352 31603 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.779224 31805 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:38.747855 31921 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:38.781143 31921 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.783459 31713 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:38.784093 31713 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:38.785058 31713 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.785852 28154 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:38.748924 31667 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:38.788282 31667 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.786949 31716 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:38.797438 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.804137 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:38.807402 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:38.789793 31667 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.810756 28085 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:38.812960 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 62514.1us
I20260709 13:12:38.817142 31716 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:38.828392 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:38.831816 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 496372us
I20260709 13:12:38.833386 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:38.836066 28232 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.838107 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:38.845116 31801 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:38.808558 31722 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:38.849109 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:38.860356 28085 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:38.868309 28125 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:38.832296 31936 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:38.811599 31805 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:12:38.878024 28125 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:38.856664 31603 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:38.887521 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 76295us
W20260709 13:12:38.889575 28757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.007s	sys 0.039s
W20260709 13:12:38.891332 28757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.007s	sys 0.039s
I20260709 13:12:38.876103 31927 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:38.891636 28226 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.005s	sys 0.041s
I20260709 13:12:38.875602 31936 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.885167 31957 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:12:38.892232 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.892361 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:38.875847 31933 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:38.902383 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53347.8us
I20260709 13:12:38.891865 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:38.915202 28227 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:38.953531 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.926795 28050 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:38.954214 28155 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:38.886615 31722 consensus_queue.cc:1059] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:12:38.959533 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:38.960366 28154 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:38.962491 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 70212.9us
I20260709 13:12:38.905125 31794 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:38.905669 31936 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:38.963826 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:38.966732 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:38.931721 31933 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:12:38.968806 28079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.020s	sys 0.025s
W20260709 13:12:38.973285 28079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.020s	sys 0.025s
W20260709 13:12:38.985553 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 93758.8us
I20260709 13:12:38.953362 31957 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:38.990433 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.995108 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:38.996678 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:38.997516 28155 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:38.931783 31927 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.025962 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:39.026649 28161 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:39.026934 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:39.027567 28159 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:38.986405 31933 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:38.989193 31580 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:39.031831 28156 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:12:39.000140 31464 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.003s
I20260709 13:12:39.034240 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:39.035245 28763 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 6: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:39.032013 28761 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:12:39.032545 28230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.021s	sys 0.034s
W20260709 13:12:39.036711 28230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.022s	sys 0.034s
I20260709 13:12:39.033154 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:39.032567 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:39.051106 28755 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:39.064107 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:39.087603 28232 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:39.020309 31662 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:39.092650 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:39.089001 28157 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.009s	sys 0.022s
W20260709 13:12:39.097461 28157 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.010s	sys 0.022s
W20260709 13:12:39.072705 31815 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.032s	sys 0.031s
I20260709 13:12:39.105393 28152 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:39.041383 31754 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:39.110785 31754 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:39.112314 31754 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:39.105024 31957 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:39.106033 31992 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:39.107064 31853 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.000s
I20260709 13:12:39.108366 31716 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:39.089697 31826 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:39.151046 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 118337us
I20260709 13:12:39.151437 28159 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:39.153920 31853 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:39.155925 31716 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:39.118177 31815 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.033s	sys 0.031s
I20260709 13:12:39.129168 31754 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:39.128636 31992 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.094776 31920 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:39.157404 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:39.170452 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.182900 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 219139us
W20260709 13:12:39.183684 31833 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.806s	user 0.011s	sys 0.000s
I20260709 13:12:39.184126 31833 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:39.185616 31916 consensus_meta.cc:287] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.559s	user 0.008s	sys 0.003s
I20260709 13:12:39.186058 31916 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:39.186691 31754 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.188179 31916 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:39.166313 31600 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:39.165112 31716 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.194257 28235 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:39.194986 31716 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:39.196866 31992 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.199299 31992 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:39.201259 31927 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:39.201102 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:39.160420 28154 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
W20260709 13:12:39.204277 28231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.018s	sys 0.028s
W20260709 13:12:39.204654 28231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.018s	sys 0.028s
I20260709 13:12:39.216112 28161 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:39.221691 31815 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:39.221920 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:39.223302 31815 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:39.223644 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:39.224647 31815 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:39.227229 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:39.228039 31464 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:39.228463 31464 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:39.228868 31464 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:39.233036 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:39.234289 31815 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:39.234860 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:39.236440 31927 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:39.222167 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:39.236443 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:39.285615 28155 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
W20260709 13:12:39.288971 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 68335.3us
W20260709 13:12:39.348415 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 112079us
I20260709 13:12:39.293684 31916 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:12:39.356746 31713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.011s	sys 0.005s
W20260709 13:12:39.357308 31713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.013s	sys 0.005s
I20260709 13:12:39.287094 31853 raft_consensus.cc:2764] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:12:39.363476 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:39.374239 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:39.361196 31747 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.077s	user 0.001s	sys 0.011s
W20260709 13:12:39.377014 31918 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.018s	sys 0.046s
W20260709 13:12:39.384446 31608 consensus_meta.cc:287] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.000s
W20260709 13:12:39.394354 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.390144 31918 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.018s	sys 0.046s
W20260709 13:12:39.390959 31990 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.317s	user 0.001s	sys 0.000s
W20260709 13:12:39.398149 28158 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.008s	sys 0.009s
W20260709 13:12:39.398664 28158 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.008s	sys 0.009s
W20260709 13:12:39.392459 31668 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.062s	sys 0.152s
I20260709 13:12:39.386178 31991 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:39.381758 31747 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:39.397905 31801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.001s	sys 0.017s
W20260709 13:12:39.418881 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.400450 31930 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.000s	sys 0.017s
I20260709 13:12:39.432156 28229 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:39.434594 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:39.406505 31668 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.062s	sys 0.153s
W20260709 13:12:39.426116 31930 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.000s	sys 0.018s
I20260709 13:12:39.396723 31608 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:39.448774 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 416232us
W20260709 13:12:39.453881 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54604.2us
I20260709 13:12:39.469105 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:39.422487 31801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.002s	sys 0.017s
I20260709 13:12:39.448031 31930 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:39.430166 31991 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:39.449324 31662 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 13:12:39.493242 28125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.010s	sys 0.003s
W20260709 13:12:39.493639 28125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.010s	sys 0.003s
I20260709 13:12:39.431933 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2194) took 1091 ms. Trace:
I20260709 13:12:39.491113 31788 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:39.506531 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 481013us
W20260709 13:12:39.491932 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.513015 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44018.3us
W20260709 13:12:39.491369 28050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.003s	sys 0.009s
W20260709 13:12:39.526906 28050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.004s	sys 0.009s
I20260709 13:12:39.476854 31992 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:39.474531 31933 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.496958 28079 rpcz_store.cc:276] 0709 13:12:38.340803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.390138 (+ 49335us) service_pool.cc:224] Handling call
0709 13:12:38.390478 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:38.390761 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:38.391788 (+  1027us) log.cc:844] Serialized 14 byte log entry
0709 13:12:39.431422 (+1039634us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:39.431502 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:39.431570 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.431669 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.431851 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":296,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":137425,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":962240,"threads_started":2,"wal-append.queue_time_us":934937}
W20260709 13:12:39.529268 28764 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.006s	sys 0.009s
W20260709 13:12:39.493971 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 616005us
W20260709 13:12:39.538100 28764 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.007s	sys 0.009s
I20260709 13:12:39.542292 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:39.553766 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 627039us
I20260709 13:12:39.558755 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:39.564572 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:39.562602 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 713) took 1235 ms. Trace:
I20260709 13:12:39.567098 28199 leader_election.cc:400] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12.
I20260709 13:12:39.567595 28199 leader_election.cc:403] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:39.503343 31464 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:39.572616 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:39.573076 28125 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:39.574478 28081 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:39.511075 31788 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:39.581966 28050 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:39.580334 31916 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:39.583294 31996 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:12:39.585731 28085 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:39.587023 31918 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:39.587469 31918 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:39.587901 31918 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:39.584643 31788 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:39.572458 31999 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:12:39.565658 28230 rpcz_store.cc:276] 0709 13:12:38.320498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.336625 (+ 16127us) service_pool.cc:224] Handling call
0709 13:12:38.356129 (+ 19504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:38.376778 (+ 20649us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:38.416214 (+ 39436us) log.cc:844] Serialized 14 byte log entry
0709 13:12:39.530568 (+1114354us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:39.530639 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:39.555322 (+ 24683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.555597 (+   275us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.555779 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1189,"prepare.queue_time_us":38982,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":139263,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":37088,"spinlock_wait_cycles":3306368,"thread_start_us":1081102,"threads_started":2,"wal-append.queue_time_us":939240}
I20260709 13:12:39.528923 31991 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:39.594920 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:39.569929 32015 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:39.590649 31667 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.763s	user 0.008s	sys 0.005s
I20260709 13:12:39.599149 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:39.580937 31747 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:39.590528 31992 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12.
W20260709 13:12:39.592448 32005 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:12:39.588073 31916 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:39.596239 32012 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:12:39.607954 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47428.1us
I20260709 13:12:39.595463 31801 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:39.601624 32015 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:39.632843 31754 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:39.634347 31754 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:39.634831 31754 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:39.636161 32019 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:39.637647 32019 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:39.638027 32019 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:39.634335 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2195) took 1293 ms. Trace:
W20260709 13:12:39.641544 32001 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:12:39.641481 28078 rpcz_store.cc:276] 0709 13:12:38.341109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.390826 (+ 49717us) service_pool.cc:224] Handling call
0709 13:12:38.391165 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:38.406880 (+ 15715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:38.406928 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:38.493414 (+ 86486us) log.cc:844] Serialized 26 byte log entry
0709 13:12:38.925085 (+431671us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:39.445281 (+520196us) log.cc:844] Serialized 14 byte log entry
0709 13:12:39.446686 (+  1405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.446799 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:39.634119 (+187320us) raft_consensus.cc:1685] finished
0709 13:12:39.634139 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.634272 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38424,"prepare.queue_time_us":671624,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":12827,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":28583,"thread_start_us":867337,"threads_started":2,"wal-append.queue_time_us":767002}
I20260709 13:12:39.600144 32004 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:39.634907 31464 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:39.644351 32004 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:39.634771 32011 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:12:39.632177 31991 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:39.626763 32027 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:39.646627 32013 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:39.648279 32027 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:39.650293 28762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 660) took 1001 ms. Trace:
I20260709 13:12:39.658016 28081 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:39.650933 28762 rpcz_store.cc:276] 0709 13:12:38.648395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.648512 (+   117us) service_pool.cc:224] Handling call
0709 13:12:38.687026 (+ 38514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:38.687481 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:38.687514 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:38.688036 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:12:39.165670 (+477634us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:39.165714 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.165787 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:39.649996 (+484209us) raft_consensus.cc:1685] finished
0709 13:12:39.650024 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.650210 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4344,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"thread_start_us":477228,"threads_started":1,"wal-append.queue_time_us":717240}
I20260709 13:12:39.598430 31788 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:12:39.667846 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:39.671396 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 658) took 1023 ms. Trace:
W20260709 13:12:39.669076 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 87783.5us
I20260709 13:12:39.652895 31801 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:39.635963 31754 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.678974 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:39.679780 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:39.620494 31930 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:39.681367 31936 consensus_meta.cc:287] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.693s	user 0.012s	sys 0.000s
I20260709 13:12:39.671800 28764 rpcz_store.cc:276] 0709 13:12:38.647694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.647976 (+   282us) service_pool.cc:224] Handling call
0709 13:12:38.740142 (+ 92166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:38.740701 (+   559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:38.740767 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:38.742516 (+  1749us) log.cc:844] Serialized 564 byte log entry
0709 13:12:39.538557 (+796041us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:39.538620 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.538726 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:39.671125 (+132399us) raft_consensus.cc:1685] finished
0709 13:12:39.671154 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:39.671338 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":795538,"threads_started":1,"wal-append.queue_time_us":848652}
I20260709 13:12:39.692806 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:39.740836 28080 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:39.694624 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:39.718174 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:39.742681 28077 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:39.728220 28756 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:39.744819 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:39.745641 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.719986 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 53781.9us
I20260709 13:12:39.649410 32027 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:39.756495 28226 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:39.757557 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.724223 32022 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:39.717839 31801 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:39.650102 32015 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:39.759509 31788 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:39.749418 28234 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:12:39.761023 28234 pending_rounds.cc:85] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 8
I20260709 13:12:39.761374 28234 pending_rounds.cc:107] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:12:39.761713 28234 raft_consensus.cc:2889] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:39.761945 28764 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:39.752568 28757 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:39.753165 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:39.764005 28077 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:39.764278 31992 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:39.754562 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:39.751000 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.765244 28762 raft_consensus.cc:2368] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10.
I20260709 13:12:39.758195 28228 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:39.758831 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.746651 28049 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:39.755698 28050 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:39.768915 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.766433 28227 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:39.786842 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:39.793823 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45801.2us
I20260709 13:12:39.812055 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:39.758869 32024 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:39.764874 31992 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.795017 31788 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:39.815629 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.820739 31957 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.796046 31853 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.796397 31600 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:39.824126 31722 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:39.784142 32021 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:12:39.826880 28079 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:12:39.829185 28227 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:39.832072 28234 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:39.746050 31936 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.843397 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:39.854359 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.855321 28230 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
W20260709 13:12:39.857688 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:39.849833 28082 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:39.859603 28757 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:39.859923 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.861032 28226 raft_consensus.cc:2393] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:39.794112 31789 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:12:39.876986 31722 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:39.896966 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.897636 31551 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:39.839726 31793 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.843752 31927 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:39.905965 28081 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:12:39.903628 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.899657 31921 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:39.898768 31600 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:39.840364 31957 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:39.920116 31551 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.922555 31833 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:39.923449 28235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 709) took 1604 ms (client timeout 1603 ms). Trace:
I20260709 13:12:39.923601 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.863358 31853 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.837724 32024 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.924420 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:39.863685 32023 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:12:39.942252 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.928850 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:39.936440 28235 rpcz_store.cc:269] 0709 13:12:38.319330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.319474 (+   144us) service_pool.cc:224] Handling call
0709 13:12:39.923240 (+1603766us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b54000951d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:39.923378 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2009180416}
I20260709 13:12:39.943471 28227 raft_consensus.cc:2376] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10. Re-sending same reply.
I20260709 13:12:39.941509 28082 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:12:39.939798 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.904518 31713 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:39.925588 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 106638us
I20260709 13:12:39.976146 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.976263 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:39.977036 28227 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:39.939783 31580 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.969967 32032 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:39.970715 31933 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:39.971256 31853 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:39.971438 32035 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:12:39.971870 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:39.972271 32042 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 13:12:39.972911 32036 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:12:39.975099 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:39.981186 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:39.971468 31551 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:39.990286 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:39.973212 32022 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.941170 31918 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:39.970732 32009 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:39.916808 31793 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:40.022346 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 43992us
I20260709 13:12:39.976028 31370 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:40.026631 28227 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:12:39.979815 31921 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.057106 31789 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:40.059089 31801 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.061622 32031 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:40.062356 32031 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.067153 31722 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:39.980343 31788 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:40.024343 28230 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:40.044554 31551 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:40.019501 31794 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:40.073539 31794 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:40.073983 28233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 707) took 1755 ms (client timeout 2247 ms). Trace:
I20260709 13:12:40.074347 31957 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:40.074602 28233 rpcz_store.cc:269] 0709 13:12:38.318643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.320495 (+  1852us) service_pool.cc:224] Handling call
0709 13:12:40.073106 (+1752611us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b54000b0ca0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5eedc0 00007f59ce67ae18 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:40.073792 (+   686us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b5400305f54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:40.073928 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1885081344}
I20260709 13:12:40.077112 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 677) took 1531 ms. Trace:
I20260709 13:12:40.077471 32015 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.015628 31580 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.071332 31793 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.071934 31992 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:40.072023 31933 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:40.073321 31551 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.046654 31954 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:12:40.078959 28078 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:12:40.077538 28158 rpcz_store.cc:276] 0709 13:12:38.545963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.560066 (+ 14103us) service_pool.cc:224] Handling call
0709 13:12:38.560423 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:38.780060 (+219637us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:39.399193 (+619133us) log.cc:844] Serialized 14 byte log entry
0709 13:12:39.568555 (+169362us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:39.568614 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:39.579600 (+ 10986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:40.076887 (+497287us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54000a28d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:40.076936 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.077053 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10777,"prepare.queue_time_us":615406,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":4898,"raft.queue_time_us":102,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":6283,"spinlock_wait_cycles":453985024,"thread_start_us":753318,"threads_started":2,"wal-append.queue_time_us":401586}
I20260709 13:12:40.048223 32045 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:12:40.072436 32046 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:40.027709 32032 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:40.099690 32040 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:12:40.100498 32032 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.034024 32047 raft_consensus.cc:3055] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:40.103880 32047 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:40.104948 31936 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.105046 32047 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.106586 32047 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:40.107416 32015 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.087765 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:40.108819 32037 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:12:39.981828 31713 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.109692 32037 consensus_queue.cc:1243] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:12:40.109762 31801 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.077432 32019 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.113500 32019 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.114639 31826 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:40.115094 31826 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:40.115345 31713 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.119091 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.122020 31918 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:40.123868 28229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 2241) took 1672 ms (client timeout 1533 ms). Trace:
W20260709 13:12:40.124503 28229 rpcz_store.cc:269] 0709 13:12:38.451244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.452657 (+  1413us) service_pool.cc:224] Handling call
0709 13:12:40.123564 (+1670907us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b54000de8d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:40.123758 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2249729792}
I20260709 13:12:40.116124 31826 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:40.134014 32006 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.669s	user 0.005s	sys 0.000s
I20260709 13:12:40.134941 31826 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:40.135507 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2240) took 1684 ms. Trace:
I20260709 13:12:40.136161 28231 rpcz_store.cc:276] 0709 13:12:38.450977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.696998 (+246021us) service_pool.cc:224] Handling call
0709 13:12:38.697455 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:38.699221 (+  1766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:38.699266 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:38.699788 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:12:39.204998 (+505210us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:39.205039 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.205103 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:40.133248 (+928145us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:40.135044 (+  1796us) raft_consensus.cc:1685] finished
0709 13:12:40.135080 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.135418 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31869824,"thread_start_us":504787,"threads_started":1,"wal-append.queue_time_us":707956}
I20260709 13:12:40.140493 31794 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.143647 31788 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.147532 32031 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.148398 32049 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.149132 32049 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.150707 28079 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:40.153206 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:40.153990 31580 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:40.057106 31921 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.157734 31921 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:40.157940 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.045572 32022 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.159123 28153 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:40.043903 31600 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.161664 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:40.097028 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 285066us
I20260709 13:12:40.078392 31370 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:40.091959 32043 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:12:40.171794 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:40.172123 28077 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:40.172305 31370 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:40.172703 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:40.174356 32010 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.702s	user 0.002s	sys 0.000s
I20260709 13:12:40.174788 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.162426 28158 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:40.175673 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 783) took 1536 ms. Trace:
I20260709 13:12:40.176142 28084 rpcz_store.cc:276] 0709 13:12:38.639782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.640351 (+   569us) service_pool.cc:224] Handling call
0709 13:12:38.644615 (+  4264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:38.750770 (+106155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:38.750818 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:38.771376 (+ 20558us) log.cc:844] Serialized 26 byte log entry
0709 13:12:39.291655 (+520279us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:39.291695 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:39.291764 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:40.173643 (+881879us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:40.175420 (+  1777us) raft_consensus.cc:1685] finished
0709 13:12:40.175438 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:40.175621 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1794,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":105,"spinlock_wait_cycles":9016576,"thread_start_us":471905,"threads_started":1,"wal-append.queue_time_us":617102}
I20260709 13:12:40.175793 28079 raft_consensus.cc:2410] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:40.177793 31794 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.178674 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.096105 31933 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:40.185153 31997 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.790s	user 0.000s	sys 0.011s
I20260709 13:12:40.164830 28755 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:40.190192 28153 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:12:40.152609 32024 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:40.160493 32022 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:40.203086 31608 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.210253 32004 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.205957 31754 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:40.207492 31551 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.205724 31933 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:40.219587 31998 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.826s	user 0.000s	sys 0.003s
I20260709 13:12:40.221629 31608 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.222278 31608 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.223587 31801 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:40.224134 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.224469 31608 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.225176 32022 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.225517 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:40.225716 32022 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.226312 31801 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:40.227156 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.227887 32022 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.230418 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.232854 31850 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:40.233333 31850 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:40.233871 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.234306 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:40.235179 28161 raft_consensus.cc:2368] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:40.234668 31850 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.233292 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.236353 28155 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:40.240617 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:40.241777 28154 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:40.245406 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:40.246126 28154 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:40.254289 28155 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:12:40.263453 28158 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:40.264824 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.269428 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.272382 28152 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:12:40.274099 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.263667 28154 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:40.279409 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.263851 31850 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:40.240654 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:40.297865 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.324586 28229 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:40.326021 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 85465.6us
I20260709 13:12:40.326272 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.329646 28158 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:40.337527 28226 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:40.337684 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.337782 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:40.338286 28159 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:40.338589 28229 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:40.339504 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:40.339855 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.340342 28152 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:40.340704 28081 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:40.343755 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:40.344234 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 895) took 1374 ms. Trace:
I20260709 13:12:40.344650 32047 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:40.344903 32027 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:40.345248 32047 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.346801 28152 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:12:40.333804 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:40.347409 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 85873.2us
I20260709 13:12:40.347915 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:40.348465 28152 pending_rounds.cc:85] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 10
I20260709 13:12:40.348788 28152 pending_rounds.cc:107] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:12:40.349113 28152 raft_consensus.cc:2889] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:40.344677 28756 rpcz_store.cc:276] 0709 13:12:38.971004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.972619 (+  1615us) service_pool.cc:224] Handling call
0709 13:12:40.344065 (+1371446us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b5400439ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:40.344179 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1395053952}
I20260709 13:12:40.360841 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.372092 28078 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:40.380291 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.381067 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:40.380353 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:40.381273 28085 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
W20260709 13:12:40.385687 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 404569us
I20260709 13:12:40.388440 28049 leader_election.cc:400] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10.
I20260709 13:12:40.389087 28049 leader_election.cc:403] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:12:40.333233 32024 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:40.394526 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:40.394881 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:40.391943 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.397689 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:40.398898 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:40.399797 28233 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:40.402547 28755 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:40.404258 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:40.371443 31155 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:40.408134 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:40.373006 32021 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:40.346475 32027 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:12:40.414201 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:40.420221 28226 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:40.421146 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:40.426934 28756 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:40.421753 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:40.334172 31927 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
W20260709 13:12:40.432242 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 59911.9us
I20260709 13:12:40.433313 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:40.434906 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:40.437270 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:40.441211 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.422588 31936 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:40.445262 31936 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.445765 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:40.447738 31936 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.448004 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:40.425338 32021 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:40.448688 32021 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:40.448868 31933 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:40.449698 31933 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:40.450802 31608 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:40.451591 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:40.452345 32022 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:40.417987 31925 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:12:40.452809 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:40.453917 32022 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:40.454345 32022 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:40.454783 32022 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:40.395524 31793 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10.
I20260709 13:12:40.423323 31155 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:40.458552 31155 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:40.454413 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:40.409876 31921 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:40.461203 31921 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.461553 32013 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:40.462165 32050 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:40.463476 31921 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:40.463686 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:40.379735 31992 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:12:40.422761 32015 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.459950 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:40.395000 31918 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:12:40.396878 31713 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:12:40.424146 31815 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:40.464359 31722 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:40.495656 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:40.493995 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:40.464473 31826 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:40.487962 31921 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:40.494833 31801 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:40.489984 31793 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:40.493882 32019 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.497725 31918 consensus_queue.cc:1243] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:12:40.491488 32015 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.533522 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:40.534395 28083 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:40.503429 31713 consensus_queue.cc:1243] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:12:40.500576 31826 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:40.506338 32047 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:40.508428 32013 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:40.506438 31793 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:40.497540 32025 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:12:40.555764 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.597849 28159 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:40.536186 32050 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:40.556524 32015 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.558992 31925 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:40.560212 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:40.606328 28762 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:40.561496 28080 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:40.561913 28764 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:40.562223 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:40.605969 31925 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:40.609480 28160 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 685) took 2010 ms (client timeout 1543 ms). Trace:
W20260709 13:12:40.610090 28160 rpcz_store.cc:269] 0709 13:12:38.600771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.600929 (+   158us) service_pool.cc:224] Handling call
0709 13:12:40.609258 (+2008329us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b5400254c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:40.609405 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4340772608}
I20260709 13:12:40.611488 28156 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:40.569132 31815 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:40.613808 28083 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:12:40.532688 32019 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.554260 32023 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:12:40.552944 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:40.625808 28077 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:40.603101 32022 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.590297 31826 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:40.563704 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:40.592664 32013 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:40.598932 31793 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:40.551095 31722 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:12:40.604605 32050 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:40.612819 31815 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:40.657130 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.640831 28154 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:40.650831 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.658476 28153 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:12:40.714591 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:40.612998 32024 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:40.716025 28161 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:40.716626 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:40.655774 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.650584 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.657016 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.718641 28152 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:40.715956 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:40.640100 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 76484.1us
I20260709 13:12:40.721901 28233 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:40.722304 28226 raft_consensus.cc:2410] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:12:40.727959 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 330329us
I20260709 13:12:40.729343 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:40.738996 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:40.740810 28083 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 7: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:40.742710 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:40.647137 32013 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:40.743396 28235 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:40.650640 31803 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:40.746212 32049 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:40.638985 31155 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.650696 32050 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:40.747536 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:40.747830 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:40.744843 31788 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:12:40.748559 28758 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:12:40.817948 28080 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:40.747587 28160 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:40.750851 28086 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:40.820137 28086 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:40.609426 31925 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.751845 32015 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:40.751660 31815 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.757062 31921 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:40.712790 32021 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.757498 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:40.758128 31722 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:40.758152 32027 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:40.759053 31833 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:40.781863 28077 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:40.791465 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:40.769629 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48898.9us
I20260709 13:12:40.801714 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:40.808912 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.828951 28152 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:40.803056 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:40.831328 28159 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 7: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:40.830433 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:40.798683 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.818049 28758 pending_rounds.cc:85] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 8
I20260709 13:12:40.833837 28758 pending_rounds.cc:107] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:12:40.834206 28758 raft_consensus.cc:2889] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:40.749019 32069 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:40.813477 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:40.794224 31927 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:40.835366 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 77956.3us
I20260709 13:12:40.842787 28199 leader_election.cc:400] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:40.843132 28199 leader_election.cc:403] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:12:40.817631 28153 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:40.835737 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.845046 28081 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:40.838382 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:40.847545 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:40.813301 32046 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.841593 28198 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:40.855448 28078 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:12:40.854975 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:40.859717 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:40.843927 32033 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:40.861624 32033 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:40.862048 32033 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:40.863020 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:40.865264 28124 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:40.852389 32069 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.813614 31803 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:40.866639 31803 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:40.885771 32047 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:40.886262 32022 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:40.887560 32061 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:12:40.843973 31551 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:12:40.823552 31921 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:40.889750 31921 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.891194 31921 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:40.853314 28086 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.823570 32027 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:40.901425 28730 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.610s (SENT) [suppressed 10 similar messages]
I20260709 13:12:40.904367 31155 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.903957 32027 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.853639 31927 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.854077 32068 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:40.876571 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.874799 32025 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:40.844441 31713 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:12:40.866633 31957 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:12:40.872217 31833 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:12:40.877431 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:40.874539 32045 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:12:40.882603 32019 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:40.873124 32046 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.866199 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:40.884375 32069 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:40.884516 32024 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:40.865782 31805 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:40.982488 31805 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.858505 31936 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:40.984881 31936 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:40.985337 31936 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:40.933323 31957 consensus_queue.cc:1243] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:40.990315 31927 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:40.990732 31805 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.991026 32069 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:40.915566 28086 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:40.835785 31722 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:40.898485 31925 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:41.003773 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.941911 28080 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:40.957136 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:40.961035 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 94911.4us
I20260709 13:12:41.013878 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:40.963485 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:40.965716 32021 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:40.929755 32027 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:41.059790 32009 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:41.060207 32009 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:40.929908 32025 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:40.966682 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 103311us
I20260709 13:12:41.060947 32025 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:41.061875 31682 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:41.062426 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:41.063385 32053 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:41.063805 28050 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:40.896258 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:40.963999 32003 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:40.954648 32065 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:41.020809 28125 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:41.021947 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:41.023082 31722 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:41.001982 31833 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:41.023595 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:41.023996 32033 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.024358 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:41.023617 31918 raft_consensus.cc:2764] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:12:41.012814 31927 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:40.930154 32068 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.029136 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:41.034745 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:41.091689 28158 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:41.036135 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2244) took 2046 ms. Trace:
I20260709 13:12:40.992333 32071 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:41.045753 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:41.075415 32019 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.095636 28152 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:41.046059 31600 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:41.046377 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:41.049611 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:41.058156 31933 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:41.058878 32045 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:41.059223 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:41.065193 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:41.102828 28761 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:41.072451 28161 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:12:41.073230 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 213577us
I20260709 13:12:41.103534 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:41.105875 28160 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:40.990741 32046 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:41.064714 32053 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:41.084242 28756 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:12:41.083654 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 182264us
I20260709 13:12:41.024850 32069 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:41.089780 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:41.117508 28233 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:41.119560 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:41.038973 31853 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:41.092545 28156 rpcz_store.cc:276] 0709 13:12:38.989406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.074421 (+ 85015us) service_pool.cc:224] Handling call
0709 13:12:40.611238 (+1536817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.613418 (+  2180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.035936 (+422518us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400254c50. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:41.036073 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3471813504}
I20260709 13:12:41.064116 32009 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.079109 28086 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:12:41.099217 28080 raft_consensus.cc:2410] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:12:41.099205 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 52903.8us
I20260709 13:12:41.121157 28086 pending_rounds.cc:85] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 9
I20260709 13:12:41.121490 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:41.095746 32022 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:41.119438 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.121768 28086 pending_rounds.cc:107] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:12:41.097399 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:41.125552 28086 raft_consensus.cc:2889] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:41.125870 28230 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:41.120234 28229 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:41.121565 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:41.101944 31662 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:41.129793 28228 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:41.130193 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:41.105475 31788 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:41.134824 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:41.166656 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 303719us
I20260709 13:12:41.168897 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:41.123236 32023 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:41.182001 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:41.186043 32033 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:41.186755 31815 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:41.187899 32013 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:41.189035 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:41.191525 28084 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:41.122520 31853 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:41.102182 32019 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:41.171252 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.132447 31805 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:41.140697 31370 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:12:41.174281 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:41.197273 28230 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:41.198421 28229 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:41.193686 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:41.136616 32021 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:41.218874 28153 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:41.172570 32009 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:41.212287 28228 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:41.136004 32022 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:41.214412 28160 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:41.119629 31793 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:41.244586 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.242901 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.131582 31936 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.273507 28229 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 7: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:41.276363 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.283159 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:41.291628 28156 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:41.302434 28077 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:41.306041 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.307658 28762 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:12:41.291627 32057 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:12:41.341806 28156 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:41.342418 28153 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:41.211239 28233 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:41.172380 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:41.372579 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 200265us
I20260709 13:12:41.373809 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:41.412284 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:41.345523 32050 consensus_meta.cc:287] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.504s	user 0.013s	sys 0.000s
W20260709 13:12:41.417572 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 396864us
I20260709 13:12:41.421732 28156 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:41.419060 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:41.435598 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.390501 32088 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.390422 32015 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:41.432482 32067 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:41.306535 31853 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.390908 32089 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:41.442500 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:41.450299 28233 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:41.418251 32087 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:41.457969 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:41.461036 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:41.467550 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:41.467998 31155 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:41.480500 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:41.441944 32086 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:41.484376 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:41.442977 31662 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:41.491400 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:12:41.481914 32077 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
I20260709 13:12:41.446807 32068 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:41.505709 28085 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:12:41.504352 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:41.508226 28086 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:41.504034 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:41.481426 31155 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:41.449551 32050 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.487262 31722 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:41.487476 32071 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:41.484822 31789 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:41.511224 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:41.482216 32093 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:12:41.517215 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:41.438169 32022 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:41.517428 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:41.515944 31793 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:41.489632 31833 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:41.522842 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:41.524655 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 908) took 1125 ms. Trace:
I20260709 13:12:41.500713 31853 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:41.565913 28758 rpcz_store.cc:276] 0709 13:12:40.399285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.399424 (+   139us) service_pool.cc:224] Handling call
0709 13:12:40.438649 (+ 39225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.844669 (+406020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.522350 (+677681us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54004371d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:41.524586 (+  2236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711889792}
I20260709 13:12:41.464027 32088 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:12:41.515775 31803 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.630s	user 0.012s	sys 0.000s
I20260709 13:12:41.506299 32021 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:41.507609 32019 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:41.556188 32097 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:12:41.518389 28080 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:41.521881 32068 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:41.598949 32101 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:41.599017 32068 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:41.599643 32101 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:41.600651 32081 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
W20260709 13:12:41.601794 32083 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:12:41.602118 32101 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:41.602473 32068 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:41.602931 31833 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.603667 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:41.565975 28079 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 6.
I20260709 13:12:41.604532 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 799) took 1084 ms. Trace:
I20260709 13:12:41.605041 28079 rpcz_store.cc:276] 0709 13:12:40.520237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.520461 (+   224us) service_pool.cc:224] Handling call
0709 13:12:41.522695 (+1002234us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:41.604471 (+ 81776us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5652736}
W20260709 13:12:41.548071 32085 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:12:41.511186 32046 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:12:41.527560 32071 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:41.511545 31682 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:12:41.510833 31551 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:12:41.507428 32087 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:12:41.627107 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:41.627935 28079 raft_consensus.cc:2368] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
I20260709 13:12:41.629366 32071 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:41.632453 31826 consensus_meta.cc:287] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.976s	user 0.014s	sys 0.005s
I20260709 13:12:41.632884 31789 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:41.633203 31826 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.544031 31713 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:41.547340 32027 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:41.564590 32090 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 13:12:41.548329 32096 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.592574 28082 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:41.556815 31936 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:41.594517 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:41.647727 28763 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:41.603471 31833 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:41.571816 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:41.488507 32095 consensus_queue.cc:1175] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0faaace2b36c4bcebf2fa6835235e92b" responder_term: 9 status { last_received { term: 9 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 9 index: 9 } }
I20260709 13:12:41.638079 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:41.586772 32021 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:41.642146 31722 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:41.585035 32053 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
W20260709 13:12:41.649559 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 77835.1us
I20260709 13:12:41.662098 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:41.586694 32003 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.619122 31793 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:12:41.657897 32021 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:41.663558 32068 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:41.663952 32100 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:12:41.664184 32068 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:41.584949 31805 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:41.641412 31801 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:41.667467 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:41.589258 31803 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.675412 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:41.675851 31722 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:41.655599 32087 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:41.677978 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:41.679977 28764 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:41.684309 28227 raft_consensus.cc:2410] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:12:41.671062 32089 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:41.686970 32089 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:41.687403 32089 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:41.687682 32102 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:12:41.691498 31921 consensus_meta.cc:287] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.800s	user 0.003s	sys 0.009s
I20260709 13:12:41.640004 31713 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.698995 28763 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:41.700596 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:41.647028 32027 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:41.662945 32003 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.676901 31992 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:41.679507 32032 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:41.637459 31662 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.681851 32086 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:41.707141 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 907) took 1309 ms. Trace:
I20260709 13:12:41.680958 32087 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:41.685580 31803 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:41.692545 31833 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:41.711603 28760 rpcz_store.cc:276] 0709 13:12:40.398130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.398280 (+   150us) service_pool.cc:224] Handling call
0709 13:12:40.424463 (+ 26183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:40.482906 (+ 58443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:40.482940 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:40.582031 (+ 99091us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b18004c3540. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a522 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:12:40.761225 (+179194us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b2c00047778. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:40.761786 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:12:40.964774 (+202988us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:40.964848 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:40.965034 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:41.706920 (+741886us) raft_consensus.cc:1685] finished
0709 13:12:41.706939 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:41.707080 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41706,"raft.queue_time_us":62782,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3414,"spinlock_wait_cycles":10013312,"thread_start_us":196702,"threads_started":1,"wal-append.queue_time_us":429732}
I20260709 13:12:41.673585 31805 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:41.726348 32080 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:12:41.726897 32068 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:41.728964 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2275) took 1067 ms. Trace:
I20260709 13:12:41.729557 28234 rpcz_store.cc:276] 0709 13:12:40.661705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.723050 (+ 61345us) service_pool.cc:224] Handling call
0709 13:12:40.723623 (+   573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:40.759874 (+ 36251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:40.759905 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:40.760394 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:12:41.009998 (+249604us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:41.010054 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.010148 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:41.728696 (+718548us) raft_consensus.cc:1685] finished
0709 13:12:41.728720 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:41.728882 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35783,"raft.queue_time_us":36832,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":249166,"threads_started":1,"wal-append.queue_time_us":585259}
I20260709 13:12:41.732110 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:41.654744 32096 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.710886 31793 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:41.732841 28154 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:41.734711 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:41.735718 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:41.739558 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:41.716835 32087 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:41.740164 28079 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:41.746829 28226 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:41.737304 28234 raft_consensus.cc:2376] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 10. Re-sending same reply.
I20260709 13:12:41.753499 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.754583 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:41.757593 28226 raft_consensus.cc:2410] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:12:41.794425 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.797788 32099 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:12:41.724494 31801 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:12:41.799230 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:41.800663 28728 leader_election.cc:400] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
I20260709 13:12:41.801043 28728 leader_election.cc:403] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:12:41.752170 31793 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:41.802062 31793 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.805033 31803 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:12:41.805500 31803 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:41.805856 31803 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:41.806468 31805 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:41.807288 31805 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
W20260709 13:12:41.791018 32091 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:12:41.818372 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.822924 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.719023 32086 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:41.826342 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1847) took 1332 ms. Trace:
I20260709 13:12:41.808319 31805 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:12:41.791640 32088 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:41.844067 28078 rpcz_store.cc:276] 0709 13:12:40.494099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.494257 (+   158us) service_pool.cc:224] Handling call
0709 13:12:40.495220 (+   963us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:40.868697 (+373477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:41.825954 (+957257us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54001ae5d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:41.826244 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":827744512}
I20260709 13:12:41.846792 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:41.847520 28226 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:12:41.823791 32117 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:12:41.864938 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:41.840184 32103 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:12:41.923053 32003 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:41.877380 32027 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:41.884265 28079 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:41.894865 32086 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:41.896188 31713 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:41.902081 32096 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:41.907861 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.911561 32050 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:41.830142 32068 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:41.938189 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.915824 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1856) took 1043 ms. Trace:
I20260709 13:12:41.939124 28229 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
W20260709 13:12:41.916244 32025 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.855s	user 0.007s	sys 0.003s
I20260709 13:12:41.932063 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:41.931207 28730 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:41.935496 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 133733us
W20260709 13:12:41.937448 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 420060us
I20260709 13:12:41.944921 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:41.938966 28077 rpcz_store.cc:276] 0709 13:12:40.872466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.873331 (+   865us) service_pool.cc:224] Handling call
0709 13:12:41.915625 (+1042294us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54001aead4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:41.915765 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":847324544}
W20260709 13:12:41.945788 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.914299 31826 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:41.953455 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.954566 28229 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:41.954794 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:41.951886 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:41.960248 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:41.953114 32003 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:41.974526 28226 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:41.912580 31803 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:41.982197 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:41.982957 28226 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:41.971135 31826 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:41.997565 32009 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.752s	user 0.012s	sys 0.000s
I20260709 13:12:41.997190 31826 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.000247 31826 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:41.992352 32119 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:41.992944 32003 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:41.957087 32025 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.011056 32025 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:42.009941 32087 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.011736 28154 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:12:42.000334 32096 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:42.012717 32096 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:42.013265 31803 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:41.960079 32068 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.013648 32096 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.014477 32087 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:42.015681 32068 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:42.016705 28229 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:42.003196 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:41.991791 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:41.991027 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:42.004637 32071 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:41.993209 31801 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.021692 31801 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.002578 32021 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.032791 32089 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.051169 31801 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:42.052176 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.052788 32089 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:42.059486 32119 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:42.060045 32071 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:42.067504 28086 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:12:42.067620 31815 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:42.061645 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:42.070111 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:42.068569 31815 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.076947 32089 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.109313 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1846) took 1626 ms. Trace:
I20260709 13:12:42.068959 32021 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:42.083595 32053 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:42.083272 31713 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:42.109963 28081 rpcz_store.cc:276] 0709 13:12:40.483338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.490103 (+  6765us) service_pool.cc:224] Handling call
0709 13:12:40.490534 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:40.845562 (+355028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.108932 (+1263370us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b5400207e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:42.109207 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814817024}
I20260709 13:12:42.123208 28226 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:42.117668 32046 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:42.146661 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 64817.9us
W20260709 13:12:42.138247 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 183571us
W20260709 13:12:42.132169 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 180389us
I20260709 13:12:42.139570 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:42.143862 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:42.159157 28231 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:12:42.145213 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:42.115583 32089 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.162446 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:42.166499 28226 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:42.168483 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:42.173786 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:42.178182 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:42.161008 32106 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.165311 32022 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:42.182785 28235 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:42.184082 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 744) took 1055 ms. Trace:
I20260709 13:12:42.186303 28226 rpcz_store.cc:276] 0709 13:12:41.129564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.183139 (+1053575us) service_pool.cc:224] Handling call
0709 13:12:42.183495 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.183702 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.183737 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:42.183788 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:42.183827 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.183885 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:42.184018 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:42.167050 31713 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:42.195096 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:42.174220 31801 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.196422 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 745) took 1063 ms. Trace:
I20260709 13:12:42.197196 28226 rpcz_store.cc:276] 0709 13:12:41.133153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.195621 (+1062468us) service_pool.cc:224] Handling call
0709 13:12:42.195924 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.196099 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.196124 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:42.196165 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:42.196195 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.196244 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:42.196361 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:42.174953 32053 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:42.184499 32049 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:42.161162 32046 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:42.188210 32123 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.120s	user 0.005s	sys 0.000s
I20260709 13:12:42.210752 31713 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.202286 28226 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:12:42.173219 32096 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:12:42.206297 32046 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:42.206881 28231 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:42.190090 32022 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.219904 32022 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:42.221954 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 746) took 1086 ms. Trace:
I20260709 13:12:42.222443 28231 rpcz_store.cc:276] 0709 13:12:41.135915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.196588 (+1060673us) service_pool.cc:224] Handling call
0709 13:12:42.203184 (+  6596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.221622 (+ 18438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.221884 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:42.208104 32053 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.201202 32049 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.217141 32071 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:42.226831 32071 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.228462 32089 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:42.229204 32071 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:42.189352 32106 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.233274 32106 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:42.234302 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2266) took 1438 ms. Trace:
I20260709 13:12:42.234905 28159 rpcz_store.cc:276] 0709 13:12:40.795733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.028837 (+233104us) service_pool.cc:224] Handling call
0709 13:12:42.234043 (+1205206us) spinlock_profiling.cc:243] Waited 969 ms on lock 0x7b540010ac54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:42.234203 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2228377984}
I20260709 13:12:42.236657 28160 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:42.236776 28050 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:42.243628 32111 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.244222 32111 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.245262 32049 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:42.245710 31370 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:42.229482 32135 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:42.247584 32135 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.247951 32135 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:42.199550 31801 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.251021 31801 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:42.246634 32111 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:42.273949 31754 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:42.292307 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 106391us
I20260709 13:12:42.291195 31815 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:42.282155 32111 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:42.302398 32130 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:42.303738 32130 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:42.304129 32130 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:42.305233 32130 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.306705 32130 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:42.309144 32111 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:42.323791 28127 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.739s (SENT)
I20260709 13:12:42.324432 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:42.325423 28159 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:42.325345 32022 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:42.328269 32013 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:42.330078 28228 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:12:42.331283 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.333887 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:42.335690 28227 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:42.336970 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.292845 31754 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.409147 28227 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:42.413496 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:42.413790 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:42.414235 28158 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:42.414314 28152 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:42.418068 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:42.418767 28159 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:42.419479 31826 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.419744 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.420011 31826 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.412886 32069 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:42.421883 28161 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:42.421537 32069 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.422300 31826 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:42.424746 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:42.425380 28157 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:42.425668 32069 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:42.425168 31155 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.432464 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:42.441920 28077 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:42.442528 31155 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:42.444196 28152 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:42.448772 32135 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:42.429143 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:42.537135 28155 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:42.539428 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:42.540092 28161 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:42.449491 28077 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
W20260709 13:12:42.455233 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.543205 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:42.544135 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:42.544873 28762 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: []
I20260709 13:12:42.545221 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:42.545807 28755 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:42.546098 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:42.546983 28759 raft_consensus.cc:2410] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:12:42.547142 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:42.547835 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:42.552167 32139 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:42.553232 32139 consensus_queue.cc:1243] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:42.555591 32130 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:42.556149 32130 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:42.557307 32130 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.558634 32130 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:42.563966 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:42.565675 32032 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:12:42.566488 32032 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:42.568070 32032 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.571441 31826 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:42.572072 31826 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.574252 31826 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:42.436941 28228 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:42.491672 28083 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:42.500414 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:42.505820 28157 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
W20260709 13:12:42.506731 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 317894us
I20260709 13:12:42.584805 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:42.584942 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:42.586587 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:42.522516 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:42.587903 28234 raft_consensus.cc:2410] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:42.588441 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1895) took 1037 ms. Trace:
I20260709 13:12:42.589432 28234 rpcz_store.cc:276] 0709 13:12:41.551106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.520527 (+969421us) service_pool.cc:224] Handling call
0709 13:12:42.588352 (+ 67825us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:42.434581 32049 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:42.592928 28127 leader_election.cc:336] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.972s (SENT)
I20260709 13:12:42.592852 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:42.593767 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:42.598152 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:42.608000 28155 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:42.608296 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1909) took 1052 ms. Trace:
I20260709 13:12:42.615250 28230 rpcz_store.cc:276] 0709 13:12:41.555463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.597801 (+1042338us) service_pool.cc:224] Handling call
0709 13:12:42.608194 (+ 10393us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:42.597986 31805 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:42.598060 31662 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:42.616863 32050 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:42.659821 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.597565 31600 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:12:42.669157 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:42.662318 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.671169 28086 raft_consensus.cc:2368] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:42.663867 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.619582 32069 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:42.673367 28154 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:42.604506 31801 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:12:42.676896 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:42.624171 31805 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:42.682349 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.683058 28078 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:42.683342 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.684238 28086 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:42.685168 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.685072 31805 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:42.624292 28757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:12:42.700601 28757 raft_consensus.cc:606] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:42.679970 32069 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.697847 32071 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:42.679031 31850 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:12:42.699506 32089 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.694553 32050 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:42.730201 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.704236 31930 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:42.694875 31662 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:12:42.810978 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1857) took 1902 ms. Trace:
I20260709 13:12:42.811460 28084 rpcz_store.cc:276] 0709 13:12:40.909297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.922866 (+ 13569us) service_pool.cc:224] Handling call
0709 13:12:42.810695 (+1887829us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b5400296dd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:42.810907 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414197120}
I20260709 13:12:42.736202 31815 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.740990 31930 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.720865 32071 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.729039 32050 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.726734 32089 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.842687 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:42.851440 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 258701us
I20260709 13:12:42.827857 32047 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.819347 32013 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:12:42.863219 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 241226us
I20260709 13:12:42.857826 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:42.833595 31930 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:42.859146 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:42.829612 31713 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:42.838915 32049 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:42.867460 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.876909 28078 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:42.864298 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:42.832067 31815 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.884387 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:42.886116 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:42.892532 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.894311 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.895885 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.902097 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.902231 28763 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:42.902756 28078 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:42.904138 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:42.880268 32144 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
W20260709 13:12:42.880719 32142 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:12:42.908115 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:42.909055 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:42.903873 28049 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:42.866544 32047 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:42.911728 32140 log.cc:927] Time spent T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:12:42.902267 32069 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:42.868115 32013 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:42.918213 31671 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:42.907203 32049 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:42.935602 32145 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:12:42.943986 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:42.944764 28764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 912) took 1922 ms (client timeout 2320 ms). Trace:
I20260709 13:12:42.881268 31713 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:42.946875 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1904) took 1392 ms. Trace:
I20260709 13:12:42.955029 32143 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:12:42.958057 32149 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:12:42.963505 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 916) took 1295 ms. Trace:
I20260709 13:12:42.965557 32025 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.966130 32089 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:42.970589 28080 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:12:42.893532 31850 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:42.982164 28229 rpcz_store.cc:276] 0709 13:12:41.554307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.959869 (+405562us) service_pool.cc:224] Handling call
0709 13:12:42.946658 (+986789us) spinlock_profiling.cc:243] Waited 55.2 ms on lock 0x7b54003566d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:42.946807 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126945152}
W20260709 13:12:42.981822 28764 rpcz_store.cc:269] 0709 13:12:41.022603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.023338 (+   735us) service_pool.cc:224] Handling call
0709 13:12:42.944496 (+1921158us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b54004c5ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:42.944665 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357400960}
I20260709 13:12:42.988049 28756 raft_consensus.cc:2410] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:12:42.987030 28758 rpcz_store.cc:276] 0709 13:12:41.668218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.668390 (+   172us) service_pool.cc:224] Handling call
0709 13:12:41.682200 (+ 13810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:41.717445 (+ 35245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:41.717490 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:41.928389 (+210899us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b18004c38a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a
0709 13:12:41.934790 (+  6401us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b2c00047778. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:12:41.940569 (+  5779us) log.cc:844] Serialized 26 byte log entry
0709 13:12:42.136261 (+195692us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:42.136981 (+   720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.963011 (+826030us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b54004371d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:42.963083 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:42.963164 (+    81us) raft_consensus.cc:1685] finished
0709 13:12:42.963184 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:42.963417 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":991,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":11568,"spinlock_wait_cycles":90118144,"thread_start_us":164201,"threads_started":1,"wal-append.queue_time_us":202557}
I20260709 13:12:42.976966 32069 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:42.978300 32050 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:42.984620 32143 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:43.052470 28080 pending_rounds.cc:85] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 6
I20260709 13:12:43.054715 28080 pending_rounds.cc:107] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:12:43.055102 28080 raft_consensus.cc:2889] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:43.014335 32025 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:43.017076 32089 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.984674 32049 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:43.101864 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:43.049431 31801 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.108635 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1905) took 1554 ms. Trace:
I20260709 13:12:43.111629 28231 rpcz_store.cc:276] 0709 13:12:41.554527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.249664 (+695137us) service_pool.cc:224] Handling call
0709 13:12:42.250082 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:42.250348 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.250375 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:42.250954 (+   579us) log.cc:844] Serialized 26 byte log entry
0709 13:12:42.497112 (+246158us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:42.532429 (+ 35317us) log.cc:844] Serialized 14 byte log entry
0709 13:12:42.585919 (+ 53490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.586023 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:43.108395 (+522372us) raft_consensus.cc:1685] finished
0709 13:12:43.108416 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.108569 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52196,"prepare.queue_time_us":34836,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":4069,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":89,"thread_start_us":245680,"threads_started":1,"wal-append.queue_time_us":279176}
I20260709 13:12:43.118003 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:43.127251 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:43.127460 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 232091us
W20260709 13:12:43.131763 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 120045us
I20260709 13:12:43.063921 31815 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:43.044030 32069 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.093427 32071 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:43.154160 28157 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.000s	sys 0.015s
I20260709 13:12:43.160785 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 735) took 2054 ms. Trace:
W20260709 13:12:43.163199 28157 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.000s	sys 0.015s
I20260709 13:12:43.161942 28232 rpcz_store.cc:276] 0709 13:12:41.106012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.412048 (+306036us) service_pool.cc:224] Handling call
0709 13:12:41.434048 (+ 22000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:41.483470 (+ 49422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:41.483531 (+    61us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:41.483571 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:41.483607 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.160465 (+1676858us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b54003ca0d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:43.160523 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.160676 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113128704}
W20260709 13:12:43.127789 32021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.014s	sys 0.000s
I20260709 13:12:43.115193 32015 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.109169 31671 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:43.142972 31155 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:43.150442 31957 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:43.094903 32089 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.219084 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 743) took 2102 ms. Trace:
I20260709 13:12:43.219853 28233 rpcz_store.cc:276] 0709 13:12:41.116663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:41.474979 (+358316us) service_pool.cc:224] Handling call
0709 13:12:41.484778 (+  9799us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:41.485003 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:42.727308 (+1242305us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b18003a2fa0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf
0709 13:12:42.727501 (+   193us) log.cc:844] Serialized 14 byte log entry
0709 13:12:43.218195 (+490694us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:43.218591 (+   396us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:43.218711 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.218837 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.219004 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34057,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"spinlock_wait_cycles":9555328,"thread_start_us":564404,"threads_started":3}
W20260709 13:12:43.194583 32021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.015s	sys 0.000s
I20260709 13:12:43.200912 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2295) took 1015 ms. Trace:
I20260709 13:12:43.161300 31815 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:43.133862 31801 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:43.227855 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 126068us
I20260709 13:12:43.238575 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:43.239861 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1906) took 1685 ms. Trace:
I20260709 13:12:43.224850 28228 rpcz_store.cc:276] 0709 13:12:42.185746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.581583 (+395837us) service_pool.cc:224] Handling call
0709 13:12:42.581993 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.582258 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:42.979009 (+396751us) log.cc:844] Serialized 14 byte log entry
0709 13:12:43.200119 (+221110us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:43.200255 (+   136us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:43.200367 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.200501 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.200779 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25116,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"thread_start_us":614805,"threads_started":2}
I20260709 13:12:43.249601 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2296) took 1061 ms. Trace:
I20260709 13:12:43.252971 28234 rpcz_store.cc:276] 0709 13:12:42.188169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.590264 (+402095us) service_pool.cc:224] Handling call
0709 13:12:42.592027 (+  1763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.592250 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:42.868578 (+276328us) log.cc:844] Serialized 14 byte log entry
0709 13:12:43.248909 (+380331us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:43.248985 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:43.249050 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.249140 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.249528 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":72996,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"thread_start_us":643947,"threads_started":3}
W20260709 13:12:43.258528 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 119825us
W20260709 13:12:43.231036 32142 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:12:43.251547 28226 rpcz_store.cc:276] 0709 13:12:41.554759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.251944 (+697185us) service_pool.cc:224] Handling call
0709 13:12:42.252282 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.252551 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:12:42.282387 (+ 29836us) log.cc:844] Serialized 14 byte log entry
0709 13:12:42.812187 (+529800us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b180034de60. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a 00007f59ce681a2a
0709 13:12:43.239392 (+427205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:43.239468 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:43.239531 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.239624 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.239785 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28356,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"spinlock_wait_cycles":24267136,"thread_start_us":426605,"threads_started":2}
W20260709 13:12:43.295262 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49306.7us
I20260709 13:12:43.229377 32159 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.297946 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:43.298354 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:43.299283 28154 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:43.304105 32161 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:43.305004 32025 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:43.305284 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:43.306169 32182 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:43.306270 32166 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:43.306725 32182 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:43.307284 32182 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:43.307410 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:43.308693 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:43.306639 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:43.312304 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:43.313439 28154 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:43.210484 32015 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:43.319885 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:43.206920 31671 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:43.323731 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:43.328246 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:43.328615 32161 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:12:43.329067 32151 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:12:43.329233 28153 raft_consensus.cc:2368] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11.
I20260709 13:12:43.329414 32161 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:43.330525 32152 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:12:43.332194 31713 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:43.268755 32047 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:43.332718 32166 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.334883 32174 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:43.335395 32166 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:43.230281 31936 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:12:43.336591 32180 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:43.322121 32013 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:43.163578 32143 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:43.296716 32156 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 13:12:43.335445 32174 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:43.346094 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:43.355100 28226 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:43.348642 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:43.309486 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:43.357826 28159 raft_consensus.cc:2368] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11.
I20260709 13:12:43.356932 28156 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:43.371239 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:43.372013 28233 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:43.242664 31815 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.407963 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:43.408895 28234 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:43.414968 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:43.417994 32150 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 13:12:43.419636 31671 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:43.518524 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:43.318070 32176 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.177s
I20260709 13:12:43.525240 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:43.544230 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:43.569533 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 357919us
I20260709 13:12:43.372543 31713 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.159s
I20260709 13:12:43.322363 32159 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.427208 32168 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:43.414323 32088 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.133s
I20260709 13:12:43.619926 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2291) took 1092 ms. Trace:
I20260709 13:12:43.646384 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 819) took 1216 ms. Trace:
I20260709 13:12:43.647001 28085 rpcz_store.cc:276] 0709 13:12:42.429715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.429924 (+   209us) service_pool.cc:224] Handling call
0709 13:12:42.430264 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:42.508588 (+ 78324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.508621 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:42.509186 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 13:12:42.746122 (+236936us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:42.746166 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:42.746248 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:43.646157 (+899909us) raft_consensus.cc:1685] finished
0709 13:12:43.646177 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.646308 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77768,"raft.queue_time_us":78654,"spinlock_wait_cycles":7752448,"thread_start_us":155526,"threads_started":1,"wal-append.queue_time_us":342445}
I20260709 13:12:43.436316 28233 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
W20260709 13:12:43.664830 28050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.004s	sys 0.009s
I20260709 13:12:43.648518 28157 rpcz_store.cc:276] 0709 13:12:42.527387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.586769 (+ 59382us) service_pool.cc:224] Handling call
0709 13:12:42.587028 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.587232 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:12:43.164395 (+577163us) log.cc:844] Serialized 14 byte log entry
0709 13:12:43.619442 (+455047us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:43.619521 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:12:43.619588 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.619682 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.619850 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15441,"prepare.queue_time_us":247912,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"thread_start_us":997372,"threads_started":2}
W20260709 13:12:43.668331 28050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.004s	sys 0.010s
I20260709 13:12:43.671458 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:43.671458 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:43.672595 28158 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:43.672693 28761 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:43.666513 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:43.673868 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 526365us
I20260709 13:12:43.683231 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:43.604657 32049 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:43.703979 28234 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
W20260709 13:12:43.710386 28155 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.003s	sys 0.022s
I20260709 13:12:43.657974 31155 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:43.703452 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 452476us
I20260709 13:12:43.653370 32176 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:12:43.667666 31788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.000s	sys 0.008s
W20260709 13:12:43.712636 28155 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.003s	sys 0.022s
W20260709 13:12:43.719913 28081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.000s	sys 0.026s
W20260709 13:12:43.723541 28081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.001s	sys 0.027s
I20260709 13:12:43.681293 32161 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:43.687534 32187 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:43.687987 32188 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:43.734776 28158 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:43.674759 32168 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.737018 32015 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:43.745030 32069 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:43.745337 31155 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:43.730044 32071 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:12:43.805127 32071 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:43.805649 32071 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:43.746264 32089 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:43.747428 32187 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:43.748893 31805 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 1.044s	user 0.013s	sys 0.000s
W20260709 13:12:43.749112 32171 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:12:43.750236 32113 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.750475 32033 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:12:43.722188 31788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.000s	sys 0.009s
I20260709 13:12:43.702365 32191 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:43.762198 28049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.024s	sys 0.096s
I20260709 13:12:43.721875 32176 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:12:43.821305 28049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.024s	sys 0.096s
I20260709 13:12:43.685019 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:43.776269 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 93154.9us
I20260709 13:12:43.792496 31815 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:43.799909 32168 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:43.616464 32174 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:43.761466 31713 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:43.851583 32021 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:43.852052 32021 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:43.853011 32021 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:43.854578 32021 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:43.757854 32161 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:43.856976 32161 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:43.857161 32200 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:43.857524 32161 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:43.857141 32071 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:43.857793 32200 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:43.858470 32071 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:43.858747 32200 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.861179 32015 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:43.818853 28199 leader_election.cc:400] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:43.865020 28199 leader_election.cc:403] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:12:43.865543 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46937.1us
I20260709 13:12:43.865883 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:43.866699 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:43.867125 31815 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:43.867903 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:43.868635 31713 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:43.869148 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:47466 (request call id 0) took 1252 ms. Trace:
I20260709 13:12:43.869552 28757 rpcz_store.cc:276] 0709 13:12:42.617063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.621700 (+  4637us) service_pool.cc:224] Handling call
0709 13:12:43.868771 (+1247071us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b5400722a20. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f4520 00007f59ce671652 00007f59d363f60a 00007f59ccbc57d4 00007f59ccbc5762 00007f59ccbc5691 00007f59ccbc560c 00007f59ccbc48c2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:43.869048 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75052928}
I20260709 13:12:43.869660 31713 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:43.812268 31801 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:43.871598 32187 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:43.872586 31936 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.872911 32197 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:43.873179 31936 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:43.873561 32197 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.876071 32197 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:43.877046 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:43.867976 31815 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:43.878058 31815 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:43.878443 31815 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:43.878648 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:43.881040 31801 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:43.881743 31801 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:43.799934 32111 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:43.888963 31930 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.015s	user 0.013s	sys 0.001s
I20260709 13:12:43.889518 31930 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.892014 31930 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:43.842507 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 157598us
I20260709 13:12:43.895007 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 821) took 1460 ms. Trace:
W20260709 13:12:43.895921 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.898842 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:43.897975 28083 rpcz_store.cc:276] 0709 13:12:42.434926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.435146 (+   220us) service_pool.cc:224] Handling call
0709 13:12:42.435452 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.581083 (+145631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.894758 (+1313675us) spinlock_profiling.cc:243] Waited 55.8 ms on lock 0x7b54002daad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:43.894945 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128390016}
W20260709 13:12:43.838634 32189 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:12:43.902846 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:43.905812 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 266965us
I20260709 13:12:43.880589 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:43.906589 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:43.903789 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 212297us
I20260709 13:12:43.907445 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:43.910384 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.905304 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:43.923599 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.809329 31805 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.928985 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:43.893447 31936 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:43.847429 32173 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:43.935150 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:43.936365 28759 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:43.904136 32069 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:43.909430 32089 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:43.959807 28760 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: []
W20260709 13:12:43.927235 32013 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.011s	sys 0.016s
I20260709 13:12:43.959129 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:43.964529 28761 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:43.926950 32187 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:43.761672 32188 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:43.840646 32191 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:43.937530 31801 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:43.844588 32113 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:44.006421 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 81164.8us
W20260709 13:12:44.019502 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 88992.7us
W20260709 13:12:43.967648 32013 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.012s	sys 0.016s
I20260709 13:12:43.946791 32069 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.073915 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:43.946746 32157 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:44.080922 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:44.068636 32191 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:44.084563 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:44.083549 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:44.083037 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:44.088506 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:44.067919 31930 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:44.014528 32173 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:44.016422 32183 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:44.080489 32069 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:44.092041 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 186809us
I20260709 13:12:44.070477 32166 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.090530 28199 leader_election.cc:400] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11.
I20260709 13:12:44.153184 28199 leader_election.cc:403] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:12:44.066370 31801 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:44.100225 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 200246us
I20260709 13:12:44.100004 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:44.183295 32182 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.871s	user 0.007s	sys 0.008s
I20260709 13:12:44.192240 32182 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.169628 31801 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:44.106659 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:44.107256 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 172202us
I20260709 13:12:43.895627 32176 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:44.213572 32176 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:44.113021 32205 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.071414 32111 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:12:44.215078 32111 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:44.214892 32205 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.215710 32111 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:44.116042 32194 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:12:44.088982 32089 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:44.175104 31930 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:44.226260 32201 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 13:12:44.134645 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 826) took 1240 ms. Trace:
I20260709 13:12:44.127292 32161 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:44.234140 28235 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:44.016141 31662 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:44.212308 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 105883us
I20260709 13:12:44.074195 32187 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:12:44.220917 32223 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:44.209843 32202 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:44.222209 32045 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:44.222541 32111 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:44.217003 31155 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:12:44.224884 32200 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:44.244499 28127 connection.cc:256] RPC call timeout handler was delayed by -1.0074953200000891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2985s for a response
I20260709 13:12:44.225860 31930 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:44.227586 28081 rpcz_store.cc:276] 0709 13:12:42.894308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.114953 (+220645us) service_pool.cc:224] Handling call
0709 13:12:43.115395 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:43.115644 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:43.115675 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:43.159916 (+ 44241us) log.cc:844] Serialized 26 byte log entry
0709 13:12:43.724019 (+564103us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:43.724063 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.724131 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:44.134404 (+410273us) raft_consensus.cc:1685] finished
0709 13:12:44.134431 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:44.134580 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":563767,"threads_started":1,"wal-append.queue_time_us":916197}
I20260709 13:12:44.142210 32113 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:44.147598 32159 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:44.170323 32166 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:44.240567 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:44.240834 28084 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:44.205057 32013 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.246133 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.246958 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:44.249483 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.261909 28083 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:44.225174 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.264178 28157 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:12:44.270675 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:44.261265 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47915.3us
I20260709 13:12:44.277150 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.278609 28081 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:44.267491 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:44.215788 31815 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:44.104945 32173 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:44.298007 28080 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:44.300443 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:44.254192 32187 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:44.266839 32223 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11.
I20260709 13:12:44.303951 28156 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:12:44.321063 28156 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:44.292651 32033 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.237953 32183 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:44.304942 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.306339 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:44.322978 28158 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:44.236459 32089 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.300014 32113 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:44.240307 31662 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.333204 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.333959 28084 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:44.335067 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.335668 28084 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:44.338773 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2294) took 1801 ms. Trace:
I20260709 13:12:44.339100 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.341035 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:44.342149 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 251768us
I20260709 13:12:44.343099 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:44.339407 28155 rpcz_store.cc:276] 0709 13:12:42.536932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.615484 (+ 78552us) service_pool.cc:224] Handling call
0709 13:12:42.615947 (+   463us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:43.056959 (+441012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:43.057003 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:43.100619 (+ 43616us) log.cc:844] Serialized 52 byte log entry
0709 13:12:43.723139 (+622520us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:43.723190 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.723263 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:44.338488 (+615225us) raft_consensus.cc:1685] finished
0709 13:12:44.338520 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:44.338699 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":298072,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"thread_start_us":947535,"threads_started":2,"wal-append.queue_time_us":767184}
I20260709 13:12:44.294106 32013 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.349824 28156 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.364306 28156 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:12:44.366817 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:44.254465 32202 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.385545 28084 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
W20260709 13:12:44.391134 28127 leader_election.cc:336] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.539s (SENT)
I20260709 13:12:44.391897 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:44.392181 28199 leader_election.cc:400] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11.
I20260709 13:12:44.392693 28199 leader_election.cc:403] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:12:44.345464 32113 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.397924 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:44.217259 32168 consensus_queue.cc:1175] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "11f10c12c03e47dcac88b050d4c1431b" responder_term: 7 status { last_received { term: 4 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: [6.5-6.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:44.332544 32033 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.398801 28227 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:44.400213 32183 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:44.402813 32195 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:44.403158 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:44.383208 32111 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:12:44.412546 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:44.401830 32204 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
W20260709 13:12:44.419104 32211 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:12:44.419950 32207 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:12:44.420820 32173 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:44.451618 28156 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:44.421903 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:44.422665 32219 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:44.423163 32181 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:44.423616 32227 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:44.424017 32228 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:44.425432 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.395649 32223 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:44.426283 28228 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:44.426635 32195 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.418809 32166 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:44.433844 28127 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.549s (SENT)
I20260709 13:12:44.387744 32213 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:44.424788 32198 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:44.424126 32069 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:44.449741 32231 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.421084 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:44.456140 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.501649 28229 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 6.
I20260709 13:12:44.502553 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.503664 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:44.426541 32071 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.002s
I20260709 13:12:44.457161 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.459654 32182 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:44.477286 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.414834 32157 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:44.421933 31815 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:44.479552 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:44.525655 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 91862.5us
W20260709 13:12:44.491420 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 190761us
I20260709 13:12:44.475490 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:44.495185 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.529971 28228 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:44.479655 28227 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:44.531474 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.459077 32227 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:44.506742 28234 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:44.504612 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.498831 32191 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:44.470116 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 94229.6us
I20260709 13:12:44.503687 28083 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:12:44.470640 32161 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:44.513834 28226 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:44.537874 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.538416 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.515348 28233 raft_consensus.cc:2368] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9.
I20260709 13:12:44.477614 32181 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:44.538473 28229 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:44.541857 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.541606 32181 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:44.542522 28229 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:44.543993 32225 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:44.491109 32195 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:44.544585 32225 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.539206 28226 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:44.518014 31930 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:44.535830 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.555086 28228 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:44.536898 28127 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.747s (SENT)
I20260709 13:12:44.534449 32111 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:12:44.532130 28227 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:44.548642 28226 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:12:44.548242 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.519037 32157 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:44.558945 32202 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:44.556758 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:44.572459 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.572991 32157 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:44.573594 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:44.559160 32229 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:12:44.542384 32191 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:44.574584 32227 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.576380 32227 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:44.557328 28127 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.757s (SENT)
W20260709 13:12:44.584621 28127 leader_election.cc:336] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.932s (SENT)
I20260709 13:12:44.531868 32089 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:44.549568 32113 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:44.579993 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.581965 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.497596 32198 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:44.585829 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:44.586527 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:44.478224 32228 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.590286 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.598595 28227 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:44.599383 28083 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:44.593333 32135 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:44.597110 28226 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:44.602672 28228 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:44.605679 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:44.606488 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:44.608999 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:44.610675 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:44.588999 32232 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:12:44.607719 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:44.615087 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.621528 28228 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:44.620036 28227 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:44.613255 32157 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:44.625016 32157 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.625489 32157 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:44.608103 32106 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:44.630089 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:44.630698 28226 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:44.620393 31930 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:44.631563 31930 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:44.631978 31930 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:44.633649 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.637460 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.638298 28228 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:44.580691 32205 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:44.639462 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:44.640247 28228 raft_consensus.cc:2376] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12. Re-sending same reply.
I20260709 13:12:44.640951 32240 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:44.641530 32240 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.643874 31805 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:44.647236 32240 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:44.647888 31754 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:44.649681 31662 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:44.652315 32236 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:44.621397 32161 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:44.655700 31957 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:44.656335 31957 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:44.657334 32172 raft_consensus.cc:993] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:12:44.657925 32172 raft_consensus.cc:1081] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:12:44.664216 28084 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:44.664635 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:44.666513 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:44.667505 28228 raft_consensus.cc:2376] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8. Re-sending same reply.
I20260709 13:12:44.668157 31957 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.674360 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:44.674926 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:44.675439 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:44.675499 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:44.676282 28156 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:44.680333 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:44.681713 28154 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:44.532866 32231 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:44.689319 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:44.690073 28154 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:44.681658 28232 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:44.605383 32013 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:44.684937 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:44.683686 28227 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:44.614048 32111 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 13:12:45.069749 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1864) took 2196 ms. Trace:
W20260709 13:12:44.703661 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 137161us
I20260709 13:12:44.692175 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.208014 28160 raft_consensus.cc:2410] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:12:44.759256 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.206461 28078 rpcz_store.cc:276] 0709 13:12:42.825732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.917073 (+ 91341us) service_pool.cc:224] Handling call
0709 13:12:42.917476 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.991505 (+ 74029us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.991542 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:42.991592 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:42.991629 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.021759 (+2030130us) spinlock_profiling.cc:243] Waited 73.8 ms on lock 0x7b54003a98d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:45.021868 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.022103 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169729792}
W20260709 13:12:45.206503 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.866020 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:45.210933 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.732717 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.211786 32135 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:45.212386 28229 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:12:45.212879 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 346961us
I20260709 13:12:45.215103 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:45.215849 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:44.804373 28226 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:45.217414 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:44.682669 32069 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:44.704640 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:45.218802 32069 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.213904 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:45.233194 28084 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:45.234819 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:44.833669 31801 consensus_meta.cc:287] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.606s	user 0.006s	sys 0.006s
I20260709 13:12:45.219563 32135 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:45.207942 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:44.811175 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:45.039887 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:44.733557 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:44.826617 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:44.803303 28156 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:12:44.687203 32106 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:45.213930 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:45.252391 28081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.007s	sys 0.022s
I20260709 13:12:45.252722 28228 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:45.253140 28081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.008s	sys 0.022s
I20260709 13:12:44.734754 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:45.227322 31930 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.595s	user 0.006s	sys 0.005s
I20260709 13:12:45.206413 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.225875 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:45.254244 28755 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:45.263623 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:45.264720 28228 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:45.211786 28078 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
W20260709 13:12:44.717013 32176 consensus_meta.cc:287] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.503s	user 0.003s	sys 0.007s
I20260709 13:12:45.249615 28153 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
W20260709 13:12:45.268446 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 60564.3us
I20260709 13:12:45.283273 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:45.259016 32106 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:45.290941 32106 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:45.291720 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:44.722380 32198 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:45.266539 32181 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.000s	sys 0.014s
I20260709 13:12:45.294909 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:45.295655 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:45.306402 28232 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:45.293727 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:45.296082 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:45.307157 28084 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:45.307747 28086 raft_consensus.cc:2410] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:12:45.206722 32228 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.309578 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.310439 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.298604 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 756) took 2404 ms. Trace:
I20260709 13:12:45.167946 32188 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.176605 31788 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:44.865056 32021 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:45.297086 32158 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:45.294952 32157 consensus_meta.cc:287] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.005s
I20260709 13:12:45.315150 28231 rpcz_store.cc:276] 0709 13:12:42.887666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.123464 (+235798us) service_pool.cc:224] Handling call
0709 13:12:43.148586 (+ 25122us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:43.277355 (+128769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:43.277403 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:43.277965 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:12:43.709808 (+431843us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:43.709851 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.248502 (+1538651us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54000951d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:45.248557 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:45.248608 (+    51us) raft_consensus.cc:1685] finished
0709 13:12:45.248627 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.291805 (+ 43178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19044,"raft.queue_time_us":83017,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":17882,"spinlock_wait_cycles":271118080,"thread_start_us":400669,"threads_started":1,"wal-append.queue_time_us":601616,"wal-append.run_cpu_time_us":2868,"wal-append.run_wall_time_us":1304073}
W20260709 13:12:45.305189 32181 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.000s	sys 0.015s
I20260709 13:12:45.308447 28161 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:12:45.336537 28161 pending_rounds.cc:85] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 4
I20260709 13:12:45.336874 28161 pending_rounds.cc:107] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:12:45.337167 28161 raft_consensus.cc:2889] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:45.326570 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:45.330171 28756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 920) took 2464 ms (client timeout 1835 ms). Trace:
I20260709 13:12:45.332691 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:45.337910 28756 rpcz_store.cc:269] 0709 13:12:42.857087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.857453 (+   366us) service_pool.cc:224] Handling call
0709 13:12:45.309145 (+2451692us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400436554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:45.321071 (+ 11926us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283572480}
I20260709 13:12:45.308158 32198 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.259238 31801 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.266985 32069 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.337970 28755 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:45.286849 31930 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:45.342340 28199 connection.cc:256] RPC call timeout handler was delayed by -1.5598759619999782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:12:45.329465 32158 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:45.320992 32231 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:45.344568 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:45.348534 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0676957969997147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
I20260709 13:12:45.352643 28153 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
W20260709 13:12:45.353006 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0010957969998344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2169s for a response
I20260709 13:12:45.342331 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 753) took 2721 ms. Trace:
I20260709 13:12:45.353856 28230 rpcz_store.cc:276] 0709 13:12:42.620677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.674167 (+ 53490us) service_pool.cc:224] Handling call
0709 13:12:42.689657 (+ 15490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.732254 (+ 42597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.732287 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:42.732331 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:42.732366 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.342090 (+2609724us) spinlock_profiling.cc:243] Waited 42.4 ms on lock 0x7b5400306454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:45.342136 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.342272 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97457536}
I20260709 13:12:45.321674 32228 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:45.335458 32188 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:45.356616 28755 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:45.358356 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.360035 28156 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:45.361828 28154 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:45.344165 32111 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.330698 31788 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.352211 32158 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:45.331110 32021 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.330336 32243 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:12:45.340130 32157 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.398051 28158 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:45.400417 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 724) took 2939 ms. Trace:
I20260709 13:12:45.403014 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.401516 28152 rpcz_store.cc:276] 0709 13:12:42.454399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.581982 (+127583us) service_pool.cc:224] Handling call
0709 13:12:42.582267 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:42.582484 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:45.208035 (+2625551us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b18001b6400. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf
0709 13:12:45.208196 (+   161us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.393118 (+184922us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.393181 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:45.393235 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.393729 (+   494us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.393888 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":224022,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":3716,"spinlock_wait_cycles":38721792,"thread_start_us":432964,"threads_started":2}
I20260709 13:12:45.406025 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.403680 32248 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:45.407907 31930 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:45.407656 32248 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.377614 32111 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.410948 32248 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.413688 28153 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:45.346659 32198 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.418867 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:45.419575 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.420193 28159 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:45.421428 32021 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.393522 32222 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:45.423348 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.400914 32250 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.368678 32188 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.365083 32228 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.436772 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:45.429936 32099 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:45.419589 31788 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.416848 32248 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.446224 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:12:45.433781 32022 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:12:45.481524 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:45.443425 32227 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.847s	user 0.006s	sys 0.007s
I20260709 13:12:45.425254 32176 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:45.449923 31930 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:45.535233 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:45.532615 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 86466.7us
I20260709 13:12:45.536422 28226 raft_consensus.cc:2410] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:12:45.534590 32249 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:45.449774 32099 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.537428 32069 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.538471 31801 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:45.539129 32099 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:45.542271 32106 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.537374 32249 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:45.547173 32249 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:45.549023 32157 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:45.531810 32248 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.551539 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:45.552237 28153 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:45.535346 31930 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:45.553306 31930 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:45.557713 32249 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.561192 28160 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:45.563993 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.566464 28153 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
W20260709 13:12:45.567495 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.567576 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:45.565419 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:45.571717 28755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:12:45.572549 28755 raft_consensus.cc:606] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:45.572992 28755 raft_consensus.cc:614] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:45.434577 32222 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.442608 32250 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.579169 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:45.567066 32183 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.139s	user 0.009s	sys 0.005s
I20260709 13:12:45.570747 27973 maintenance_manager.cc:419] P fea5cdb8fcad442a901284de409f4a11: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033366
I20260709 13:12:45.582175 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.565160 32227 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:45.569327 32069 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.587095 28153 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:45.574337 32157 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:45.575352 32106 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.591194 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:45.596238 28084 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:45.563241 32176 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.598572 32188 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:45.606161 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 123055us
I20260709 13:12:45.611878 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.613373 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:45.586771 32183 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11.
I20260709 13:12:45.613351 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:45.612466 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:45.618786 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.590709 32022 raft_consensus.cc:2764] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:12:45.606557 32195 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.061s	user 0.011s	sys 0.003s
I20260709 13:12:45.622671 28049 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:45.622247 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:45.626966 31722 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:45.583585 31801 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:12:45.638489 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:45.596746 32157 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:45.628885 28161 raft_consensus.cc:2368] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12.
I20260709 13:12:45.649062 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 927) took 1418 ms. Trace:
I20260709 13:12:45.649514 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.633271 32198 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:12:45.632413 32248 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.640439 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:45.638397 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:45.642875 28153 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:45.645170 32023 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:45.645434 31788 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:45.639016 31722 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:45.646008 28082 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2228) took 2955 ms (client timeout 2169 ms). Trace:
I20260709 13:12:45.647003 32228 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:45.647082 32022 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.590798 32069 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.637005 32188 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:45.747726 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 109383us
I20260709 13:12:45.649452 28760 rpcz_store.cc:276] 0709 13:12:44.231159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.231348 (+   189us) service_pool.cc:224] Handling call
0709 13:12:44.234064 (+  2716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.234322 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:44.248212 (+ 13890us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.648233 (+1400021us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b18004c3960. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a
0709 13:12:45.648436 (+   203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.648489 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:45.648533 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.648593 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.649005 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":668,"prepare.queue_time_us":15349,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":7089,"spinlock_wait_cycles":2699520,"thread_start_us":49714,"threads_started":1,"wal-append.queue_time_us":136620}
I20260709 13:12:45.605635 32227 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:45.750300 28082 rpcz_store.cc:269] 0709 13:12:42.665548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.666072 (+   524us) service_pool.cc:224] Handling call
0709 13:12:45.613212 (+2947140us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552e9c 00007f59cb552a5b 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:12:45.620431 (+  7219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2922624}
I20260709 13:12:45.750619 28161 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:45.661849 32157 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:45.794523 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:45.795161 28161 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
W20260709 13:12:45.753590 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 113263us
I20260709 13:12:45.751673 31722 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:45.801793 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.803222 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.751852 32248 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:45.804416 32248 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:45.804816 32248 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:45.743608 32183 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:45.805665 32183 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:45.806056 32183 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:45.806313 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:45.806492 32157 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.661283 32161 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:45.757687 32022 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.810977 32022 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.757874 32023 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:45.811733 32023 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:45.759022 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:45.812134 32023 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:45.812445 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53487.6us
I20260709 13:12:45.814056 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:45.814865 32255 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:45.815261 32255 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:45.815610 32255 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:45.815653 31930 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.817047 32228 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:45.796939 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2309) took 1104 ms. Trace:
I20260709 13:12:45.818423 32228 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:45.820482 32069 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:45.818207 28161 rpcz_store.cc:276] 0709 13:12:44.692539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.796082 (+1103543us) service_pool.cc:224] Handling call
0709 13:12:45.796410 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.796591 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.796618 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.796665 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:45.796696 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.796777 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.796885 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.822328 28230 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:45.823886 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:45.824486 28230 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:45.822266 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2310) took 1113 ms. Trace:
I20260709 13:12:45.825934 32250 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:45.751921 32243 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:45.826159 28161 rpcz_store.cc:276] 0709 13:12:44.709090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.821494 (+1112404us) service_pool.cc:224] Handling call
0709 13:12:45.821780 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.821953 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.821975 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:45.822018 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:45.822049 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.822106 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:45.822214 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.826895 32243 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.827651 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:45.828297 28230 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:45.830406 31801 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:45.830958 31801 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.832667 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1940) took 1199 ms. Trace:
I20260709 13:12:45.833205 28227 rpcz_store.cc:276] 0709 13:12:44.633243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.633389 (+   146us) service_pool.cc:224] Handling call
0709 13:12:45.780590 (+1147201us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b5400355cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:45.832560 (+ 51970us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113498368}
I20260709 13:12:45.772984 28759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6810464df4a245f49df908fc877020d7"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:12:45.834820 28759 raft_consensus.cc:606] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:45.781591 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2318) took 1058 ms. Trace:
I20260709 13:12:45.835759 28160 rpcz_store.cc:276] 0709 13:12:44.723030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.291494 (+568464us) service_pool.cc:224] Handling call
0709 13:12:45.781325 (+489831us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54003430d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:45.781502 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576403200}
I20260709 13:12:45.778414 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2306) took 1087 ms. Trace:
I20260709 13:12:45.837108 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:45.789426 32163 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 2.676s	user 0.017s	sys 0.019s
W20260709 13:12:45.785188 32153 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 2.809s	user 0.009s	sys 0.027s
I20260709 13:12:45.811340 32158 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.846683 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:45.849639 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:45.857299 28227 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 7.
I20260709 13:12:45.852236 28228 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:45.837335 28154 rpcz_store.cc:276] 0709 13:12:44.691421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.699357 (+  7936us) service_pool.cc:224] Handling call
0709 13:12:45.778121 (+1078764us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54003773d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:45.778327 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234351872}
I20260709 13:12:45.795059 32227 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.861020 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.868137 28756 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:45.861330 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.862668 28228 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:45.869118 28762 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:45.825941 32023 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.873669 32183 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.875213 32023 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.851212 32250 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:45.883514 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.865796 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:45.879459 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:45.882963 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 760) took 1799 ms. Trace:
I20260709 13:12:45.885545 28235 rpcz_store.cc:276] 0709 13:12:44.085684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.085858 (+   174us) service_pool.cc:224] Handling call
0709 13:12:44.086266 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.234659 (+148393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.882727 (+1648068us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54000de3d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:45.882906 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339571968}
I20260709 13:12:45.886134 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.650372 32195 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:45.888362 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:45.864228 28154 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:45.832058 32255 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.890468 28763 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:45.897027 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2313) took 1176 ms. Trace:
I20260709 13:12:45.896988 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.897678 28154 rpcz_store.cc:276] 0709 13:12:44.720249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.863706 (+1143457us) service_pool.cc:224] Handling call
0709 13:12:45.864055 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.896502 (+ 32447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.896695 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.891470 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.898954 28759 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:45.899142 28154 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:45.899935 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2316) took 1177 ms. Trace:
I20260709 13:12:45.894968 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.900353 28154 rpcz_store.cc:276] 0709 13:12:44.722516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.898514 (+1175998us) service_pool.cc:224] Handling call
0709 13:12:45.898795 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.899710 (+   915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.899884 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:45.876066 32183 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:45.904788 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:45.909751 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.881567 32222 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.878316 31930 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:45.923882 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 309484us
I20260709 13:12:45.924685 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:45.926623 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:45.947417 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:45.954892 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.881675 32250 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:45.954917 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:45.958963 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:45.953298 32200 consensus_meta.cc:287] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 1.678s	user 0.012s	sys 0.004s
I20260709 13:12:45.945457 32222 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.963975 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:45.957170 28198 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.196:46855 timed out after 1.798s (SENT) [suppressed 21 similar messages]
I20260709 13:12:45.971475 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:45.965955 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 54996.7us
I20260709 13:12:45.964923 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:45.971618 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:45.950695 32243 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:45.973495 28762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:12:45.978721 28762 raft_consensus.cc:606] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Received request to transfer leadership to TS 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:45.979256 28762 raft_consensus.cc:614] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:45.949075 31957 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:45.975067 28228 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:45.983268 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:45.986951 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:45.956439 32157 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.990396 32033 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:45.989822 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:45.996907 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:45.975059 32249 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.977552 32139 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:45.976483 32096 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.005214 28198 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:45.991204 31801 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:45.992106 32183 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:45.996786 32176 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:45.989929 31850 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:12:46.027515 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:45.986122 32222 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:46.028491 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 196546us
I20260709 13:12:46.015939 32252 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:46.031276 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:46.036998 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:46.018743 32227 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:46.037859 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:46.022857 31930 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:46.037705 28764 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:46.040433 28198 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:46.022606 32257 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:46.022960 32161 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:46.028798 32183 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:46.033838 32264 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:46.039806 32200 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:46.037148 32252 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:46.045405 32227 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:46.042451 31801 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:46.020098 32096 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.042784 32176 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.074721 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:46.084327 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 101191us
I20260709 13:12:46.085403 28227 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:46.075270 28764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:12:46.075872 28232 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:12:46.086102 28764 raft_consensus.cc:606] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:46.070122 32069 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:46.032979 32138 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:12:46.081152 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:46.042665 32260 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:46.059271 32257 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:46.036631 32139 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:12:46.067230 32183 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:46.017123 31722 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.087656 28227 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:46.088450 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:46.088824 28232 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:12:46.079867 32252 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:46.092129 32195 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:46.075690 32200 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:46.101091 28231 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:46.102205 32257 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:46.134584 32188 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:46.134934 32248 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.136570 28230 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:46.101415 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:46.082213 32227 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:46.111833 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:46.142868 28227 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:46.139672 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:46.097817 32222 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.112390 32138 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:46.115298 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:46.098243 32176 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.116230 31722 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.147208 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:46.148010 28231 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:46.148610 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.149782 28230 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:12:46.152110 28227 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:46.141731 28228 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:46.142386 32250 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:46.121860 32161 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.140152 32200 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:46.164608 28052 connection.cc:256] RPC call timeout handler was delayed by -1.8829956979998315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
W20260709 13:12:46.165292 28052 connection.cc:256] RPC call timeout handler was delayed by -1.8784956979998242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 13:12:46.170898 28052 connection.cc:256] RPC call timeout handler was delayed by -1.0355334599998969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
I20260709 13:12:46.173422 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:46.174729 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:46.182878 32248 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:46.179762 32195 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.183873 32195 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.186025 32195 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.186513 32195 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:46.187048 32195 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.178066 32109 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:46.188862 32109 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.190697 32023 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:46.190999 32109 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:46.191250 32023 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.181051 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:46.167152 32250 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:46.217741 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:46.218348 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:46.164301 32139 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:12:46.181497 32198 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:46.182075 32188 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:46.178501 32021 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:46.235399 32248 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:46.155702 32138 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.192170 32257 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.255939 32198 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.280814 32277 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:46.282567 28152 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:46.219550 32250 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.287261 32069 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.283253 32248 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:46.283707 31850 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:12:46.308310 28231 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:46.319998 31850 consensus_queue.cc:1243] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:46.291872 32198 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:46.272933 32021 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.308919 32023 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.282011 32183 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.347277 28086 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:46.349803 32138 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.339246 32109 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:46.350734 32275 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:12:46.351102 32109 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:46.352272 32271 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:12:46.353391 28082 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:12:46.353547 32109 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:46.354140 32109 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.354619 32109 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.356222 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.357070 28079 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:46.358639 28079 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:46.360242 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.362251 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.363564 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.356609 32109 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.305704 32277 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.382578 28227 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:46.387408 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:46.391634 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 687) took 2458 ms. Trace:
I20260709 13:12:46.392098 28761 rpcz_store.cc:276] 0709 13:12:43.927732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.929009 (+  1277us) service_pool.cc:224] Handling call
0709 13:12:43.929281 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.965131 (+ 35850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.386194 (+2421063us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b54006504d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:46.386370 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80410880}
I20260709 13:12:46.394377 28760 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:46.373709 32188 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.397194 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.399067 28050 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:46.388571 32111 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.400007 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:46.400566 32195 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:46.402654 32183 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:46.403038 32143 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:46.404233 32176 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:46.405229 28077 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 8: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:46.388931 32225 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.406562 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 1923) took 1942 ms. Trace:
I20260709 13:12:46.407047 32188 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.329473 32069 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.416278 28229 rpcz_store.cc:276] 0709 13:12:44.473342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.543365 (+ 70023us) service_pool.cc:224] Handling call
0709 13:12:45.211987 (+668622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.212881 (+   894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.406347 (+1193466us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54003bfbd0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:46.406497 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1536803072}
I20260709 13:12:46.398218 32109 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.398017 32023 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:46.418885 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 123576us
W20260709 13:12:46.420234 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 320828us
I20260709 13:12:46.427664 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:46.422920 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:46.428432 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.428576 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:46.429579 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:46.430253 28760 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:46.433090 28077 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:46.387246 32252 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:46.435707 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 263451us
I20260709 13:12:46.441133 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:46.452482 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.454114 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.454517 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.456847 28084 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:36768
I20260709 13:12:46.457605 28084 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:46.458058 28084 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:46.459127 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 691) took 1133 ms. Trace:
I20260709 13:12:46.459544 28763 rpcz_store.cc:276] 0709 13:12:45.325776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.325953 (+   177us) service_pool.cc:224] Handling call
0709 13:12:46.458935 (+1132982us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54007693d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:46.459070 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1269648256}
I20260709 13:12:46.461164 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.461845 28078 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:46.462893 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.463444 28078 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:46.464195 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:46.465248 28234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1918) took 2051 ms (client timeout 2082 ms). Trace:
I20260709 13:12:46.465890 32277 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:46.465693 28234 rpcz_store.cc:269] 0709 13:12:44.413639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.415669 (+  2030us) service_pool.cc:224] Handling call
0709 13:12:46.465085 (+2049416us) spinlock_profiling.cc:243] Waited 898 us on lock 0x7b5400288054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:46.465189 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2064768}
I20260709 13:12:46.398233 32248 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.468952 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.467458 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.470263 28079 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:46.461879 32255 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:46.424976 32195 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:46.467535 28084 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.480643 28077 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:46.484644 28230 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:46.485931 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.505107 28079 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
W20260709 13:12:46.505344 28080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1877) took 2282 ms (client timeout 1737 ms). Trace:
I20260709 13:12:46.505203 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.507270 28083 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:46.454996 32023 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:46.512820 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:46.513690 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:46.505896 28080 rpcz_store.cc:269] 0709 13:12:44.222610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.227858 (+  5248us) service_pool.cc:224] Handling call
0709 13:12:46.505121 (+2277263us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b54001adbd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:46.505270 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67098496}
I20260709 13:12:46.424474 32109 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.508663 28052 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:46.510646 28084 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:46.479645 32257 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.482067 32250 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:46.481472 32195 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.520926 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.521723 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.521952 28083 raft_consensus.cc:2410] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:12:46.522383 28079 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:46.527096 28228 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:46.531857 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.533767 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.536957 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.537671 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.536250 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.541747 32292 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:46.537377 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.538877 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:46.525503 32023 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:46.515515 32284 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:12:46.548022 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.522151 32195 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:46.537734 32234 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:46.537905 32206 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:46.526081 32033 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:12:46.533403 32139 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:12:46.566416 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.552048 32021 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:46.526818 32281 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:12:46.573917 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.559988 32234 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.560010 32206 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:46.536449 32291 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:12:46.562966 32248 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.567049 32289 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:46.580832 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.582746 28079 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:46.579087 32200 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.591259 32200 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.592125 28764 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:46.559836 32288 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:12:46.596067 32200 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:46.596613 32200 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:46.597676 32200 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.576016 32109 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.601359 32227 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.602608 28235 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:46.578774 32234 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:46.605037 28158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2301) took 1931 ms (client timeout 2061 ms). Trace:
I20260709 13:12:46.583027 32188 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:46.605502 28158 rpcz_store.cc:269] 0709 13:12:44.673935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.674061 (+   126us) service_pool.cc:224] Handling call
0709 13:12:46.604799 (+1930738us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b54000a37d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:46.604973 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1662099840}
I20260709 13:12:46.607255 28153 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:46.579442 32206 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:46.613487 32069 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:46.580806 32248 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.611534 32188 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:46.631026 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.638878 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.639705 28079 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
W20260709 13:12:46.643210 32247 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.295s	user 0.000s	sys 0.003s
I20260709 13:12:46.648948 27983 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 5 to 7, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 7 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:46.650931 28758 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
W20260709 13:12:46.651788 32270 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.536s	user 0.002s	sys 0.000s
I20260709 13:12:46.651152 27983 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 7 to 8, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:12:46.652717 28758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35896 (request call id 878) took 2573 ms (client timeout 1798 ms). Trace:
W20260709 13:12:46.653182 28758 rpcz_store.cc:269] 0709 13:12:44.079512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.079748 (+   236us) service_pool.cc:224] Handling call
0709 13:12:46.652519 (+2572771us) spinlock_profiling.cc:243] Waited 2.46 s on lock 0x7b54001b9754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:46.652643 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5648037632}
I20260709 13:12:46.653785 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 739) took 1221 ms. Trace:
I20260709 13:12:46.654129 32277 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:46.654706 32277 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:46.654189 28159 rpcz_store.cc:276] 0709 13:12:45.432660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.945661 (+513001us) service_pool.cc:224] Handling call
0709 13:12:45.957675 (+ 12014us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:45.958070 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:45.958097 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:45.973056 (+ 14959us) log.cc:844] Serialized 52 byte log entry
0709 13:12:46.050990 (+ 77934us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:46.051039 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.051113 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.651180 (+600067us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:46.653471 (+  2291us) raft_consensus.cc:1685] finished
0709 13:12:46.653494 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.653726 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":68472,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":6151,"spinlock_wait_cycles":2926464,"thread_start_us":36978,"threads_started":1,"wal-append.queue_time_us":118878}
I20260709 13:12:46.655756 32277 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.654196 32003 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.657495 32003 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.659173 32277 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:46.662509 32096 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.664904 32198 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.667192 32174 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:46.667625 32198 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:46.676576 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 122150us
I20260709 13:12:46.699405 28227 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:46.700111 28235 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:46.636201 32290 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:12:46.708467 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 672872us
I20260709 13:12:46.722762 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:46.729794 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:46.730847 28235 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:46.733786 28758 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:46.706342 32096 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:12:46.708909 32195 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.740968 31930 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:46.742568 32195 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.744537 32260 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:46.742129 31930 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.757123 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2311) took 2037 ms. Trace:
I20260709 13:12:46.759222 28161 rpcz_store.cc:276] 0709 13:12:44.719608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.827166 (+1107558us) service_pool.cc:224] Handling call
0709 13:12:45.827611 (+   445us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:45.975723 (+148112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.976792 (+  1069us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:45.993372 (+ 16580us) log.cc:844] Serialized 52 byte log entry
0709 13:12:46.083214 (+ 89842us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:46.112870 (+ 29656us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.756784 (+643914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.756861 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.756906 (+    45us) raft_consensus.cc:1685] finished
0709 13:12:46.756920 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.757065 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":643622,"prepare.queue_time_us":41398,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":79792,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4682,"thread_start_us":207045,"threads_started":2,"wal-append.queue_time_us":141431}
I20260709 13:12:46.634733 32188 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:46.756659 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 668244us
I20260709 13:12:46.762797 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:46.762885 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:46.761148 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:46.765626 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:46.766113 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:46.766302 28227 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:46.766460 32198 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:46.767102 32069 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:46.767243 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:46.768122 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:46.771751 32250 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:46.772356 32250 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:46.772833 32069 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:46.773269 32069 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:46.781131 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:46.773510 32250 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:46.776145 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.745929 32260 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.790613 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:46.792213 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:46.792930 32159 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:46.793351 32159 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:46.793752 32159 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:46.796067 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.776619 32222 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:12:46.781071 32294 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:12:46.781977 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2252) took 1471 ms. Trace:
I20260709 13:12:46.789222 32158 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:46.790647 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.811517 28079 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:46.796878 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:46.797232 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.798200 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.812723 28080 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:46.799027 28082 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 13:12:46.799717 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.799880 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.799973 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:46.812252 28081 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:46.815918 28158 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 7.
I20260709 13:12:46.816529 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.816895 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:46.817539 28158 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:46.814380 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.800410 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:46.815296 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 52576.9us
I20260709 13:12:46.815596 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.821254 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:46.800683 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:46.802727 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:46.804250 28152 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:46.823446 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:46.807176 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:46.822666 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 285350us
I20260709 13:12:46.825002 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:46.825968 28156 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:46.809124 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:46.808080 28153 rpcz_store.cc:276] 0709 13:12:45.310102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.903443 (+593341us) service_pool.cc:224] Handling call
0709 13:12:46.606955 (+703512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:46.607901 (+   946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.781768 (+173867us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54000a37d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:46.781917 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594276096}
I20260709 13:12:46.824950 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:46.814482 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.828457 28084 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:46.787422 32069 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:46.820094 28077 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:12:46.831651 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2317) took 2108 ms. Trace:
I20260709 13:12:46.830206 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:46.830940 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.829523 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:46.831854 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:46.844062 28159 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:46.844851 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:46.845714 28152 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:12:46.805071 32198 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:46.852356 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:46.853183 28161 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
W20260709 13:12:46.848502 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 47423.3us
I20260709 13:12:46.846660 28156 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:46.849784 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:46.805405 32248 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:46.855156 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:46.855523 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 63376.9us
I20260709 13:12:46.842226 28154 rpcz_store.cc:276] 0709 13:12:44.722777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.901408 (+1178631us) service_pool.cc:224] Handling call
0709 13:12:45.901676 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.901870 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:46.035761 (+133891us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.831064 (+795303us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.831297 (+   233us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:46.831346 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:46.831392 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.831479 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.831591 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":690485,"prepare.queue_time_us":178953,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":660138,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":224916,"threads_started":3,"wal-append.queue_time_us":110947}
I20260709 13:12:46.849298 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:46.805938 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:46.854422 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:46.862782 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56931us
I20260709 13:12:46.858662 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.829312 32157 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:46.869757 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:46.864979 28152 raft_consensus.cc:2393] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:46.870350 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:46.861670 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:46.865399 28158 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:46.842095 32293 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:46.806492 32022 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:12:46.849082 32277 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:46.880868 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2312) took 2161 ms. Trace:
I20260709 13:12:46.850980 32198 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:46.861183 32023 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:46.880560 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:46.895164 28080 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:46.895879 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:46.897985 32190 raft_consensus.cc:993] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: : Instructing follower 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:12:46.866428 31722 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:46.881686 32021 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:46.919545 32282 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:46.919970 32021 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:46.866043 32003 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:46.877206 32157 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:46.921370 32157 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:46.919561 31722 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.922623 31957 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:12:46.922713 28226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1946) took 2287 ms (client timeout 2021 ms). Trace:
I20260709 13:12:46.923313 31722 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:46.923245 28226 rpcz_store.cc:269] 0709 13:12:44.635154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.217116 (+581962us) service_pool.cc:224] Handling call
0709 13:12:46.922516 (+1705400us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b5400305f54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:46.922646 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730937088}
I20260709 13:12:46.903481 32277 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:46.924640 32021 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:46.854789 32292 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:12:46.903784 32227 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:46.915623 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:46.928928 32227 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:46.929410 32227 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:46.929819 32227 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:46.931442 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61165.6us
I20260709 13:12:46.932930 32260 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:46.859923 32281 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:12:46.934271 32292 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:46.934649 32292 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:46.935465 32292 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.936672 32292 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:46.939909 32244 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.584s	user 0.000s	sys 0.004s
I20260709 13:12:46.932713 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:46.942878 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:46.943687 32257 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:46.944069 32257 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:46.944619 32257 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:46.945811 32222 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:46.849206 32158 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:12:46.911171 32200 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:46.859095 32291 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:46.863610 32255 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:46.913933 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:46.863862 32261 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:46.968542 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 929) took 2363 ms. Trace:
I20260709 13:12:46.969063 28757 rpcz_store.cc:276] 0709 13:12:44.605081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.606974 (+  1893us) service_pool.cc:224] Handling call
0709 13:12:44.623709 (+ 16735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:44.687291 (+ 63582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:44.687332 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:44.691459 (+  4127us) log.cc:844] Serialized 26 byte log entry
0709 13:12:44.957419 (+265960us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:44.957464 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.957548 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.939182 (+1981634us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:46.968231 (+ 29049us) raft_consensus.cc:1685] finished
0709 13:12:46.968258 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.968478 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98215,"raft.queue_time_us":68428,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":24767,"spinlock_wait_cycles":54098688,"thread_start_us":265519,"threads_started":1,"wal-append.queue_time_us":626067}
I20260709 13:12:46.867641 32168 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:46.971654 32168 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.972090 32168 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:46.978152 32032 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:46.978837 32032 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:46.918141 32183 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:46.982060 32183 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:46.982638 32183 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.985042 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:46.985652 28228 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:46.986550 32286 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:46.987099 32286 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:46.988561 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 111892us
I20260709 13:12:46.896571 32297 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:12:46.920043 32282 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:46.920913 28233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 1916) took 2509 ms (client timeout 1539 ms). Trace:
W20260709 13:12:47.001505 28233 rpcz_store.cc:269] 0709 13:12:44.411440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.411585 (+   145us) service_pool.cc:224] Handling call
0709 13:12:46.920642 (+2509057us) spinlock_profiling.cc:243] Waited 7.86 ms on lock 0x7b54001a0254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:46.920814 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18083712}
I20260709 13:12:46.902648 32248 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:46.925462 32277 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.933375 32260 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:46.907265 32159 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.947106 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:47.020175 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 63064.7us
I20260709 13:12:46.906734 32206 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:46.952863 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:46.881279 28160 rpcz_store.cc:276] 0709 13:12:44.719947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.836973 (+1117026us) service_pool.cc:224] Handling call
0709 13:12:45.884693 (+ 47720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:45.884925 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:12:45.960119 (+ 75194us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.674507 (+714388us) spinlock_profiling.cc:243] Waited 8.77 ms on lock 0x7b18003838c0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:12:46.871749 (+197242us) log.cc:844] Serialized 14 byte log entry
0709 13:12:46.880522 (+  8773us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:46.880576 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:46.880617 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:46.880682 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.880812 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111203,"prepare.queue_time_us":159528,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":6844,"raft.queue_time_us":689253,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":22470,"spinlock_wait_cycles":20166272,"thread_start_us":304066,"threads_started":3,"wal-append.queue_time_us":115900}
I20260709 13:12:46.960212 28235 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:46.943966 32195 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:46.923151 31957 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:47.034977 32195 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.035560 31957 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.008327 32258 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:47.036823 31957 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:47.037375 32188 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:46.915035 32198 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:46.932619 32252 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:47.010931 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 97075.8us
I20260709 13:12:46.950451 32264 raft_consensus.cc:2764] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:12:47.014915 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.015719 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:46.946252 32222 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:47.019515 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:47.020478 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73455.8us
I20260709 13:12:47.054090 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:47.055243 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:46.954918 32291 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:47.026127 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:47.027482 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:46.882457 32265 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.128s
I20260709 13:12:46.988217 32286 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:47.032128 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.028486 32158 consensus_queue.cc:1243] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:47.032996 32206 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:47.033625 28081 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:47.033532 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:47.034260 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:47.079874 28234 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:46.990036 32183 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:47.046378 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:47.047086 28086 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:47.015635 32190 raft_consensus.cc:1081] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Signalling peer 0faaace2b36c4bcebf2fa6835235e92b to start an election
I20260709 13:12:47.056469 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:46.882766 32293 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:46.956406 32234 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.064136 28230 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:47.058374 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:47.021696 32260 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:46.980062 32032 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:47.100425 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54155.6us
I20260709 13:12:47.083704 28080 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.104223 28125 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:47.060918 28232 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:47.044181 32261 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:47.085942 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:47.097663 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 147620us
I20260709 13:12:47.106719 28227 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:47.070447 32291 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.119056 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:47.108991 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:47.109944 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51647.5us
I20260709 13:12:47.107517 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:47.123374 28199 leader_election.cc:400] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12.
I20260709 13:12:47.123939 28199 leader_election.cc:403] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:47.125841 28158 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:47.130538 28080 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:47.075500 32259 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:12:47.144004 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:47.144744 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:47.095306 32292 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:12:47.113713 32022 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:47.144062 28232 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:47.152159 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:47.115916 32045 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:47.155536 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:47.099484 32281 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:12:47.156486 32191 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:12:47.152570 32250 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12.
I20260709 13:12:47.170020 32159 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:47.176894 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.075775 32282 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.139889 31803 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:47.177548 32205 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:47.178442 28730 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:47.212991 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 68339.3us
I20260709 13:12:47.180040 32293 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:47.143762 32195 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:47.181211 32257 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.158021 32265 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:47.187094 32234 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:47.133579 32183 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:47.150651 32032 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:47.244525 28227 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:47.195161 32227 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.203467 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.204478 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:47.205658 28154 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:47.246503 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.247459 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.247617 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:47.213299 31803 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.230387 32099 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:47.249271 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:47.214308 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:47.157148 32045 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:47.275950 32291 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.223729 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.286078 28156 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:47.287652 28156 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:47.225250 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:47.211984 32174 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.004s
I20260709 13:12:47.247714 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:47.209666 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:47.299160 28084 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:47.256829 28153 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:47.256584 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:47.258021 28159 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:47.217399 32205 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:47.268608 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54388.8us
I20260709 13:12:47.210523 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:47.242856 32168 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.334995 28158 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:47.278293 32257 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:47.218312 32021 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:12:47.249702 32265 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:47.131659 32222 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:47.381894 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48817.3us
I20260709 13:12:47.383054 28083 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
W20260709 13:12:47.388114 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.281610 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 230454us
W20260709 13:12:47.397855 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.219215 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:47.243001 32183 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.401010 28232 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:47.325183 28155 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:12:47.401587 28155 pending_rounds.cc:85] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 4
I20260709 13:12:47.320791 31826 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:47.401865 28155 pending_rounds.cc:107] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:12:47.402503 28155 raft_consensus.cc:2889] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:47.325701 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 243225us
W20260709 13:12:47.331215 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.311388 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:47.332114 32205 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.264012 31803 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:47.335597 32250 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:47.334893 28763 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:47.350314 32284 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:47.412170 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:47.285264 32045 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:47.297626 32198 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.264842 32069 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.392765 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:47.393615 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.396641 28152 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:47.218513 32157 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.431036 27983 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 8 to 9, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 9 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:47.407610 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:47.443087 28078 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.408299 28084 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6810464df4a245f49df908fc877020d7"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
 from {username='slave'} at 127.0.0.1:36768
I20260709 13:12:47.409205 28086 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 13:12:47.444135 28084 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:47.444554 28084 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:47.411046 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 191908us
I20260709 13:12:47.441375 28229 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:47.445266 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:47.376888 32022 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:47.401330 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.404917 32240 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.461561 28758 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:47.461457 32240 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.462915 32320 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.463783 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:47.463521 32320 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.427596 32196 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:47.465469 32196 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:47.404560 32183 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:47.467432 28728 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:47.358872 31788 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:12:47.427598 27981 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58554 (request call id 282) took 26421 ms (client timeout 15000 ms). Trace:
I20260709 13:12:47.429466 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:47.336864 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 126403us
I20260709 13:12:47.431607 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:47.473906 28227 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:47.424896 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.447259 28159 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:47.449236 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.451918 32168 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:47.452708 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:47.476854 28764 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:47.453233 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:47.478459 28760 raft_consensus.cc:2368] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12.
I20260709 13:12:47.454525 32158 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:47.485996 32206 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:47.486562 32206 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.486918 32206 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.487609 32069 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:47.428305 32188 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:47.457808 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:47.460327 32312 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:12:47.428254 32101 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:47.490445 32101 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.490952 32101 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:47.491515 28078 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:47.446233 28228 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:47.469275 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:47.472407 27983 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 5 to 7, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 7 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:47.390297 32252 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:12:47.502566 28157 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:47.426070 32284 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:47.514289 28161 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:47.516038 27983 catalog_manager.cc:5697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 3 to 9, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 9 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:47.521020 32157 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:47.525547 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:47.471700 31788 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:47.527029 28154 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:47.429596 31826 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.531040 32291 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:47.567580 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:47.497768 32158 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:47.488562 32069 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:12:47.508658 32200 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:47.506925 32268 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:47.517592 32198 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:47.569048 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.570628 28157 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:47.538118 28198 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:47.538053 32252 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:12:47.572629 28154 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.573565 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.662526 28156 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:47.574177 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.574474 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.683766 28153 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:47.572057 32258 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:47.576151 28159 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:47.521646 32265 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.581821 27983 catalog_manager.cc:5697] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:47.585070 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 199289us
W20260709 13:12:47.688181 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 184626us
I20260709 13:12:47.589040 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:47.594250 28127 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.021s (SENT)
I20260709 13:12:47.688515 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:47.689190 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:47.571166 28084 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.690457 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:47.471870 27981 rpcz_store.cc:269] 0709 13:12:20.987466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:21.009027 (+ 21561us) service_pool.cc:224] Handling call
0709 13:12:47.368669 (+26359642us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:37.408320 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:37.419932 (+ 11612us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:37.419971 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:12:37.419987 (+    16us) tablet.cc:662] Decoding operations
0709 13:12:37.439323 (+ 19336us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:37.439393 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:37.439423 (+    30us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:12:37.440647 (+  1224us) tablet.cc:701] Row locks acquired
0709 13:12:37.440663 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:12:37.440852 (+   189us) write_op.cc:270] Start()
0709 13:12:37.440987 (+   135us) write_op.cc:276] Timestamp: P: 1783602757440789 usec, L: 0
0709 13:12:37.441003 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:12:37.441556 (+   553us) log.cc:844] Serialized 5572 byte log entry
0709 13:12:38.045993 (+604437us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:38.729152 (+683159us) op_driver.cc:464] REPLICATION: finished
0709 13:12:39.448810 (+719658us) write_op.cc:301] APPLY: starting
0709 13:12:39.466872 (+ 18062us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:39.598162 (+131290us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:39.613612 (+ 15450us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:39.640695 (+ 27083us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:39.641216 (+   521us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:12:39.641301 (+    85us) write_op.cc:312] APPLY: finished
0709 13:12:39.641888 (+   587us) log.cc:844] Serialized 101 byte log entry
0709 13:12:39.766482 (+124594us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:39.809167 (+ 42685us) write_op.cc:454] Released schema lock
0709 13:12:39.816356 (+  7189us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:39.895753 (+ 79397us) spinlock_profiling.cc:243] Waited 62.4 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67432a 00007f59ce5f9f14 00007f59ce5f9ec3 00007f59ce5f9e38 00007f59ce5f9df3 00007f59ce5f90dc 00007f59ce622e16 00007f59ce60acbb 00007f59ce5f539d 00007f59ce5f8d75
0709 13:12:39.902662 (+  6909us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:39.907025 (+  4363us) write_op.cc:454] Released schema lock
0709 13:12:40.142772 (+235747us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b1800247780. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a522 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
Metrics: {"mutex_wait_us":458454,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":668598,"apply.run_cpu_time_us":14931,"apply.run_wall_time_us":431647,"num_ops":14,"prepare.queue_time_us":359998,"prepare.run_cpu_time_us":17778,"prepare.run_wall_time_us":335897,"raft.queue_time_us":521745,"raft.run_cpu_time_us":41747,"raft.run_wall_time_us":1276799,"replication_time_us":1287872,"spinlock_wait_cycles":229091840,"thread_start_us":1363095,"threads_started":5,"wal-append.queue_time_us":523931,"wal-append.run_cpu_time_us":74136,"wal-append.run_wall_time_us":2751109}]]}
I20260709 13:12:47.661257 32099 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:47.660459 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 122439us
I20260709 13:12:47.695604 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:47.544055 31957 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 13:12:47.687659 32320 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:47.657444 32227 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:47.647579 32200 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:47.682868 32303 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.013s	sys 0.013s
I20260709 13:12:47.542326 32196 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:47.693647 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 104755us
I20260709 13:12:47.694267 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:47.698699 32318 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:47.694438 28084 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:47.698958 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:47.699424 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 105230us
I20260709 13:12:47.683525 32198 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:47.703804 27983 catalog_manager.cc:5697] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 7 to 8, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 8 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:47.647107 31803 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.712790 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:47.838688 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 126000us
I20260709 13:12:47.577382 31788 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.643956 32284 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.720294 32282 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.723414 32310 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:47.737250 32250 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:47.739719 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:47.743917 32314 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.583s	user 0.000s	sys 0.003s
I20260709 13:12:47.740504 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:47.750516 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.860373 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:47.861199 28157 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:47.754424 32258 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:47.759558 32255 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:47.764523 32264 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:47.765908 32286 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:47.769032 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:47.842695 32198 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.781034 32023 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:47.842109 32161 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:47.809634 28154 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
W20260709 13:12:47.815176 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 287344us
I20260709 13:12:47.819891 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2300) took 3146 ms. Trace:
I20260709 13:12:47.704562 32022 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:47.661263 32268 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.824685 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:47.831046 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 136824us
W20260709 13:12:47.929412 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.881677 32258 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:47.851686 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:47.823196 32200 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.854164 28760 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
W20260709 13:12:47.881850 32003 consensus_queue.cc:1175] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "611ec64ec58244fe82058d6130e4e6f3" responder_term: 8 status { last_received { term: 8 index: 9 } last_committed_idx: 6 last_received_current_leader { term: 8 index: 9 } }
W20260709 13:12:47.854202 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 158595us
I20260709 13:12:47.860603 28764 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:47.865952 28757 raft_consensus.cc:2368] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:47.867092 31803 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:47.873626 31826 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:47.847823 32190 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:47.844208 32227 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:47.880241 32240 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:47.825702 32331 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:47.886737 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:47.891630 28152 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:47.770797 32222 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:47.900413 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.841821 31801 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:47.793109 32248 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.924577 28155 rpcz_store.cc:276] 0709 13:12:44.673575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.677020 (+  3445us) service_pool.cc:224] Handling call
0709 13:12:44.720374 (+ 43354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:47.402994 (+2682620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.819622 (+416628us) spinlock_profiling.cc:243] Waited 2.6 s on lock 0x7b5400375fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:47.819822 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5977133440}
I20260709 13:12:47.929814 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:47.821659 32303 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.013s	sys 0.013s
I20260709 13:12:47.931555 28158 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:47.915741 32032 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.667s	user 0.012s	sys 0.000s
I20260709 13:12:47.709008 32254 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 13:12:47.893290 32128 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:47.881439 32255 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:47.713212 32158 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.969604 32268 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:47.917256 32045 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.939708 28764 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:47.972832 32045 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:47.924835 32330 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:12:47.946310 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 947) took 1149 ms. Trace:
I20260709 13:12:47.974560 28763 rpcz_store.cc:276] 0709 13:12:46.796531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:46.796637 (+   106us) service_pool.cc:224] Handling call
0709 13:12:47.946040 (+1149403us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400722254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:47.946207 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079841536}
W20260709 13:12:47.975157 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 88542.7us
I20260709 13:12:47.976437 31850 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:47.916642 32023 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:47.956722 32331 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:47.977888 32331 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:47.887976 32264 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:47.978343 32023 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.979974 32023 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.980015 32264 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.853626 32295 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:12:47.962395 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:47.916955 32198 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:47.985447 32334 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:47.986064 32334 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:47.896082 32205 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.939669 32241 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.924012 32157 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.981805 28226 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:47.982159 32295 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:47.990849 32295 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:47.982259 32255 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.991034 32157 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.992326 32255 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:47.993719 32205 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:47.995074 32241 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:47.984411 32264 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:47.996867 32343 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:47.998116 28226 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:48.000646 32188 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:48.001228 32188 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:47.991732 32336 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:12:48.001616 32295 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.005448 32243 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:48.006033 32243 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:48.007056 32243 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.008374 32243 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:48.012956 31788 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:47.972086 28764 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.015592 28764 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:48.007624 32260 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.877s	user 0.005s	sys 0.004s
I20260709 13:12:47.987448 32334 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.025074 32227 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:48.027405 32227 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.928884 32161 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:48.028858 32335 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 13:12:48.023093 32279 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.012s	sys 0.008s
I20260709 13:12:48.024017 32241 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:47.997713 32343 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:47.962532 32248 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:48.049474 32279 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.013s	sys 0.008s
I20260709 13:12:48.029027 32260 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.056270 32334 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:48.054575 32250 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:48.073922 32260 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:48.079986 32343 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:47.836321 27983 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 3 to 5, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 5 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:48.115005 27983 catalog_manager.cc:5697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 5 to 7, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 7 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:48.120370 27983 catalog_manager.cc:5697] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:48.126139 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:48.126366 27983 catalog_manager.cc:5697] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:48.128216 28049 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:47.839375 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:48.132215 27983 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 7 to 8, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 8 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
W20260709 13:12:48.133945 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 294511us
I20260709 13:12:48.136420 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:48.135133 27983 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 4 to 5, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 5 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:48.141741 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:48.141815 27983 catalog_manager.cc:5697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 4 to 5, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 5 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
W20260709 13:12:48.143635 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.834913 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:48.187709 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 257806us
I20260709 13:12:48.065779 32250 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:48.191115 32250 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.195899 32248 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:48.077723 32331 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.199061 32279 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:48.200970 32343 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:48.201443 28758 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:48.202064 32291 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:48.203712 28079 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:48.203394 32291 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:48.187783 27983 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 8 to 10, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 10 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:48.207499 32303 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.156381 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:48.161018 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:48.209908 32282 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:48.210312 28157 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:48.161599 28161 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:48.163100 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 744) took 1054 ms. Trace:
I20260709 13:12:48.209771 32331 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:48.213270 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:48.211621 28160 rpcz_store.cc:276] 0709 13:12:47.108008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:47.108167 (+   159us) service_pool.cc:224] Handling call
0709 13:12:47.131963 (+ 23796us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:47.132278 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:47.132306 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:47.132761 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:12:47.353158 (+220397us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b18004294c0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:12:47.566581 (+213423us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:47.566622 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:47.567748 (+  1126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:48.162023 (+594275us) raft_consensus.cc:1685] finished
0709 13:12:48.162090 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:48.162744 (+   654us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":260,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":138,"spinlock_wait_cycles":5993344,"thread_start_us":425680,"threads_started":1,"wal-append.queue_time_us":528120}
W20260709 13:12:48.058919 32183 consensus_meta.cc:287] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.004s
I20260709 13:12:48.026140 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:48.216347 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 190102us
I20260709 13:12:48.216099 32183 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.217389 32258 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.218556 32340 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:48.219513 32340 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:48.219609 32333 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:48.222054 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 387000us
I20260709 13:12:48.223671 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:48.223839 32183 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:48.225255 32183 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.175336 32161 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:48.178074 32260 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:47.967319 31801 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.179564 32088 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:48.055393 32101 consensus_meta.cc:287] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.564s	user 0.006s	sys 0.006s
I20260709 13:12:48.201184 32289 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:48.210690 32282 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.247815 32088 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.208547 32303 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.249004 32282 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:48.251724 32088 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:48.229259 32343 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:48.170229 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.159431 28124 leader_election.cc:400] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12.
I20260709 13:12:48.200254 32279 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:48.233049 32333 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:48.217788 27983 catalog_manager.cc:5697] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 5 to 6, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 6 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:48.233500 32340 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.210832 28152 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:48.238371 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:48.238991 28757 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:48.239377 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:48.245224 32250 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.245504 32101 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.251785 32303 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:48.254493 28124 leader_election.cc:403] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:12:48.228243 32206 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.741s	user 0.013s	sys 0.000s
I20260709 13:12:48.256816 32279 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.257768 31801 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:48.257943 32260 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:48.277458 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 124866us
I20260709 13:12:48.277949 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:48.279305 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:48.281471 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 140119us
I20260709 13:12:48.283177 32206 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.284972 32289 raft_consensus.cc:2764] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:12:48.285466 28762 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:12:48.285898 28762 pending_rounds.cc:85] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 4
I20260709 13:12:48.286167 28762 pending_rounds.cc:107] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.5
I20260709 13:12:48.286614 28762 raft_consensus.cc:2917] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Aborting config change with OpId 6.5: Aborted: Op aborted by new leader
I20260709 13:12:48.286641 32206 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:48.287218 28762 raft_consensus.cc:2889] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 8 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:48.289294 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50167.5us
I20260709 13:12:48.289633 32101 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:48.293231 32252 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:48.294088 32252 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:48.294862 32260 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:48.299197 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 59257.4us
I20260709 13:12:48.295445 32088 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:48.304282 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:48.286032 32289 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:48.306255 32101 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.311461 32101 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.314388 32198 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.316325 32157 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:48.317862 32101 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:48.319154 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:48.320205 28757 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:48.321996 32198 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:48.326599 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:48.327287 32295 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.327905 28763 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:48.307799 27983 catalog_manager.cc:5697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 2 to 5, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 5 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:48.334326 28764 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
W20260709 13:12:48.346113 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.346930 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:48.349457 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.349555 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:48.350879 28049 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:48.351320 32303 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:48.352159 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:48.295496 32252 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.359994 28755 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:12:48.361304 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 79024.1us
I20260709 13:12:48.362576 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:48.369443 32289 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:48.370362 32289 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:48.370927 32289 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:48.371250 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:48.373196 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.323846 32183 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:48.379146 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 43543.8us
I20260709 13:12:48.372102 32291 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:48.364420 32282 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.374007 32343 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:48.384300 27983 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 8 to 9, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 9 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:48.390034 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:48.382848 32333 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:48.394946 32292 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:48.395390 32320 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:12:48.398944 28761 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:48.401099 32281 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:48.401875 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:48.403152 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.406023 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:48.406522 28078 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:48.406210 27983 catalog_manager.cc:5697] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 5 to 6, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 6 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:48.416621 32200 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.419287 32200 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:48.420321 28760 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:48.357877 32188 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.404706 32258 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:48.421993 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 57364.2us
I20260709 13:12:48.428828 28158 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:48.430362 32264 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12.
I20260709 13:12:48.432055 32128 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:48.404187 32292 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.433302 32292 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:48.438644 32183 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:48.444483 32128 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.445612 32128 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.447916 32188 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:48.448971 32128 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:48.452879 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69453.4us
I20260709 13:12:48.454805 32303 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.455685 32303 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.457062 32088 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:48.458241 32303 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:48.460827 32023 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:48.462113 32282 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:48.465246 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:48.467223 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:48.468348 31788 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:48.469619 31788 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.469309 32303 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:48.472074 32303 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:48.473124 28762 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:48.474133 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:48.412417 32281 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.475024 32281 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:48.476691 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:48.478667 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:48.479549 28764 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:48.483098 32320 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:48.482319 32303 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:48.494256 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:48.487685 32284 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.495460 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:48.496676 28081 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:48.497843 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:48.494189 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.483727 32320 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:48.501744 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:48.502786 28086 raft_consensus.cc:2368] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9.
I20260709 13:12:48.503659 32157 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.502384 32320 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.504392 32157 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.505200 32320 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:48.507733 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:48.508914 28158 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:48.387966 32289 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.511133 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:48.511924 28156 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:48.514694 32339 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:48.515225 32339 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:48.519933 28757 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:48.508945 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:48.534792 32344 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.469s	user 0.000s	sys 0.004s
I20260709 13:12:48.535815 32310 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:48.536545 32310 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.537895 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:48.538782 28226 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:48.539249 32310 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:48.536783 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:48.540424 28231 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:48.545504 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:48.546350 28757 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:48.547663 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:48.548638 28761 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:48.538045 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:48.549165 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:48.545504 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:48.553099 27983 catalog_manager.cc:5697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 5 to 6, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 6 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:48.516352 32339 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.554807 27983 catalog_manager.cc:5697] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 6 to 8, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 8 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:12:48.556631 32339 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:48.558220 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:48.558478 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:48.559069 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:48.559281 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:48.560231 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:48.562496 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:48.556759 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:48.565266 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:48.565618 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:48.566184 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:48.566386 32157 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:48.567169 28198 leader_election.cc:400] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:48.567286 32310 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:48.567560 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:48.568143 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:48.568526 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:48.571041 32088 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:48.572190 32188 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:48.586441 32264 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:48.588450 32222 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:48.595223 31788 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:48.603178 28198 leader_election.cc:403] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:12:48.603895 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 140487us
I20260709 13:12:48.599025 27983 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 6 to 8, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 8 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:48.612152 28761 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:48.608071 28231 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:48.620122 28050 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:48.615346 28757 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:12:48.631289 28081 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
W20260709 13:12:48.639601 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 72707.8us
I20260709 13:12:48.644399 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:48.618620 32292 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.650156 28198 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:48.650642 28158 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:48.651121 28078 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:48.647929 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 80143.7us
W20260709 13:12:48.661458 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.662940 28161 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:48.645478 32282 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:12:48.665761 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 45799.7us
I20260709 13:12:48.611688 32310 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.672983 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:48.674494 28079 raft_consensus.cc:2368] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12.
I20260709 13:12:48.631806 32279 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:12:48.631853 32258 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:12:48.642246 32188 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.671396 32183 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:48.666540 32250 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:48.685115 32284 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:48.694128 32183 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.684178 32281 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.695784 32282 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:48.721881 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 82596.8us
I20260709 13:12:48.715881 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:48.718281 32138 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:12:48.700193 32254 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:48.707520 32284 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.723467 32333 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:48.717090 32183 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:48.730777 32333 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:48.730000 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61639.6us
I20260709 13:12:48.731864 32279 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.736028 28080 raft_consensus.cc:2368] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11.
I20260709 13:12:48.729450 32254 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:48.726797 32284 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:48.742864 28198 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:48.743062 28231 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:48.737883 32227 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.741704 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:48.730901 32222 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:48.747418 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:48.752635 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:48.747462 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 142857us
I20260709 13:12:48.729557 32157 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:48.757405 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:48.758919 28227 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:12:48.760846 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.762873 28228 raft_consensus.cc:2410] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:48.767376 28230 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:48.777562 28125 leader_election.cc:400] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9.
I20260709 13:12:48.777966 28125 leader_election.cc:403] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:12:48.752437 32252 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:48.750737 32128 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:12:48.763379 32157 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:48.787285 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:48.787947 28757 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:48.776985 32292 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:48.791059 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:48.792227 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:48.793463 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:48.794400 28235 raft_consensus.cc:2410] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:12:48.752715 32303 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:12:48.785907 32291 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:48.806501 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:48.772162 32227 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.823294 28125 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:48.790521 32252 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:48.818755 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:48.805197 32258 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9.
I20260709 13:12:48.807871 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:48.808243 31801 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:48.808898 32310 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:48.831067 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:48.810062 32291 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:48.834214 28125 leader_election.cc:400] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12.
I20260709 13:12:48.839493 28125 leader_election.cc:403] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:48.826011 32188 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:48.841452 28234 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.782764 32282 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.841399 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:48.846911 28232 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:12:48.848332 32241 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:48.848819 32241 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:48.849829 32241 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.851315 32241 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:48.855199 28233 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 8: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:48.842765 32291 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.862337 28232 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:48.863763 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:48.865146 28230 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:48.865744 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:48.845551 32281 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:48.869690 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:48.862907 32222 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:48.879055 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:48.801427 32333 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.873960 32243 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:48.885097 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 162074us
I20260709 13:12:48.809517 32254 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.870456 32264 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:12:48.891649 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 211592us
W20260709 13:12:48.894325 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48804.1us
I20260709 13:12:48.896440 28228 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:12:48.846316 32157 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:48.907616 28078 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:48.893153 32264 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.913439 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:48.923089 28757 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:48.896572 32243 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:48.928818 28233 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:48.933001 32200 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:48.934341 32200 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:48.934796 32200 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:48.935864 32200 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:48.941327 32200 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:48.943288 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:48.948676 32206 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:48.949189 32206 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.949544 32206 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:48.896374 32281 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:12:48.953150 32101 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:48.957087 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:48.963068 32227 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:48.966367 28728 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:48.925668 32264 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:48.968365 28234 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
W20260709 13:12:48.970934 32361 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:12:48.959941 32362 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
I20260709 13:12:48.972340 28229 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:49.019767 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.022641 28228 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:49.031118 28232 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:49.031281 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.032153 28230 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:49.035384 28233 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:48.972352 32333 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:48.971287 32128 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 8 LEARNER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:12:49.050557 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.917495 32252 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:49.059221 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 92915.3us
W20260709 13:12:49.060817 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.060915 32370 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.062395 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.065119 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.065173 28228 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:49.065802 28226 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:49.066532 32243 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.052565 32188 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:49.073336 32188 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.074664 28228 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:49.075520 32188 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:49.077932 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.078678 28232 raft_consensus.cc:2368] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9.
I20260709 13:12:49.050523 32258 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.082114 32258 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:49.085667 28230 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:49.089876 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:49.091344 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.028352 32264 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:49.041468 32157 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:49.098569 28226 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:49.098536 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:49.099696 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.107941 28228 raft_consensus.cc:2410] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:12:49.109010 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:49.109850 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.110244 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.109010 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.110809 28078 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:49.111028 28079 raft_consensus.cc:2368] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:49.111387 28228 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:49.111583 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:49.112514 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.113929 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:49.114477 28078 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:49.114825 28086 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:49.114926 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:49.113991 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.110916 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.116241 28228 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:49.118621 28077 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:49.121059 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.123310 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.125111 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.149695 28228 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
W20260709 13:12:49.154078 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48979.4us
I20260709 13:12:49.155172 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:49.155701 28230 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:49.156342 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:49.157374 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.157966 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.158111 28233 raft_consensus.cc:2368] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11.
I20260709 13:12:49.163913 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.168192 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.170346 28235 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:49.075981 32370 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:49.092599 32206 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.178699 28080 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:49.180464 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.176978 28228 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:49.159533 32310 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:49.187203 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:49.187637 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:49.165032 32279 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:49.192168 28228 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:49.193503 28235 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:49.193692 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.194721 28226 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:49.195255 28232 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:12:49.195744 28228 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:49.203115 28231 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:49.205149 32264 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.206382 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:49.206933 32264 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.207278 28078 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:49.207617 32264 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:49.209069 28086 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:49.210309 28083 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:49.216187 28232 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:49.197155 32128 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:49.221179 32128 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:49.221510 28226 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:49.221675 32128 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:49.222226 32264 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.217849 28160 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:49.228801 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.199895 32222 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12.
I20260709 13:12:49.197986 32258 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.194869 32310 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:49.196094 32291 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:49.242849 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.245076 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.245584 32310 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:49.248858 32375 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:49.249531 32320 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:49.250036 32320 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:49.257452 28086 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:49.260550 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:49.262849 28226 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:49.263057 28228 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:49.265115 28156 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.000s
I20260709 13:12:49.265620 28156 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:49.273177 28084 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:49.275498 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 63407.7us
I20260709 13:12:49.277729 32138 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:49.278024 32375 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:49.278674 32138 consensus_queue.cc:1243] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:49.285146 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:49.286211 28156 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:49.289126 28161 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:12:49.288856 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.289721 28161 pending_rounds.cc:85] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 4
I20260709 13:12:49.294724 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.294902 28161 pending_rounds.cc:107] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.5
I20260709 13:12:49.291963 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.294157 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.302285 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:49.303408 28156 raft_consensus.cc:2368] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:49.301518 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.306931 28161 raft_consensus.cc:2917] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Aborting config change with OpId 6.5: Aborted: Op aborted by new leader
I20260709 13:12:49.307734 28161 raft_consensus.cc:2889] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:49.240587 32200 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:12:49.311167 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:49.314364 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:49.312320 28227 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:49.314981 28152 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:49.298558 32138 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.298185 32222 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:49.320026 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 97188.5us
I20260709 13:12:49.324949 28086 pending_rounds.cc:85] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 7
I20260709 13:12:49.324987 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:49.325533 28086 pending_rounds.cc:107] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:12:49.325810 28086 raft_consensus.cc:2889] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:49.308329 32384 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:49.337693 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:49.338107 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:49.339156 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.317997 32291 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.283849 32320 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:49.343950 28152 raft_consensus.cc:2368] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:49.345158 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:49.355130 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.356024 28226 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:49.360947 32381 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.165s	user 0.002s	sys 0.000s
W20260709 13:12:49.362136 32376 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:12:49.364635 32384 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:49.367269 32384 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.371254 32206 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.355214 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:49.355589 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.375897 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:49.385047 28227 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:49.393666 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:49.398839 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:49.399598 28161 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:49.326743 32138 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.357100 32310 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.339823 32222 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.412745 28758 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
W20260709 13:12:49.413132 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61882.9us
W20260709 13:12:49.396701 32284 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.002s
I20260709 13:12:49.420742 28156 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:49.393059 32385 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:49.422411 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.422829 28199 leader_election.cc:400] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:49.426640 28199 leader_election.cc:403] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:49.421842 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:49.429869 28227 raft_consensus.cc:2410] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:49.435752 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:49.436966 28152 raft_consensus.cc:2368] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:49.405253 32370 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.451514 28226 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:49.451985 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:49.452293 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.452634 28156 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:49.454468 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:49.444454 32385 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:49.444386 32343 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 8.
I20260709 13:12:49.426290 32284 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.423089 32310 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.479593 32255 raft_consensus.cc:2764] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:12:49.483484 32389 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:12:49.488070 28757 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:49.451390 32188 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:12:49.499063 28161 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:49.502820 32310 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:49.503721 32310 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:49.478195 32292 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.512156 32292 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.514464 32292 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:49.516115 32138 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.518209 32222 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:49.520162 32284 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.521107 32260 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:49.521555 32260 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:49.522589 32260 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:49.523698 32383 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.182s	user 0.000s	sys 0.002s
I20260709 13:12:49.523883 32260 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:49.543103 28157 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:49.555922 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:49.532598 32241 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.681s	user 0.011s	sys 0.005s
I20260709 13:12:49.558279 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.559608 28232 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:49.562052 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:49.614176 28152 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:49.616834 28231 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:49.544483 32128 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.537225 32243 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:49.629871 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 64588us
W20260709 13:12:49.631743 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.632025 28758 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:49.634673 28730 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:49.641851 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:49.643198 28763 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
W20260709 13:12:49.646404 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 191992us
I20260709 13:12:49.629468 32390 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.655279 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:49.658349 28231 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:49.650416 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:49.653992 32157 consensus_meta.cc:287] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.004s
I20260709 13:12:49.648963 32200 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:49.645380 32254 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:49.668493 28160 raft_consensus.cc:2410] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:12:49.670288 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:49.665939 32399 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:12:49.672914 32339 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:12:49.673187 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:49.658238 32188 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:49.674810 32369 log.cc:927] Time spent T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.354s	user 0.000s	sys 0.003s
I20260709 13:12:49.658892 32281 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:49.659226 32390 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:49.682725 32395 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:12:49.658157 32343 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:12:49.691919 28758 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:49.562361 32385 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.686133 28760 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:49.633742 32243 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.684942 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:49.704453 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.696556 32399 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.697556 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.667865 32200 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:49.692106 32291 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:12:49.707367 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.707612 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:49.707924 32396 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.218s	user 0.000s	sys 0.002s
W20260709 13:12:49.709782 32365 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.556s	user 0.001s	sys 0.002s
I20260709 13:12:49.710284 32281 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:49.710175 32254 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.716948 32200 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:49.710990 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:49.713912 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.714557 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:49.715726 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.729828 28230 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:49.729961 28228 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:49.678882 32390 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.718106 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:49.732921 28079 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:49.668567 32292 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:12:49.727455 28755 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:49.725147 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.725549 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:49.735600 28233 raft_consensus.cc:2376] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10. Re-sending same reply.
I20260709 13:12:49.721666 28229 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:49.728528 28227 raft_consensus.cc:2376] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10. Re-sending same reply.
W20260709 13:12:49.782655 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 69744.2us
I20260709 13:12:49.783815 28049 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:49.787441 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 52560.8us
W20260709 13:12:49.788374 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.788897 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:49.789275 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:49.724687 32281 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:49.726099 32128 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.727291 32254 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.796819 32281 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:49.720707 32399 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.818477 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:49.798180 32157 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:49.821290 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 962) took 1092 ms. Trace:
I20260709 13:12:49.831076 28231 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:49.842178 28228 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:12:49.847606 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 133135us
I20260709 13:12:49.812285 32289 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 13:12:49.845562 32183 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.096s	user 0.006s	sys 0.009s
I20260709 13:12:49.849749 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:49.821874 28761 rpcz_store.cc:276] 0709 13:12:48.724333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.724508 (+   175us) service_pool.cc:224] Handling call
0709 13:12:48.734055 (+  9547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:48.742194 (+  8139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:48.742256 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:48.746456 (+  4200us) log.cc:844] Serialized 26 byte log entry
0709 13:12:48.937919 (+191463us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:48.937967 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:48.938054 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:49.685276 (+747222us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:49.816613 (+131337us) raft_consensus.cc:1685] finished
0709 13:12:49.816641 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:49.816968 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2298,"raft.queue_time_us":2510,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":1676,"spinlock_wait_cycles":7273856,"thread_start_us":166416,"threads_started":1,"wal-append.queue_time_us":319688}
I20260709 13:12:49.850941 32222 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:49.854266 32243 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:49.848912 32292 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:49.858157 32399 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:49.859927 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 47909.6us
I20260709 13:12:49.861091 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:49.848304 32333 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:12:49.871812 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 185190us
I20260709 13:12:49.837078 32157 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.881539 32260 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:49.882099 32260 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:49.882558 32260 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:49.879195 32292 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:49.885336 32410 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:12:49.886282 32292 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.875734 32333 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:49.877930 32291 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.865449 32183 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.880641 32416 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:12:49.899928 32183 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:49.900467 28235 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
W20260709 13:12:49.900749 32409 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:12:49.893373 32333 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:49.895630 32417 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
W20260709 13:12:49.911002 32402 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:12:49.914130 32157 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:49.925096 32421 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:49.960588 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:49.948966 32421 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:49.919129 32281 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:49.963737 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 90182.7us
I20260709 13:12:49.970449 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:49.972150 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:49.898069 32291 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:49.977921 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 88659.6us
I20260709 13:12:49.991451 28761 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:49.992542 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:49.993027 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1936) took 1088 ms. Trace:
I20260709 13:12:49.968068 32157 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.993399 28084 rpcz_store.cc:276] 0709 13:12:48.880230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:48.880351 (+   121us) service_pool.cc:224] Handling call
0709 13:12:48.900365 (+ 20014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:49.276544 (+376179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:49.968551 (+692007us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b54001ae0d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:49.968820 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":841309952}
I20260709 13:12:49.964563 31801 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:49.999887 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:49.999744 31801 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:50.000816 32200 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:49.996675 32303 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:50.002974 32222 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:50.004668 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:49.995481 32157 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.018721 32281 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.007058 32157 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:49.971655 32370 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:12:50.027577 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:50.031603 28761 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:50.035066 32421 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.038122 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:50.039121 32370 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:50.040149 32370 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:50.040544 32370 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:50.041427 32370 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.042702 32370 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:50.053893 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 54091.3us
I20260709 13:12:50.055080 28127 leader_election.cc:400] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9.
I20260709 13:12:50.055639 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:50.057358 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:50.059229 28153 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:50.062911 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:50.065129 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:50.066088 28155 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:50.030668 32303 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:50.077451 32303 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.046389 32157 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:50.054209 32260 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.083284 32157 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:50.085253 32157 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.086668 28157 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:50.087616 32260 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.088424 32260 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:50.088874 32260 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:50.089493 32291 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.090484 32260 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.091857 32260 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:50.100103 32260 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:50.105580 32260 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.106624 32260 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:50.106539 32279 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:50.107882 32279 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:50.108804 32279 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.110229 32279 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:50.112272 32387 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.770s	user 0.006s	sys 0.000s
W20260709 13:12:50.115072 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.117919 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:50.096359 32200 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.137506 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.139525 28079 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:50.156905 32260 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.159651 32260 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.163414 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:50.164793 28155 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:50.165055 32260 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.164976 31801 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:50.165724 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:50.166091 32260 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.145488 32289 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:12:50.167425 28154 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:50.083751 32255 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.168558 32255 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.172194 32255 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.172933 32303 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.175580 32339 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:50.176033 32339 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:50.178887 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.179989 32310 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:50.180424 32310 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:50.181504 32310 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.182574 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:50.182938 32310 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:50.183364 28153 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:50.184286 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:50.186383 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:50.187219 28156 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 8: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:50.189074 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:50.189673 28158 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:12:50.189714 28159 raft_consensus.cc:2376] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9. Re-sending same reply.
I20260709 13:12:50.191404 28156 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:50.192649 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:50.196764 32088 consensus_queue.cc:1175] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "11f10c12c03e47dcac88b050d4c1431b" responder_term: 10 status { last_received { term: 9 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:50.201472 32416 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:50.202282 32416 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:50.203190 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 85378.8us
I20260709 13:12:50.203578 32418 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:50.203990 32418 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.204351 32418 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:50.204697 32416 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.166221 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:50.209327 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 77650.9us
I20260709 13:12:50.212628 28228 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:12:50.213099 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:50.214653 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:50.215539 28228 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:50.225407 32023 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:12:50.225603 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:50.226004 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.226562 28758 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:50.227759 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:50.216329 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:50.229266 28234 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:50.232894 28079 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:50.233004 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.234136 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:50.233706 28083 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:50.238740 28228 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:50.258996 28154 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:12:50.263280 28161 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:12:50.263314 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 207290us
I20260709 13:12:50.234176 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:50.245158 28049 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:50.201546 31788 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.210866 32399 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.281215 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:50.217777 32183 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:50.226042 32023 raft_consensus.cc:1068] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:12:50.292586 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:50.293322 28758 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:50.230257 32200 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:50.196521 32333 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.239157 32370 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:50.238202 32188 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:50.296371 31788 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.313977 31788 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.314566 32373 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:50.314699 32400 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:50.315131 32373 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:50.315565 32373 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:50.315310 32400 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.317595 32255 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.318280 32255 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.322904 32400 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.323978 32255 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.330562 28758 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:50.331323 32260 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.332959 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:50.333798 32424 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:50.334333 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:50.334470 32320 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:50.334792 32424 consensus_queue.cc:1243] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:12:50.335216 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 54123.2us
I20260709 13:12:50.336133 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:50.336318 32424 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:50.337138 32424 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:50.337366 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:50.338452 32424 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.340299 32424 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.340615 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2349) took 1245 ms. Trace:
I20260709 13:12:50.341054 28235 rpcz_store.cc:276] 0709 13:12:49.094985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.197655 (+102670us) service_pool.cc:224] Handling call
0709 13:12:49.237228 (+ 39573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:49.905962 (+668734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.340292 (+434330us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b54003b3654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:50.340514 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1524816128}
I20260709 13:12:50.345242 28758 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:50.353315 32429 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:50.354035 32429 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:50.354957 32429 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.311092 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.368634 28079 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:50.372895 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:50.281042 28125 connection.cc:256] RPC call timeout handler was delayed by -1.1865440409999337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
I20260709 13:12:50.299950 32416 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:50.378579 28078 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:12:50.381739 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:50.301090 32200 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.310003 32370 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:50.384002 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.388461 28079 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:50.278213 32303 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 13:12:50.395680 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.396425 31788 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.396905 32427 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:50.397523 31801 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.402779 32281 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:50.278645 32292 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 13:12:50.294499 32289 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:12:50.416657 32320 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.417222 32292 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:50.417320 32320 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.418053 32292 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:50.419865 32320 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.419478 32292 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.402803 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:50.416078 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.387223 32416 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:50.421437 32416 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:50.361049 32426 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:50.421818 32420 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.422246 32426 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:50.422526 32420 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.425016 32427 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:50.425880 32420 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.427614 32333 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.391065 32412 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:12:50.385699 32088 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:12:50.300773 32399 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.394292 32200 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:50.306439 32183 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:50.431885 32399 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:50.429068 32292 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:50.442921 32183 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:50.380600 32128 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.449376 32128 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.412087 31788 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.455878 31788 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.381044 32260 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:50.456625 32260 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.457065 32260 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:50.457962 31788 raft_consensus.cc:476] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Not starting pre-election -- already a leader
I20260709 13:12:50.461163 32413 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:12:50.461455 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 52581.4us
I20260709 13:12:50.461791 32413 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.464133 32413 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.467495 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:50.469863 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.483024 28079 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:50.369936 32188 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.497444 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:50.512928 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.518023 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:50.518723 28081 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:50.452593 32128 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.426067 32427 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.532323 28081 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:50.523456 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:50.535802 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:50.540030 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.481777 32206 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.483604 31788 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.541405 28079 raft_consensus.cc:2376] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12. Re-sending same reply.
I20260709 13:12:50.525082 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:50.553797 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 86424.4us
I20260709 13:12:50.554595 28763 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:50.490523 32333 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.500310 32243 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.510381 32426 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:50.561995 28085 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:50.564395 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:50.472481 32281 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.466042 31801 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.521948 32393 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.558398 32279 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.569619 32279 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.572067 32279 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.562054 31788 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.573444 32023 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:12:50.573906 32023 raft_consensus.cc:1068] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:12:50.574623 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.566907 32243 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.578742 32188 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.579272 32243 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.544095 32183 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:50.580036 32183 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.583060 32183 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.553474 32399 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.584112 32260 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.520623 32157 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:50.599871 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 218226us
I20260709 13:12:50.588471 31801 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:50.678869 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.560148 32200 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.558620 32206 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.695806 28125 leader_election.cc:400] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11.
I20260709 13:12:50.696219 28125 leader_election.cc:403] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:12:50.593745 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 259946us
I20260709 13:12:50.590338 32393 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.565397 32370 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.710216 32370 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:50.713893 32423 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:50.721024 32423 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:50.722824 32423 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.724557 32423 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:50.736940 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:50.743417 32310 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.003s
W20260709 13:12:50.748268 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 64601.4us
I20260709 13:12:50.749325 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2363) took 1114 ms. Trace:
I20260709 13:12:50.750344 28127 leader_election.cc:400] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11.
W20260709 13:12:50.751564 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 69603.1us
I20260709 13:12:50.751611 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:50.704101 32303 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:50.752941 31801 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11.
I20260709 13:12:50.753252 32303 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.754365 32429 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:50.702999 28758 raft_consensus.cc:2410] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:12:50.688058 32333 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.753708 31801 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:50.717723 32416 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.593770 32399 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.710471 32200 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.756337 32303 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:50.757688 32402 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:12:50.777765 28755 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:50.749799 28152 rpcz_store.cc:276] 0709 13:12:49.634829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.636381 (+  1552us) service_pool.cc:224] Handling call
0709 13:12:49.636903 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:49.652904 (+ 16001us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:12:49.667765 (+ 14861us) log.cc:844] Serialized 14 byte log entry
0709 13:12:50.706575 (+1038810us) spinlock_profiling.cc:243] Waited 5 ms on lock 0x7b1800448360. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a
0709 13:12:50.748979 (+ 42404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:50.749034 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:50.749075 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:50.749138 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:50.749265 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42750,"prepare.queue_time_us":50426,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":43969,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":11500928,"thread_start_us":26078,"threads_started":1,"wal-append.queue_time_us":48804}
I20260709 13:12:50.780354 28080 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:50.787721 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:50.776844 32427 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.793430 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:50.796185 32281 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:50.798179 28199 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.008s	sys 0.022s
W20260709 13:12:50.798584 28199 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.008s	sys 0.022s
W20260709 13:12:50.798867 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 623719us
I20260709 13:12:50.780320 32429 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:12:50.810663 31801 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:50.811172 31801 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.811569 31801 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:50.820266 32222 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:50.829828 32222 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:50.832358 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:50.833676 28231 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:50.836423 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:50.838155 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:50.838881 28233 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:50.841395 28227 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:12:50.849251 28198 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:50.807235 32310 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:50.834754 32373 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.004s
I20260709 13:12:50.786092 32439 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:12:50.873747 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:50.875754 28231 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:50.878335 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:50.805897 32427 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.883244 32393 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:50.884244 32427 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.806694 32440 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 13:12:50.888839 32443 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 13:12:50.889856 32432 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 13:12:50.892274 32445 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:12:50.893131 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:50.899132 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:50.901377 32442 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:12:50.903090 32441 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:12:50.905232 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:50.905656 28228 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:50.906013 28229 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.914948 31788 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.920845 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.921221 32206 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:50.921676 28077 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:50.865577 32390 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.923593 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:50.923808 32390 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.926311 32390 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:50.874123 32183 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:50.927259 32183 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.927704 32183 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:50.929189 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:50.930671 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:50.931257 28233 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:50.933450 32416 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.934365 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.934672 32222 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.935566 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:50.936112 32222 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:50.937885 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:50.938642 28226 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:50.939987 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:50.940636 28760 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:50.940421 32339 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.941520 32339 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:50.942279 32260 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:50.943703 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:50.944382 28760 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:50.945684 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:50.870888 32310 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:50.862022 32370 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:50.936151 28083 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:50.956715 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 169249us
I20260709 13:12:50.957006 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:50.957647 28231 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:50.957770 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:50.915148 28229 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
W20260709 13:12:50.947212 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 147761us
W20260709 13:12:50.927066 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:50.960228 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:50.961079 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:50.961007 28229 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:50.946363 28078 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:50.947922 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:50.924304 28227 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:50.965051 28227 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:50.966756 28227 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:50.968806 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:50.972293 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:50.978884 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:50.991660 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:50.991964 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:50.992538 28227 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:50.992612 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:51.000015 28761 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:51.000913 28763 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:51.005699 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:51.011765 28049 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:51.011893 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:51.006814 32188 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:12:50.954589 32260 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.013727 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:51.015589 28764 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:51.011770 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:51.024199 28228 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:12:51.036763 28231 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:12:50.992944 32429 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:50.873189 32373 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:50.996495 32440 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:51.013669 32291 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:51.002874 32343 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:51.046344 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:51.047744 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:51.004321 32439 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:51.040550 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:51.050453 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:51.075479 28755 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:51.075819 28227 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:51.008955 32393 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:51.056807 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:51.062351 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:51.046222 32420 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
W20260709 13:12:51.082477 32418 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.873s	user 0.009s	sys 0.002s
I20260709 13:12:51.082935 32418 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.055300 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:51.084180 28228 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:51.062546 32399 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.085122 28084 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:51.086705 32373 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.075778 28080 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:51.063220 28085 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:51.078032 28078 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:51.079201 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:51.080334 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:51.087994 28757 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:51.103605 28227 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:51.075070 32343 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.109772 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:51.110519 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.112387 28231 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:51.109280 32343 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:51.117705 32423 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:12:51.118882 32423 raft_consensus.cc:1068] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:12:51.119971 32310 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:12:51.058326 32157 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.110464 28760 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:51.066280 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:51.127848 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:51.041498 32260 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.077579 32393 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:51.137844 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 344513us
I20260709 13:12:51.040138 32416 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:51.137880 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:51.140852 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:51.076910 32291 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.144400 28763 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:51.157439 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:51.176798 28760 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:12:51.057657 32429 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.141260 32393 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:51.184062 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.085222 32399 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:12:51.192765 28231 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
W20260709 13:12:51.193854 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.194965 32260 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.196821 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:51.209408 28078 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:51.181720 32206 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:51.108302 28758 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:51.122246 32440 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 13:12:51.207506 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.142643 32416 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:51.157485 32291 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:51.217432 32449 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:51.177232 32157 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:51.219875 32449 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.202698 32303 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.179739 28083 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:51.220129 32413 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:51.221809 32416 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:51.222376 32206 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:51.226212 32260 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:51.228684 32418 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.245091 32449 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:51.266701 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 266597us
W20260709 13:12:51.272224 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.279438 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:51.283934 28052 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:51.284988 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:51.286793 31788 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:51.249053 32303 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:51.291324 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 151305us
I20260709 13:12:51.248960 32157 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.297320 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:51.291132 28231 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:51.308176 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:12:51.318147 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.292953 31788 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.325197 31788 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:51.330432 32303 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.334692 32423 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:51.340353 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:51.302170 32418 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:51.364849 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 285332us
I20260709 13:12:51.336812 32417 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:51.361063 32333 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.361845 32423 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:51.359897 32260 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:51.364053 32418 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:51.389643 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 88941.5us
I20260709 13:12:51.364547 32448 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:51.362157 32370 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:12:51.395308 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 125139us
W20260709 13:12:51.478335 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:51.479089 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.495080 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:51.524552 28228 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:51.525799 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:51.494031 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:51.394773 32427 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:51.389360 32447 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:51.389323 31801 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.577s	user 0.012s	sys 0.003s
I20260709 13:12:51.364931 32183 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.391113 32455 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:51.384593 32426 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.368142 32417 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:51.360193 32206 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.501998 32429 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.536435 32427 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:12:51.389478 28050 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.132s (SENT) [suppressed 20 similar messages]
I20260709 13:12:51.503655 32343 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.556241 32455 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:51.506042 28231 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:51.518362 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:51.583478 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 65271.5us
I20260709 13:12:51.525949 32448 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.497361 32423 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:51.533192 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:51.537644 28228 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:51.496604 32418 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:51.538470 28229 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
W20260709 13:12:51.544870 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.587227 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:51.502151 32469 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.588181 32469 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.590584 32469 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.591204 32469 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:51.559688 32454 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:12:51.573004 28233 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:12:51.595285 32281 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:51.595738 32281 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.573428 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:51.559494 32468 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.596091 32281 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:51.596553 32469 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:51.597157 32469 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:51.596571 32468 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.598184 32469 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.600070 32468 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:51.508553 32222 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.008s
I20260709 13:12:51.565330 32370 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.584367 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:51.601537 32370 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.602317 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:51.603648 32370 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.603894 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:51.604274 32370 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:51.519268 28198 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:51.585508 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 52440.9us
I20260709 13:12:51.566251 32417 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:51.608801 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:51.556084 31801 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.551668 32459 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:12:51.622431 28226 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:51.619583 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:51.573961 32260 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:51.554924 32447 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:51.584209 32452 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
W20260709 13:12:51.625095 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 105950us
I20260709 13:12:51.625138 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.625406 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.612303 32468 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.609146 32157 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.624563 32424 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:51.583801 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:51.628407 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:51.631774 28152 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:51.638187 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.622846 32222 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:51.627182 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.627746 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.639935 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.661650 28230 raft_consensus.cc:2368] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
I20260709 13:12:51.640681 28155 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:12:51.662374 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 89034.4us
W20260709 13:12:51.662142 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 99353.6us
I20260709 13:12:51.650328 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.631393 28161 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:51.659174 28154 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:51.661561 32424 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.665638 32424 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:51.665925 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:51.666692 32373 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:51.667248 32373 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:51.668200 32373 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.649194 32423 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:51.651808 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.631974 28198 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:51.626097 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:51.688290 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.569643 32426 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:51.690946 28153 raft_consensus.cc:2410] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:12:51.661545 28159 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:51.651006 32303 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:51.706328 32303 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:51.707412 32303 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:51.716953 32339 consensus_meta.cc:287] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.775s	user 0.007s	sys 0.010s
I20260709 13:12:51.717247 28157 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:51.718812 32183 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.718709 32449 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.720628 32343 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.656225 32468 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.722558 32333 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.723126 32333 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.723501 32468 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.724157 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:51.724807 28158 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:51.725366 32333 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.660869 32416 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.730808 32416 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.644994 32427 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.732571 32427 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:51.735052 32420 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.735635 32420 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.650715 32260 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.645435 32455 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.737865 32455 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:51.742908 32281 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:51.745061 32456 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.302s	user 0.004s	sys 0.004s
I20260709 13:12:51.651314 32470 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:51.638964 32471 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:51.645548 32459 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:51.749086 32459 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:51.750123 32459 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.751304 32459 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:51.764668 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.767761 28157 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:51.767911 28156 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
W20260709 13:12:51.769202 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 379704us
W20260709 13:12:51.692416 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 60507us
W20260709 13:12:51.782742 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.801213 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:51.661950 32222 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:51.653949 32390 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:51.780594 28198 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:51.779171 28158 raft_consensus.cc:2410] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:51.802312 28160 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:51.814004 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:51.814636 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:51.814730 28230 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:51.815570 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:51.816465 28227 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:51.817344 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 207277us
W20260709 13:12:51.823436 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 217050us
I20260709 13:12:51.824384 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:51.782048 32157 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.825865 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:51.828104 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:51.833786 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:51.758512 32476 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:51.832273 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 140366us
I20260709 13:12:51.801342 32471 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:51.839694 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2376) took 1004 ms. Trace:
I20260709 13:12:51.813535 32222 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:51.822284 32292 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.845875 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:51.837414 32476 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:51.846746 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:51.825286 32477 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.847452 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:51.826767 32465 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:51.841711 28232 rpcz_store.cc:276] 0709 13:12:50.835447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.835583 (+   136us) service_pool.cc:224] Handling call
0709 13:12:50.853832 (+ 18249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:50.871613 (+ 17781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:50.871669 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:50.885027 (+ 13358us) log.cc:844] Serialized 26 byte log entry
0709 13:12:51.129577 (+244550us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:51.129619 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.129688 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:51.839481 (+709793us) raft_consensus.cc:1685] finished
0709 13:12:51.839501 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:51.839633 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17235,"raft.queue_time_us":17946,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":9549,"thread_start_us":244135,"threads_started":1,"wal-append.queue_time_us":374574}
I20260709 13:12:51.813192 32475 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:51.839793 32339 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:51.852630 28763 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:51.850639 28758 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:12:51.863992 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:51.848448 32200 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:51.871341 32475 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:51.871459 32200 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.873667 31801 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.826184 32390 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.847805 32474 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:51.847934 32260 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.887001 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:51.886758 32260 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.887559 28159 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:51.887880 28155 raft_consensus.cc:2410] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:12:51.889233 28154 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:51.385228 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:51.861097 28199 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.935s (SENT)
I20260709 13:12:51.882243 28157 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:51.889765 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2292) took 1674 ms. Trace:
I20260709 13:12:51.872361 32440 raft_consensus.cc:3055] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:51.891929 28084 rpcz_store.cc:276] 0709 13:12:50.207453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.207662 (+   209us) service_pool.cc:224] Handling call
0709 13:12:51.881971 (+1674309us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b540017c854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:51.882178 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1973940608}
I20260709 13:12:51.857017 32476 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:51.894368 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:51.865208 32465 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:51.897414 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:51.865331 32448 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.877166 32339 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.840262 32157 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.895432 32429 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:51.890214 32474 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:51.894191 32260 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:51.898042 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:51.853152 32292 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:51.893507 32291 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.674s	user 0.012s	sys 0.000s
W20260709 13:12:51.904247 32467 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 13:12:51.905972 32471 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.899296 32390 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.911121 32426 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.899703 32465 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:51.913180 32449 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.888602 32484 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:51.891655 31801 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:51.893273 32416 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:51.915088 32424 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.919718 32413 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.944657 28161 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:51.919842 32292 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.922447 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:51.865813 32477 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.931207 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.913980 32339 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:51.933792 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:51.956106 28157 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:51.913247 32440 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:51.958993 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.959661 28154 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:51.912041 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2379) took 1068 ms. Trace:
I20260709 13:12:51.915175 32429 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.951448 28161 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:51.872592 32475 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.920464 32310 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:51.957195 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:51.929045 32447 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:51.964555 28125 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:51.969483 28086 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:51.962224 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:51.961192 28233 rpcz_store.cc:276] 0709 13:12:50.843233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.931861 (+ 88628us) service_pool.cc:224] Handling call
0709 13:12:50.932172 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:51.594908 (+662736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.911675 (+316767us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b54000951d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:51.911959 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1472939392}
I20260709 13:12:51.968345 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:51.968640 28084 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:51.970161 28085 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:51.970252 28158 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:51.971951 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:51.953202 32413 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:51.972756 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.978763 28157 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:12:51.929293 32291 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:51.981702 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:51.968910 31788 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.643s	user 0.004s	sys 0.009s
I20260709 13:12:51.982288 28161 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:12:51.985510 28161 pending_rounds.cc:85] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 8
I20260709 13:12:51.985877 28161 pending_rounds.cc:107] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:12:51.986192 28161 raft_consensus.cc:2889] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:51.966262 32429 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:51.983497 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:51.955262 32423 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:51.987843 28760 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:51.982801 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 130677us
W20260709 13:12:51.988806 28199 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.010s (SENT)
I20260709 13:12:51.991647 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:51.992460 28161 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:51.995646 32477 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:51.997390 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:51.999970 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.003402 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.004392 28161 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:51.956200 32416 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:51.978223 31801 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:52.008083 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:51.971853 32426 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:51.967881 32449 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:12:52.019920 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:51.984861 31788 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.025301 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:52.032960 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:52.035326 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:52.042255 28158 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:52.039692 28730 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:52.043769 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.047118 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:52.050146 28161 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:52.051738 28157 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:51.956037 32484 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.052857 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:52.016455 31801 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:52.054235 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:52.055451 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.056097 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:52.054127 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.058166 28161 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:52.060007 28233 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:52.061417 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:52.062247 28764 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:52.062260 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.066078 28152 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:52.083348 32200 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.007128 32423 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.037705 32310 raft_consensus.cc:2764] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:12:52.042567 32448 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:52.041185 32399 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:52.093464 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.057325 32390 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:52.045370 32373 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:52.096766 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:52.098359 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54061.8us
I20260709 13:12:52.052215 32292 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:52.101169 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 112416us
I20260709 13:12:52.089334 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.105242 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:52.106333 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:52.060197 32449 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.107558 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:52.063524 31801 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:52.059940 32457 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:52.060799 32439 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:52.110491 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:52.125342 28155 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:52.125113 32439 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.116390 28082 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
I20260709 13:12:52.119673 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:52.041323 32426 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.120039 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:52.120543 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.122426 32423 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:52.044385 32477 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:12:52.128470 28235 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:52.089346 32487 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:12:52.178879 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:52.169112 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 76618.5us
I20260709 13:12:52.129130 28231 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:52.173346 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:52.172468 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55722.9us
I20260709 13:12:52.111338 32373 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:52.104508 32448 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:52.190460 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:52.186997 28153 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:52.130185 28199 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.048s (SENT)
W20260709 13:12:52.190016 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:52.190460 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.001263 32474 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.106199 32447 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:12:52.238914 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:52.118552 32390 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:52.233422 32200 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:12:52.250548 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 135297us
W20260709 13:12:52.251125 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0183336040001905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.246s for a response
I20260709 13:12:52.252489 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:52.190791 32157 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:52.195722 32373 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:52.261811 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.055518 32440 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:52.262651 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.178279 32499 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:12:52.170352 32449 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.193964 32489 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
W20260709 13:12:52.276077 32486 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.401s	user 0.000s	sys 0.002s
I20260709 13:12:52.277508 32390 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:52.278745 32427 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.546s	user 0.005s	sys 0.004s
I20260709 13:12:52.128940 32292 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:52.253320 32418 consensus_meta.cc:287] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.004s
I20260709 13:12:52.255323 32424 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.268673 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:52.254638 32455 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.006s
I20260709 13:12:52.270273 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:52.295864 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:52.241590 32439 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.197359 32457 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.294390 28153 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:52.271297 32399 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:12:52.294117 32488 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 13:12:52.281008 32291 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:52.319643 28760 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:52.259550 32200 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:12:52.327042 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:52.330001 28153 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:52.330454 32292 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:52.332515 32492 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:12:52.332945 32206 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:12:52.333463 32206 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:52.333738 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2371) took 1508 ms. Trace:
I20260709 13:12:52.334569 32206 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.323078 32399 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:52.336957 32399 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:52.338461 32420 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.291590 32501 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.293846 32418 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.339779 28152 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:52.299189 32472 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:52.339406 32416 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.294159 32448 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.375296 32448 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:52.376771 32501 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.377861 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.378571 28156 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:52.379012 32501 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:52.379526 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.380182 28156 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
W20260709 13:12:52.357312 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:52.326629 32393 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 1.109s	user 0.009s	sys 0.005s
I20260709 13:12:52.381393 32393 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.383724 32393 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.359061 32420 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.281229 32470 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:12:52.361805 32426 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.323006 32477 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:52.387950 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:52.391754 28757 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:52.384580 32420 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.393878 31788 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.387106 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:52.394412 32418 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.365264 32457 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:52.328085 32424 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:52.370460 32449 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:52.327647 32440 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:52.373677 32484 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.358316 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:52.384543 28755 raft_consensus.cc:2368] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
W20260709 13:12:52.401276 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.399566 28235 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
W20260709 13:12:52.415560 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 57352.6us
I20260709 13:12:52.334129 28234 rpcz_store.cc:276] 0709 13:12:50.825264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.825394 (+   130us) service_pool.cc:224] Handling call
0709 13:12:50.838327 (+ 12933us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:50.838822 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:50.955382 (+116560us) log.cc:844] Serialized 14 byte log entry
0709 13:12:51.267463 (+312081us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:51.307056 (+ 39593us) log.cc:844] Serialized 26 byte log entry
0709 13:12:51.336953 (+ 29897us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:51.337001 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.337072 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:52.330349 (+993277us) spinlock_profiling.cc:243] Waited 36.9 ms on lock 0x7b24001df378. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4109c4 00007f59cb40eacb 00007f59cb5c9dc9 00007f59cb5c9a55 00007f59cb5c8fb0 00007f59ce23f488 00007f59ce23fbe8 00007f59ce217398 00007f59ce21276a 00007f59ce211bb6
0709 13:12:52.333390 (+  3041us) raft_consensus.cc:1685] finished
0709 13:12:52.333510 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:52.333670 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12680,"prepare.queue_time_us":305477,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":4935,"raft.queue_time_us":249060,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":27426,"spinlock_wait_cycles":84764928,"thread_start_us":370086,"threads_started":2,"wal-append.queue_time_us":350375}
I20260709 13:12:52.416323 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:52.405197 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.417148 28156 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:52.418041 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.387466 32281 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.438162 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.450475 28156 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:52.440420 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.441733 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:52.453893 28152 raft_consensus.cc:2376] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 12. Re-sending same reply.
I20260709 13:12:52.436703 28757 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:12:52.457317 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:52.405383 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:52.388000 32333 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:12:52.497123 32333 raft_consensus.cc:1068] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:12:52.452443 28160 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:52.394012 32420 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:52.425988 32417 consensus_meta.cc:287] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.800s	user 0.011s	sys 0.002s
I20260709 13:12:52.516573 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.426574 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:52.518581 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:52.406023 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 110237us
I20260709 13:12:52.466156 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:52.520385 28079 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:52.520511 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:52.471280 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:52.479758 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.414121 32501 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:12:52.414276 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:52.405527 32424 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:52.421640 32447 raft_consensus.cc:2764] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:12:52.522588 28756 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 13:12:52.523007 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 288573us
I20260709 13:12:52.523685 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.529527 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:52.543072 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 491686us
I20260709 13:12:52.415917 32418 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:52.550315 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 357718us
W20260709 13:12:52.552279 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 138086us
I20260709 13:12:52.553082 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:52.497575 32339 consensus_meta.cc:287] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.531s	user 0.005s	sys 0.005s
I20260709 13:12:52.563015 28085 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:52.536347 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:52.537365 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.564254 28156 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:52.539793 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:52.541466 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:52.527925 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:52.528872 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:52.548055 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:52.565821 28084 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:52.445017 32457 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:52.525941 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:52.548599 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:52.549038 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:52.567382 28080 raft_consensus.cc:2410] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:12:52.569643 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:52.573508 32469 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.972s	user 0.004s	sys 0.005s
W20260709 13:12:52.573933 32469 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.973s	user 0.004s	sys 0.005s
I20260709 13:12:52.580713 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:52.581096 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:52.516024 32023 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:52.583741 28152 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:52.584782 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.585536 28156 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:52.588536 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.589869 28153 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
W20260709 13:12:52.546255 32222 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.693s	user 0.009s	sys 0.002s
W20260709 13:12:52.561982 32476 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.652s	user 0.009s	sys 0.000s
I20260709 13:12:52.596550 28049 leader_election.cc:400] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:12:52.615180 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.615712 28049 leader_election.cc:403] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:12:52.566854 32420 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:52.542102 32310 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:52.621063 32310 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.623337 32310 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:52.561992 32470 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:52.446096 31801 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.626607 28086 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:52.570510 28050 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:52.596637 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 56958.9us
I20260709 13:12:52.598816 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.630227 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:52.631914 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51278.8us
I20260709 13:12:52.638324 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:52.579414 32291 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:52.590901 32333 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:12:52.642148 32463 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.848s	user 0.000s	sys 0.003s
I20260709 13:12:52.642833 32508 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:12:52.644865 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.646111 28079 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:12:52.644573 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:52.670652 28158 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:12:52.579793 32487 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:12:52.673012 28159 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:52.592758 32413 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.002s
I20260709 13:12:52.653331 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:52.628680 32506 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:52.655040 28730 leader_election.cc:400] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
I20260709 13:12:52.677321 28153 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
W20260709 13:12:52.662185 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 91727.6us
I20260709 13:12:52.561817 32375 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:12:52.664673 28080 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:12:52.626344 32023 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:52.651554 28155 consensus_meta.cc:287] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.010s
I20260709 13:12:52.581981 32484 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:12:52.567389 32418 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.639612 32499 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:52.674438 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:52.654435 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.645624 32495 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:52.639559 32447 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.721668 32447 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.721998 27980 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 11, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 11 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:52.723834 32447 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:52.723704 27980 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:52.677691 28730 leader_election.cc:403] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:12:52.530247 32417 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.664615 32333 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:52.668955 32292 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:52.583504 32339 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.626973 32476 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.680663 28155 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
W20260709 13:12:52.690235 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48279.6us
I20260709 13:12:52.686906 28154 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:52.679701 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:52.708696 28158 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:12:52.713717 28152 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:52.669924 32508 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:52.714991 28159 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:52.716360 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:52.717597 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 979) took 1628 ms. Trace:
I20260709 13:12:52.718472 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:52.589614 32303 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:12:52.724496 32469 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:52.725016 32426 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:52.640169 32222 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.678444 32498 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:52.756425 32508 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:52.760161 32498 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:52.760635 32498 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:52.645265 32424 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:52.727093 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 85463.2us
I20260709 13:12:52.758939 32469 consensus_queue.cc:1243] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:12:52.766214 32291 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:52.766708 32291 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:52.768932 28161 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:52.769712 32222 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.770079 32393 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:52.767627 32291 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.770910 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:52.771716 32291 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:52.728555 28084 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
W20260709 13:12:52.692480 32459 consensus_meta.cc:287] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.930s	user 0.011s	sys 0.004s
W20260709 13:12:52.736442 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.738899 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:52.779469 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:52.742887 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 194357us
I20260709 13:12:52.745806 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:52.743386 32292 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:52.755779 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:52.755914 28160 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:52.778355 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:52.719333 32499 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:52.678643 32457 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:12:52.825974 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:52.832201 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 249800us
I20260709 13:12:52.754477 28761 rpcz_store.cc:276] 0709 13:12:51.089448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.089646 (+   198us) service_pool.cc:224] Handling call
0709 13:12:51.114226 (+ 24580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:51.125502 (+ 11276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:51.125539 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:51.142312 (+ 16773us) log.cc:844] Serialized 26 byte log entry
0709 13:12:51.358383 (+216071us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:51.358425 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:51.358498 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:52.429977 (+1071479us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:52.717224 (+287247us) raft_consensus.cc:1685] finished
0709 13:12:52.717256 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:52.717517 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10800,"raft.queue_time_us":11184,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":11420,"spinlock_wait_cycles":102702976,"thread_start_us":145607,"threads_started":1,"wal-append.queue_time_us":391717}
I20260709 13:12:52.758358 28078 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:52.678855 32413 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.776177 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:52.776700 28235 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:52.772434 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:52.742246 32390 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
W20260709 13:12:52.746726 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 67092.3us
I20260709 13:12:52.838168 28226 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:52.801839 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:52.838847 28231 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:52.744105 32333 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:52.746253 32284 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:12:52.787387 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:52.808518 28084 pending_rounds.cc:85] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 7
I20260709 13:12:52.846868 28084 pending_rounds.cc:107] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:12:52.847148 28084 raft_consensus.cc:2889] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:52.757498 32489 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:52.832489 28233 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:52.834605 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:52.786098 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:52.839579 28230 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:52.816915 32292 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:52.804023 32310 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:52.858095 32418 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:52.860913 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 74981.2us
I20260709 13:12:52.782460 32426 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:52.842031 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:52.765606 32424 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:52.834851 32499 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:52.880888 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:52.882010 28235 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:12:52.881752 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:52.821599 32469 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:52.870107 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 82804.6us
I20260709 13:12:52.885241 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:52.885947 28761 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:52.801359 32506 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:52.781790 32470 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:52.840399 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:52.850996 32489 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:52.882624 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:52.874022 32310 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:52.904551 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:52.905535 28233 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:52.886466 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:52.906373 32469 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:52.890713 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:52.907397 28229 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:52.908481 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:52.907639 32469 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.909307 28229 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:52.910993 28229 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 13:12:52.896977 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 56653.4us
I20260709 13:12:52.872997 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:52.899326 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:52.915719 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:52.776515 32503 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.250s
I20260709 13:12:52.882587 28228 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:52.920387 32503 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:52.901511 28226 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:52.875187 32487 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:52.922392 32487 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:52.922808 32487 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:52.886546 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:52.923686 28235 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:52.925987 28228 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:52.900218 28231 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:52.864984 32393 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:12:52.828900 32023 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.886786 32418 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:12:52.901888 32399 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 13:12:52.890357 32506 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:52.895936 32501 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:52.937945 28235 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:52.978713 32515 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
W20260709 13:12:52.980953 32429 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.975s	user 0.013s	sys 0.003s
I20260709 13:12:52.982811 32508 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:52.983896 32420 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:52.983753 32426 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:52.984628 32514 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:52.938330 32375 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:12:52.984809 32511 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:12:52.903632 32489 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:52.985584 32509 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 13:12:52.990962 32418 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:52.989140 28160 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:52.899618 31801 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.936559 28233 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:52.898979 32455 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:12:52.894994 32477 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 13:12:52.988195 32501 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:53.009009 32206 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:53.013233 28755 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:12:52.985846 32476 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:52.986925 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73777.5us
I20260709 13:12:53.048838 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:52.987967 32506 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:53.017712 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 110737us
I20260709 13:12:52.938514 32292 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.055599 32393 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:53.056448 32393 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:53.057508 32292 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:52.988610 32399 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:53.059636 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.022630 32310 raft_consensus.cc:2764] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:12:52.996624 32429 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.063135 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:53.018947 32375 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:53.025163 32339 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:53.065745 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:53.051298 32501 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:53.045639 32417 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:53.057875 32393 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.075094 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:53.075354 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 202391us
W20260709 13:12:53.076665 32517 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:12:53.080888 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2371) took 1022 ms. Trace:
I20260709 13:12:53.067451 32476 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.081425 28154 rpcz_store.cc:276] 0709 13:12:52.058324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.058457 (+   133us) service_pool.cc:224] Handling call
0709 13:12:52.058765 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:52.745435 (+686670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.080622 (+335187us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b5400252e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:53.080824 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1443984128}
I20260709 13:12:53.096506 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:53.074251 32459 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:53.111294 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.111996 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:53.070274 32498 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:53.119040 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.085837 32413 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:53.115111 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.133683 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:53.134625 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.138402 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:12:53.144819 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.147893 28078 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:52.902071 32470 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.152701 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.157228 28228 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:53.153201 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.114295 28077 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
W20260709 13:12:53.153784 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 81406.4us
I20260709 13:12:53.106388 32508 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:53.160017 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:53.161017 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:53.164026 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:53.164584 28159 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:53.166342 28078 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:53.169165 28761 raft_consensus.cc:2368] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12.
I20260709 13:12:53.123538 32417 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:53.172209 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 58767.5us
I20260709 13:12:53.172433 28085 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:53.131660 32455 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:53.161144 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:53.092842 32476 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.180586 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.181182 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:53.181427 28231 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:53.182277 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:53.183360 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:53.185766 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.186506 28230 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:53.188512 28228 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:53.173822 32526 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:53.177623 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:53.175186 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:53.195984 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:53.196256 28159 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:12:53.196557 28154 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:53.197506 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:53.198259 28159 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:53.198163 32373 consensus_meta.cc:287] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.917s	user 0.011s	sys 0.000s
I20260709 13:12:53.218250 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:53.236043 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 74965.6us
W20260709 13:12:53.236867 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59316.1us
I20260709 13:12:53.109126 32418 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.169399 32519 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.242087 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:53.183218 32455 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:53.249027 28154 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:53.251600 28159 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:53.153093 32469 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 13:12:53.260267 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 78030.4us
I20260709 13:12:53.263221 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:53.256894 32519 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.264328 28050 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:53.179381 32475 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:12:53.179531 32474 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:12:53.185563 32417 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.241031 32373 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.237658 32471 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:12:53.241423 32535 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:53.169839 32375 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.278760 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:53.268805 28231 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:53.285503 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:53.243052 32518 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:53.254827 32455 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:53.325313 32417 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:53.326061 32536 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:53.326882 32543 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:53.329600 32291 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.000s
I20260709 13:12:53.329854 32532 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:53.330799 32523 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:12:53.336277 32459 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:53.337080 32471 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:53.336866 32339 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:53.339396 32429 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:53.341629 32373 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:53.342378 32465 consensus_meta.cc:287] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.387s	user 0.013s	sys 0.000s
W20260709 13:12:53.284775 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:53.355424 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 76824.7us
I20260709 13:12:53.405962 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:53.317175 32281 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:53.371417 32469 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:12:53.397593 32440 consensus_meta.cc:287] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.979s	user 0.008s	sys 0.004s
I20260709 13:12:53.370301 28230 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:53.270295 32447 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:53.261791 32510 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:53.420713 28157 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:53.397748 32459 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:53.285746 32457 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.398106 32471 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:53.257860 32333 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.371346 32420 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:12:53.407243 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 142982us
I20260709 13:12:53.286598 32477 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:53.412992 28052 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:53.415266 32469 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:53.406728 32543 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:53.429591 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 187574us
I20260709 13:12:53.430047 32546 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.446261 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:53.466713 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53807.7us
I20260709 13:12:53.259159 32520 raft_consensus.cc:2764] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:12:53.467585 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:53.468240 32291 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:53.468773 32291 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:53.469310 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:53.469694 32291 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.470296 32547 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:53.343286 32399 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:53.472790 32518 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:53.465888 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:53.473461 32518 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:53.474436 32518 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.398829 32536 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.466912 32546 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.454102 32498 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:53.419298 32281 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:53.480975 32519 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:53.420446 32447 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:53.481964 32546 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:53.481895 32281 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.482825 32546 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:53.483196 32546 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:53.483577 32546 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:53.420145 32279 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:53.485105 32279 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:53.486011 32279 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.423946 32544 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:53.489037 32543 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:53.490530 32333 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:53.401651 32532 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.491446 28082 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:53.492303 32476 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:53.492717 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:53.493234 32471 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.494288 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:53.494521 32471 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:53.495039 28158 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:53.496285 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:53.497463 32489 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.466038 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:53.499622 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:53.499994 32489 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:53.500252 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.500746 32520 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:53.502154 32448 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.126s	user 0.017s	sys 0.000s
I20260709 13:12:53.502496 32532 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:53.505533 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.460167 32420 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:53.536043 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.539472 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.539795 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.536147 32420 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:53.540167 28234 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:53.541540 28757 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:53.543133 28156 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:12:53.544260 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.546126 28228 raft_consensus.cc:2410] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:12:53.546736 28234 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:12:53.548846 28234 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: [11.6-12.7]
I20260709 13:12:53.549649 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:53.552841 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.549691 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.569274 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.461882 32474 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:53.575791 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.570497 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.577035 28226 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:12:53.575445 32469 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.578452 32426 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:53.579576 28763 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:12:53.579484 32426 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.459046 32457 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.553805 32519 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:12:53.482514 32447 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.582535 32447 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:53.582798 32514 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.002s
I20260709 13:12:53.583281 32514 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:53.411013 32465 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.585389 28233 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:53.586458 32465 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:53.587242 32455 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.587255 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.589007 28230 raft_consensus.cc:2368] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10.
W20260709 13:12:53.589793 32206 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.531s	user 0.005s	sys 0.005s
I20260709 13:12:53.571156 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.572702 28231 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 7. Current term is 8. Ops: []
W20260709 13:12:53.517241 32424 consensus_meta.cc:287] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.621s	user 0.005s	sys 0.003s
W20260709 13:12:53.544262 32499 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.644s	user 0.006s	sys 0.004s
I20260709 13:12:53.574440 32440 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:53.574743 32390 raft_consensus.cc:670] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:53.462949 32477 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:53.575650 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.656236 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.656594 28235 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:53.500622 32489 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:12:53.590298 32506 consensus_meta.cc:287] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.526s	user 0.005s	sys 0.003s
I20260709 13:12:53.590507 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.669476 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:53.591746 32495 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.854s	user 0.008s	sys 0.001s
I20260709 13:12:53.588268 32459 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.582821 32457 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:53.593297 32519 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:53.669397 32560 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.593844 32546 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:12:53.595019 32487 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.000s
I20260709 13:12:53.596071 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:53.596107 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54592.1us
I20260709 13:12:53.755014 28760 raft_consensus.cc:2368] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13.
I20260709 13:12:53.681186 32476 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:53.657500 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.655071 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 1971) took 1083 ms. Trace:
W20260709 13:12:53.663043 32501 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.586s	user 0.007s	sys 0.008s
I20260709 13:12:53.665351 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:12:53.665978 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.667147 28198 leader_election.cc:400] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12.
I20260709 13:12:53.612915 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:53.760066 28198 leader_election.cc:403] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:12:53.760391 28154 raft_consensus.cc:2368] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:53.575981 32470 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:53.591516 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.781596 28233 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:53.655714 32440 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:53.698192 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 228925us
I20260709 13:12:53.784862 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:53.785995 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:53.689886 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.688347 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:53.689141 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:53.690433 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:53.690968 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:53.794937 28758 raft_consensus.cc:2368] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:53.703492 28761 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:53.681669 28763 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:53.689841 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:53.793157 28227 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:53.796440 28759 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:53.681293 28230 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:53.664362 32477 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:53.655535 32333 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:53.671072 32532 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.707430 28226 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
W20260709 13:12:53.715909 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.611624 32499 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.663751 32373 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:53.663844 32489 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:53.753355 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.571652 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:53.760835 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 102731us
I20260709 13:12:53.759069 28081 rpcz_store.cc:276] 0709 13:12:52.571690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.571855 (+   165us) service_pool.cc:224] Handling call
0709 13:12:53.610535 (+1038680us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b54000c80d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:53.654994 (+ 44459us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1504941440}
I20260709 13:12:53.833714 28227 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:53.825568 28230 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:12:53.841395 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.842830 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:53.844120 28231 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:53.510208 28755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
W20260709 13:12:53.781236 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 259062us
I20260709 13:12:53.783144 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:53.845393 28755 raft_consensus.cc:606] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:53.669878 32310 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 13:12:53.796466 28153 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
W20260709 13:12:53.846737 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:53.848395 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50921.1us
I20260709 13:12:53.600811 32474 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.703574 32547 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:53.807178 28234 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 9: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:53.725661 32495 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:12:53.593263 32426 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:53.725571 32526 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:53.883288 28081 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:53.756402 32544 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:53.891510 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.845749 28755 raft_consensus.cc:614] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:53.855804 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:53.892459 28764 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:53.893415 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:53.895383 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.685271 32506 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.897497 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:53.855880 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.898952 28084 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:12:53.899852 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:53.900533 28086 raft_consensus.cc:2410] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:12:53.907701 28231 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:53.854349 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:53.886663 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:53.826009 32390 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:53.919361 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:53.826308 32424 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:53.892264 28227 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:53.884306 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:53.898120 28077 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:53.914256 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:53.755944 32519 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.939597 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:53.807686 32470 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:53.782176 32476 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.917683 28083 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:12:53.920089 28081 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:53.806824 32510 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:53.853560 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:53.783031 32339 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 13:12:53.925098 28085 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:53.847229 32489 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:12:53.942237 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 58021.8us
I20260709 13:12:53.946341 28231 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:53.947237 28226 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:53.891710 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:53.781242 32501 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:53.762095 32487 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.038825 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:53.947075 32552 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:12:53.848875 32333 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.040378 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:54.058423 28084 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:54.039206 28154 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:54.058768 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:53.849107 32532 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.052829 28153 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 8.
I20260709 13:12:53.764247 32560 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:53.882560 32373 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.080312 32373 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:54.064149 28230 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:53.950906 32448 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:54.066133 28079 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:54.046561 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:54.085217 28080 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:54.101197 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:54.102180 28154 raft_consensus.cc:2368] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 10.
I20260709 13:12:54.067689 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:54.071962 28756 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:54.076880 32554 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:12:54.078248 32559 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:12:54.108958 28127 leader_election.cc:400] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10.
I20260709 13:12:54.108903 32554 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:54.110137 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:54.111127 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:54.111855 32548 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:54.112466 32548 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:54.113324 28230 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:54.114796 28230 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:54.115178 32557 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:54.078377 32560 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:54.062525 32535 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.689s	user 0.006s	sys 0.004s
I20260709 13:12:54.128357 32535 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:54.053840 32457 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.129483 32457 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.132009 32457 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:54.132591 28228 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:12:53.941447 32390 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.089046 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:54.057018 28231 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:12:54.103544 28229 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:54.065428 32424 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:54.145534 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 302846us
I20260709 13:12:54.146826 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:54.147830 28154 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:53.915030 32544 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.116271 32426 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:12:54.116014 32557 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:54.102877 32510 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:54.185937 32471 consensus_meta.cc:287] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.691s	user 0.011s	sys 0.001s
I20260709 13:12:54.185768 32554 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.186803 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:54.187212 32534 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.187791 32534 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.188107 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:54.190644 32424 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.195196 28158 raft_consensus.cc:2368] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11.
I20260709 13:12:54.200829 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2409) took 1022 ms. Trace:
I20260709 13:12:54.201995 28161 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:54.182410 32447 consensus_meta.cc:287] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.001s
I20260709 13:12:54.201350 28159 rpcz_store.cc:276] 0709 13:12:53.178209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.258244 (+ 80035us) service_pool.cc:224] Handling call
0709 13:12:53.259088 (+   844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:53.601754 (+342666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:53.601782 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:53.610639 (+  8857us) log.cc:844] Serialized 26 byte log entry
0709 13:12:53.832744 (+222105us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:53.832799 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.832890 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:54.200545 (+367655us) raft_consensus.cc:1685] finished
0709 13:12:54.200575 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:54.200757 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2671,"raft.queue_time_us":6198,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":113,"thread_start_us":467605,"threads_started":2,"wal-append.queue_time_us":379426}
I20260709 13:12:54.210104 28153 raft_consensus.cc:2368] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 9.
I20260709 13:12:54.205950 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:54.212545 28154 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:12:54.217941 32563 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:12:54.219651 32506 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:54.210002 32459 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:54.221058 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:54.222023 28231 raft_consensus.cc:2410] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:12:54.208894 32524 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:12:54.222890 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:54.223304 28157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2373) took 2161 ms (client timeout 1938 ms). Trace:
I20260709 13:12:54.223495 28231 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:54.224078 32524 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:54.224558 32524 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:54.224713 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:54.225461 32524 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.226506 32532 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:54.223764 28157 rpcz_store.cc:269] 0709 13:12:52.061829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.061973 (+   144us) service_pool.cc:224] Handling call
0709 13:12:54.223107 (+2161134us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400254c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:54.223215 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3121627392}
I20260709 13:12:54.195207 32546 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:54.081831 32526 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.220399 32506 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:12:54.246119 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:54.211547 32390 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:54.247975 28157 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:54.248966 32506 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:54.249202 32487 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:54.249509 32506 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:54.250183 32476 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:54.250492 32506 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.251892 32506 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:54.253027 32510 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.256814 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:54.246265 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:54.257076 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:54.257579 28762 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:54.257649 28154 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:54.259966 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:54.260603 28154 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:54.261541 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:54.262753 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:54.263309 28154 raft_consensus.cc:2376] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12. Re-sending same reply.
I20260709 13:12:54.264156 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:54.269356 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:54.270215 28755 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:54.270442 28198 leader_election.cc:400] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:54.270918 28198 leader_election.cc:403] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:12:54.271474 28154 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:54.271646 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:54.290717 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:54.306800 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 598793us
I20260709 13:12:54.332270 28157 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:54.338788 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:54.336319 28759 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.535s	user 0.013s	sys 0.001s
I20260709 13:12:54.116438 32547 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.339699 28759 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:12:54.345506 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:54.347637 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:54.348578 28231 raft_consensus.cc:2376] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13. Re-sending same reply.
I20260709 13:12:54.266216 32499 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:54.332474 32291 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:54.338343 32476 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:54.376257 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:54.333154 32474 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:54.390226 28154 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 13:12:54.364887 32519 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:54.402284 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 63406.3us
W20260709 13:12:54.297394 28757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.001s	sys 0.009s
W20260709 13:12:54.403606 28757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.004s	sys 0.011s
I20260709 13:12:54.177682 32477 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.383566 32532 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:54.374801 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:54.419160 28154 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:12:54.384222 32291 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:54.434096 28050 leader_election.cc:336] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.041s (SENT)
I20260709 13:12:54.438686 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:54.443307 28154 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 13:12:54.419127 32532 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.460984 32544 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:54.334918 32548 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.490059 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 926) took 1400 ms. Trace:
I20260709 13:12:54.490589 28082 rpcz_store.cc:276] 0709 13:12:53.089189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.089655 (+   466us) service_pool.cc:224] Handling call
0709 13:12:53.089896 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:53.491908 (+402012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:54.489780 (+997872us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b5400209754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:54.489988 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":923007872}
I20260709 13:12:54.420503 32424 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.493050 32424 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:54.493721 32578 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:12:54.494091 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:54.472458 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:54.495922 32489 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:54.473189 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:54.497635 32424 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:54.498278 32424 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:54.498678 32424 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:54.499076 32424 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:54.499861 32579 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:54.500478 32579 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.502884 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:54.503016 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:54.415772 32393 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:54.454481 32499 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:54.467348 32291 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.482570 32534 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:54.476675 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:54.485828 32532 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:54.486114 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:54.534258 28235 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:54.473256 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:54.505357 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:54.507756 28231 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:54.509038 28199 leader_election.cc:400] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9.
I20260709 13:12:54.511638 32474 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:54.512778 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:54.510265 32544 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:54.519258 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:54.537038 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 60452.7us
I20260709 13:12:54.547389 32584 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:12:54.552866 32448 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:54.554981 32448 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:54.544078 32457 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:12:54.557281 32470 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.000s
I20260709 13:12:54.557763 32470 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12.
W20260709 13:12:54.556495 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47629.5us
I20260709 13:12:54.559067 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:54.560484 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:54.565596 28757 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:12:54.519881 32499 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:54.541019 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:54.514982 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:54.595731 28084 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:54.597455 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:54.598492 28084 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:54.599660 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 84795.7us
I20260709 13:12:54.600759 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:54.601972 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:54.602458 32568 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:54.603111 32568 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:54.603578 32568 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:54.606871 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:54.608263 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 67350.2us
I20260709 13:12:54.585388 28755 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:54.606869 32532 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:54.610458 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:54.612766 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:54.537359 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:54.617645 28154 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:54.610348 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:54.620630 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:54.630898 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 70569.4us
I20260709 13:12:54.632062 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:54.542421 32519 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:54.540061 32506 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.526374 32577 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:12:54.647424 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:54.648932 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:54.682220 28755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:12:54.567986 32023 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.677465 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:54.683831 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:54.542699 32560 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:54.684139 28154 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:54.613528 32499 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:54.688637 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:54.697937 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65987.8us
I20260709 13:12:54.615940 32532 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.627609 32474 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:54.616415 32476 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:54.703141 28199 leader_election.cc:400] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 10.
I20260709 13:12:54.711479 28199 leader_election.cc:403] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:12:54.644840 32546 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:54.704370 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:54.625324 32536 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:54.697062 32459 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:12:54.682121 32584 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.565179 32544 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.707458 32532 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:54.708242 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:54.675213 32508 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:12:54.744786 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:54.769884 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 224063us
I20260709 13:12:54.772078 28757 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:12:54.643580 32484 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:12:54.713858 32560 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:54.734653 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:54.695684 28231 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:54.772979 28124 leader_election.cc:400] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13.
I20260709 13:12:54.699146 32506 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.779111 28235 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:54.681936 32429 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:54.725235 32526 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:54.785053 28124 leader_election.cc:403] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:54.700156 32023 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.778249 28154 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:12:54.688503 32519 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.785902 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:54.827895 28084 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 8: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:12:54.750252 32503 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.027s	sys 0.015s
I20260709 13:12:54.604617 32568 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:54.831461 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 58639.1us
I20260709 13:12:54.832810 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:54.833194 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:54.833658 28235 raft_consensus.cc:2410] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:12:54.768092 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 265294us
I20260709 13:12:54.699594 32426 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:12:54.789592 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:54.739611 32536 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 10.
I20260709 13:12:54.506151 27972 maintenance_manager.cc:643] P fea5cdb8fcad442a901284de409f4a11: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 8.872s	user 0.211s	sys 0.084s Metrics: {"bytes_written":30448,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":23303,"dirs.run_cpu_time_us":527,"dirs.run_wall_time_us":23582,"drs_written":1,"lbm_read_time_us":422,"lbm_reads_lt_1ms":4,"lbm_write_time_us":274374,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":9,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":120405,"threads_started":2,"update_count":22,"wal-append.queue_time_us":164100,"wal-append.run_cpu_time_us":589,"wal-append.run_wall_time_us":1016890}
I20260709 13:12:54.750046 32474 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:54.826571 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:54.769945 32557 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.784248 32459 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:54.820129 32543 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.329s	user 0.005s	sys 0.015s
I20260709 13:12:54.836212 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:54.836680 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 47175.1us
I20260709 13:12:54.816682 32574 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:54.855211 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:12:54.902281 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66329.1us
I20260709 13:12:54.903534 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:54.832530 32503 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.028s	sys 0.015s
I20260709 13:12:54.816891 32587 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:54.829249 32534 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.792847 32515 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:12:54.855331 32420 consensus_meta.cc:287] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.285s	user 0.013s	sys 0.000s
I20260709 13:12:54.977356 32420 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.886667 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:54.981343 28235 raft_consensus.cc:2376] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13. Re-sending same reply.
I20260709 13:12:54.979197 32476 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:54.814975 32560 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:12:54.834668 32555 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:54.834985 32489 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:54.862076 32429 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13.
I20260709 13:12:54.922178 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:54.985323 28154 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:54.853180 32597 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:54.907135 27983 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 260) took 24067 ms (client timeout 15000 ms). Trace:
I20260709 13:12:54.987345 32597 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:54.987851 32597 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:54.989743 32589 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 13:12:54.991436 32596 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:12:54.932377 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 99251.6us
I20260709 13:12:54.900447 32557 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.996805 32557 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:54.934161 32423 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:54.998435 32423 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:54.998708 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 990) took 1862 ms. Trace:
I20260709 13:12:54.999260 28756 rpcz_store.cc:276] 0709 13:12:53.136260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.136424 (+   164us) service_pool.cc:224] Handling call
0709 13:12:53.153578 (+ 17154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:54.104098 (+950520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:54.998331 (+894233us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b5400439ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:54.998642 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2111441152}
I20260709 13:12:55.000934 32476 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.935667 32593 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.937117 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 848) took 1315 ms. Trace:
I20260709 13:12:54.853703 32508 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:54.853931 32474 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:54.831640 32526 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.795099 27980 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 12, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 12 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:54.908090 32543 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.968422 32515 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:54.968914 32534 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:54.968969 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:54.969403 32459 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:54.975888 32499 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:54.982913 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:55.009262 28232 rpcz_store.cc:276] 0709 13:12:53.621525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.056634 (+435109us) service_pool.cc:224] Handling call
0709 13:12:54.081421 (+ 24787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:54.203538 (+122117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:54.203588 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:54.203648 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:54.203693 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:54.936874 (+733181us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54001f54d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:54.936920 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:54.937057 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280250112}
W20260709 13:12:55.009944 28156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2405) took 2587 ms (client timeout 2041 ms). Trace:
I20260709 13:12:55.011948 28155 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:12:54.987416 27983 rpcz_store.cc:269] 0709 13:12:30.821262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.821508 (+   246us) service_pool.cc:224] Handling call
0709 13:12:54.888711 (+24067203us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:49.118925 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:49.119591 (+   666us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:49.119634 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:12:49.119667 (+    33us) tablet.cc:662] Decoding operations
0709 13:12:49.221740 (+102073us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:49.221861 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:49.221887 (+    26us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:12:49.246904 (+ 25017us) tablet.cc:701] Row locks acquired
0709 13:12:49.246965 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:12:49.247196 (+   231us) write_op.cc:270] Start()
0709 13:12:49.247339 (+   143us) write_op.cc:276] Timestamp: P: 1783602769247087 usec, L: 0
0709 13:12:49.247384 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 13:12:49.248219 (+   835us) log.cc:844] Serialized 8710 byte log entry
0709 13:12:49.435254 (+187035us) op_driver.cc:464] REPLICATION: finished
0709 13:12:49.701477 (+266223us) write_op.cc:301] APPLY: starting
0709 13:12:49.701580 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:49.888407 (+186827us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b1800030160. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59ce5f4190 00007f59ce5edae7 00007f59ce5ecdd9
0709 13:12:49.897705 (+  9298us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:49.898237 (+   532us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:50.034403 (+136166us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:50.069392 (+ 34989us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=44
0709 13:12:50.069479 (+    87us) write_op.cc:312] APPLY: finished
0709 13:12:50.070100 (+   621us) log.cc:844] Serialized 152 byte log entry
0709 13:12:50.070341 (+   241us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:50.070569 (+   228us) write_op.cc:454] Released schema lock
0709 13:12:50.070649 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:50.071115 (+   466us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:50.071148 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3159160,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":265813,"apply.run_cpu_time_us":10218,"apply.run_wall_time_us":369640,"num_ops":22,"prepare.queue_time_us":25629,"prepare.run_cpu_time_us":26915,"prepare.run_wall_time_us":208090,"raft.queue_time_us":26665,"raft.run_cpu_time_us":35312,"raft.run_wall_time_us":267704,"replication_time_us":157536,"spinlock_wait_cycles":5111680,"thread_start_us":276683,"threads_started":4,"wal-append.queue_time_us":81506,"wal-append.run_cpu_time_us":22608,"wal-append.run_wall_time_us":1741172}]]}
W20260709 13:12:54.906699 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 218118us
W20260709 13:12:55.016048 28153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2319) took 1525 ms (client timeout 1932 ms). Trace:
W20260709 13:12:54.993569 32588 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
W20260709 13:12:54.997601 32549 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.414s	user 0.007s	sys 0.006s
I20260709 13:12:54.999930 32423 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.001340 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 927) took 1911 ms. Trace:
I20260709 13:12:54.935350 27973 maintenance_manager.cc:419] P fea5cdb8fcad442a901284de409f4a11: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10109 bytes on disk
I20260709 13:12:55.123517 32526 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:55.001857 32519 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:55.002756 28234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2015) took 1434 ms (client timeout 1524 ms). Trace:
W20260709 13:12:55.124418 28234 rpcz_store.cc:269] 0709 13:12:53.568532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.570923 (+  2391us) service_pool.cc:224] Handling call
0709 13:12:55.002525 (+1431602us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400391cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:55.002671 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481402752}
I20260709 13:12:55.126251 27972 maintenance_manager.cc:643] P fea5cdb8fcad442a901284de409f4a11: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":331,"lbm_reads_lt_1ms":4}
W20260709 13:12:54.953828 32586 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:12:54.752974 32584 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.130010 32584 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.011298 27980 catalog_manager.cc:5697] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 1 to 6, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 6 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:55.150777 27980 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:55.153296 32607 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.153977 32607 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:55.160230 32558 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.081s	user 0.003s	sys 0.000s
I20260709 13:12:55.169275 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2428) took 1350 ms. Trace:
I20260709 13:12:55.024361 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:54.908877 32471 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:55.017917 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:55.172554 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 154688us
I20260709 13:12:55.006326 28154 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
W20260709 13:12:55.021443 28156 rpcz_store.cc:269] 0709 13:12:52.422982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.652767 (+229785us) service_pool.cc:224] Handling call
0709 13:12:55.009716 (+2356949us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b540010bb54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:55.009875 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1903256448}
W20260709 13:12:55.175625 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 151318us
W20260709 13:12:55.024329 28153 rpcz_store.cc:269] 0709 13:12:53.490768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.067309 (+576541us) service_pool.cc:224] Handling call
0709 13:12:55.015834 (+948525us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54001afed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:55.015983 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242482944}
W20260709 13:12:55.021076 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 118017us
W20260709 13:12:55.006845 32592 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 13:12:55.057444 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:55.015434 32534 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:55.033474 32508 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:12:55.116153 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:55.008693 32591 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:12:55.054275 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 319929us
I20260709 13:12:55.199898 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:55.035838 28079 rpcz_store.cc:276] 0709 13:12:53.089501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.089665 (+   164us) service_pool.cc:224] Handling call
0709 13:12:53.090114 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:54.081860 (+991746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.001121 (+919261us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b540017d254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:55.001285 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2244053248}
W20260709 13:12:55.013748 32424 consensus_meta.cc:287] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 13:12:55.010025 32474 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:55.120564 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:55.177822 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:55.208942 28159 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:55.177891 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:55.007802 32544 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:55.014420 32506 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.014443 32546 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:55.013052 32543 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.169890 28230 rpcz_store.cc:276] 0709 13:12:53.818869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.115412 (+296543us) service_pool.cc:224] Handling call
0709 13:12:54.115689 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:54.336757 (+221068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:54.336789 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:54.344206 (+  7417us) log.cc:844] Serialized 26 byte log entry
0709 13:12:54.626080 (+281874us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:54.626117 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:54.626193 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:55.168960 (+542767us) raft_consensus.cc:1685] finished
0709 13:12:55.168990 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.169194 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84282,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":13923,"thread_start_us":387859,"threads_started":2,"wal-append.queue_time_us":395884}
I20260709 13:12:55.183064 28156 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:12:55.241394 32601 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.242203 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.242043 32601 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.242590 32465 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:55.243038 32465 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.243372 32465 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:55.243440 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.244088 28156 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:12:55.246160 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2025) took 1662 ms. Trace:
I20260709 13:12:55.246577 28229 rpcz_store.cc:276] 0709 13:12:53.584209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.945165 (+360956us) service_pool.cc:224] Handling call
0709 13:12:53.945852 (+   687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:54.136926 (+191074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.245903 (+1108977us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54003b3b54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:55.246097 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362144128}
I20260709 13:12:55.196695 32574 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:55.190937 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:55.115159 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:55.226686 27980 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 5 to 8, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 8 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:55.251804 32574 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.116761 32597 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.252684 28232 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:55.203166 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:55.255162 28077 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:55.172147 32607 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:55.256378 32607 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.036136 32557 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.158s
I20260709 13:12:55.258160 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.223384 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 797) took 2062 ms. Trace:
I20260709 13:12:55.190344 31801 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:55.111720 32476 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.231568 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:55.009130 32593 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.236976 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2420) took 1116 ms. Trace:
I20260709 13:12:55.204396 32595 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:55.254530 32597 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.174300 32471 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.256955 32607 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.278700 28229 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:55.230625 32544 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.201871 32560 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:55.244513 32601 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:55.268006 32557 consensus_queue.cc:1243] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:12:55.281039 32471 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:55.268178 31801 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:55.234570 32604 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.271997 28156 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:55.013746 32515 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.190702 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:55.214478 32426 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:55.274596 28157 rpcz_store.cc:276] 0709 13:12:54.120381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.500346 (+379965us) service_pool.cc:224] Handling call
0709 13:12:54.500670 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:54.501026 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:54.501054 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:54.501530 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:12:54.700532 (+199002us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:54.827355 (+126823us) log.cc:844] Serialized 14 byte log entry
0709 13:12:54.827585 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:54.827672 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:55.236704 (+409032us) raft_consensus.cc:1685] finished
0709 13:12:55.236746 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.236899 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6210,"prepare.queue_time_us":231187,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":56,"raft.queue_time_us":7831,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":21680,"thread_start_us":285992,"threads_started":2,"wal-append.queue_time_us":305552}
I20260709 13:12:55.305384 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.306092 28157 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:55.307520 28157 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:12:55.031759 32587 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.258977 32546 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:55.259677 32506 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.286410 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.336329 28081 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:55.264460 28152 rpcz_store.cc:276] 0709 13:12:53.160663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.172750 (+ 12087us) service_pool.cc:224] Handling call
0709 13:12:53.173083 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:53.173420 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:53.173456 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:53.179953 (+  6497us) log.cc:844] Serialized 26 byte log entry
0709 13:12:53.401772 (+221819us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:53.401823 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:53.401892 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:54.222301 (+820409us) spinlock_profiling.cc:243] Waited 9.41 ms on lock 0x7b480024c258. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:12:54.996583 (+774282us) spinlock_profiling.cc:243] Waited 71.7 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:55.223091 (+226508us) raft_consensus.cc:1685] finished
0709 13:12:55.223117 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.223308 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":433,"raft.run_cpu_time_us":15803,"raft.run_wall_time_us":1045374,"spinlock_wait_cycles":186639872,"thread_start_us":141522,"threads_started":1,"wal-append.queue_time_us":303931}
I20260709 13:12:55.214740 32424 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:55.298566 28049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.011s	sys 0.065s
W20260709 13:12:55.342615 28049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.011s	sys 0.066s
I20260709 13:12:55.299932 28156 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:55.301120 28153 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:55.347199 28156 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:55.350502 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:55.338851 28159 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
W20260709 13:12:55.340428 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 166452us
I20260709 13:12:55.341068 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:55.355123 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.356081 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:55.356936 28154 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 8: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:55.361034 28077 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:55.364408 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:55.365363 28079 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:55.369501 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:55.370217 28077 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
W20260709 13:12:55.370874 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 257817us
I20260709 13:12:55.366725 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:55.372066 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:55.348034 28156 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.371114 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:55.373732 28077 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:55.374650 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:55.373221 28156 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:55.338179 32597 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:55.378747 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 178963us
I20260709 13:12:55.379468 28082 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:55.381289 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:55.382992 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:55.386796 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:55.389278 28153 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:12:55.295711 32544 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.402133 28077 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:55.404413 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:55.405171 32611 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:55.405808 32611 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.410485 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:55.413285 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:55.414186 28079 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:55.416278 28084 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:55.418818 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.421483 28728 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:55.422431 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:55.433866 28153 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:12:55.358387 32427 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.437546 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:55.339280 32506 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.442054 28156 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:12:55.317155 32426 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.445425 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:55.409708 32543 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:55.447070 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:55.459694 28757 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:55.305962 32604 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.435688 32614 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:55.281670 32595 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.447814 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:55.448313 32557 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:12:55.449218 32608 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:12:55.400877 32519 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:55.441049 32544 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:55.401887 32560 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.452838 28156 pending_rounds.cc:85] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 8
I20260709 13:12:55.504596 28156 pending_rounds.cc:107] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:12:55.459528 32543 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.456298 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:55.435122 32501 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.434738 32423 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:55.461923 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 795) took 2295 ms. Trace:
I20260709 13:12:55.464818 32499 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:55.445655 31788 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:55.449090 32594 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:55.467365 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:55.465197 28759 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
W20260709 13:12:55.486420 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 182604us
I20260709 13:12:55.317479 32508 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.344319 31801 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.412638 32597 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.424084 32279 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:12:55.460095 32200 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:55.461566 32429 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:55.504905 28156 raft_consensus.cc:2889] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:55.559610 28156 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:55.445539 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:55.561302 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 115817us
I20260709 13:12:55.563090 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:12:55.565721 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:55.568825 28156 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:55.497344 28757 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:55.461325 32614 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:55.511801 28152 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:55.256886 27980 catalog_manager.cc:5697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:55.461915 32607 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:55.515580 28160 rpcz_store.cc:276] 0709 13:12:53.160102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.162083 (+  1981us) service_pool.cc:224] Handling call
0709 13:12:53.162384 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:53.162708 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:53.162736 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:53.174063 (+ 11327us) log.cc:844] Serialized 26 byte log entry
0709 13:12:55.237516 (+2063453us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b18001a5fc0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:12:55.424499 (+186983us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:55.451204 (+ 26705us) log.cc:844] Serialized 14 byte log entry
0709 13:12:55.451449 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.451530 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:55.451575 (+    45us) raft_consensus.cc:1685] finished
0709 13:12:55.455618 (+  4043us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.455757 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2012,"prepare.queue_time_us":27388,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":7846,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":9718,"spinlock_wait_cycles":24281216,"thread_start_us":186783,"threads_started":1,"wal-append.queue_time_us":2247959}
I20260709 13:12:55.445765 32503 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:12:55.513913 32543 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:55.456660 32459 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:12:55.464984 32476 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.457197 32427 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.500000 32519 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.533080 28153 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:55.566687 32429 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.572299 32632 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:55.572597 32520 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:55.573371 32606 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:12:55.559809 32555 consensus_meta.cc:287] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.003s
I20260709 13:12:55.574234 32616 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.126s
I20260709 13:12:55.580008 28154 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:12:55.653693 28154 pending_rounds.cc:85] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 6
I20260709 13:12:55.654079 28154 pending_rounds.cc:107] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:12:55.578083 32503 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:55.511901 32560 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.578243 32614 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:55.589329 28158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2314) took 2090 ms (client timeout 1788 ms). Trace:
I20260709 13:12:55.551194 32423 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:55.585987 32557 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:55.552060 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 855) took 1206 ms. Trace:
I20260709 13:12:55.592149 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:55.575121 32200 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:55.654335 28154 raft_consensus.cc:2889] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:55.582377 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:55.764786 28232 rpcz_store.cc:276] 0709 13:12:54.344625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.253738 (+909113us) service_pool.cc:224] Handling call
0709 13:12:55.254077 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.254287 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:55.378026 (+123739us) log.cc:844] Serialized 14 byte log entry
0709 13:12:55.550549 (+172523us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:55.550619 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:55.550680 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.550771 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.550960 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":261853,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":7161,"raft.queue_time_us":9069,"spinlock_wait_cycles":17081600,"thread_start_us":293010,"threads_started":2,"wal-append.queue_time_us":274784}
I20260709 13:12:55.656774 28049 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:55.590420 32424 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.658604 32609 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:55.659155 32627 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:55.588454 32593 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:55.600508 32632 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:55.600247 32429 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:55.602170 32520 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:55.661549 32625 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:12:55.663442 32474 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.589293 32607 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:55.667140 32587 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.668339 32426 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.670384 32427 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:55.676048 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 254606us
I20260709 13:12:55.696532 32595 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.598910 32519 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:55.698908 28158 rpcz_store.cc:269] 0709 13:12:53.489265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.496016 (+  6751us) service_pool.cc:224] Handling call
0709 13:12:55.579065 (+2083049us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b5400252e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:55.579266 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605477632}
I20260709 13:12:55.668337 32597 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:55.700618 32503 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:12:55.677639 32555 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:55.652942 32611 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:55.742204 32532 consensus_meta.cc:287] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.956s	user 0.011s	sys 0.001s
W20260709 13:12:55.767629 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 110970us
I20260709 13:12:55.781852 32609 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:55.788975 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 332719us
I20260709 13:12:55.601755 32605 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.340s
I20260709 13:12:55.770884 32557 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.794027 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:55.827319 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 235275us
I20260709 13:12:55.770058 32574 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:55.775486 32629 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:55.777690 32627 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:55.815527 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 856) took 1462 ms. Trace:
I20260709 13:12:55.816419 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2429) took 1318 ms. Trace:
I20260709 13:12:55.790304 32506 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:55.711311 32601 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:55.821326 28125 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:55.822307 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:55.822546 32623 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:12:55.828140 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:55.793443 32333 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:55.815730 32471 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.002s
I20260709 13:12:55.861711 32471 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.864113 32471 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.865018 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:55.776352 32423 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.870426 32474 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:55.834328 28235 rpcz_store.cc:276] 0709 13:12:54.353530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.279370 (+925840us) service_pool.cc:224] Handling call
0709 13:12:55.301667 (+ 22297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.301908 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:55.458464 (+156556us) log.cc:844] Serialized 14 byte log entry
0709 13:12:55.815079 (+356615us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:55.815139 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:55.815227 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.815307 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.815466 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":245117,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":107,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":4173,"thread_start_us":477642,"threads_started":2}
I20260709 13:12:55.834460 28232 rpcz_store.cc:276] 0709 13:12:54.497567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.792217 (+1294650us) service_pool.cc:224] Handling call
0709 13:12:55.815838 (+ 23621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.816037 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:55.816072 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:55.816114 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:55.816148 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.816202 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.816336 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.838591 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:55.838713 32634 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.838891 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:55.838670 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:55.835695 32508 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:55.831226 32557 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:55.848125 32546 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.503s	user 0.007s	sys 0.000s
I20260709 13:12:55.848495 32630 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:55.882721 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:55.849038 32631 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:12:55.824360 32532 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:55.851388 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:55.853549 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:55.856702 28199 leader_election.cc:400] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 7. Current term is 9.
I20260709 13:12:55.858340 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:55.852814 32514 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:12:55.851943 32601 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:55.890172 28127 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:55.851569 32632 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.860312 32607 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:55.830680 32629 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.880781 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:55.904619 28760 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 8.
I20260709 13:12:55.859375 32595 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:55.833787 32200 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.874684 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2431) took 1376 ms. Trace:
W20260709 13:12:55.884807 32625 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:12:55.793378 27980 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:55.883119 32630 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.897617 28758 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:12:55.904376 32607 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:55.902236 32603 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:12:55.886732 32546 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.905953 28232 rpcz_store.cc:276] 0709 13:12:54.498153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.873722 (+1375569us) service_pool.cc:224] Handling call
0709 13:12:55.874088 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.874343 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:55.874374 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:55.874417 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:55.874459 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.874513 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:55.874621 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.830775 32606 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.907341 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:55.907573 28235 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:12:55.904374 32629 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:55.873776 32638 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:12:55.887122 32465 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.643s	user 0.012s	sys 0.001s
I20260709 13:12:55.908411 31801 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:55.853479 32520 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.878309 32506 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.876744 32423 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.879990 32474 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.877276 32609 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:55.930302 32474 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.932648 32474 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:55.900696 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46805.9us
I20260709 13:12:55.876497 32560 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:55.913648 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0566035690001172s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
I20260709 13:12:55.915247 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2430) took 1417 ms. Trace:
I20260709 13:12:55.933707 28235 rpcz_store.cc:276] 0709 13:12:54.497891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.880314 (+1382423us) service_pool.cc:224] Handling call
0709 13:12:55.907264 (+ 26950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.914967 (+  7703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.915180 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.915714 28759 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:12:55.888823 32508 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:55.935127 28235 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:55.935679 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2433) took 1437 ms. Trace:
I20260709 13:12:55.935982 32519 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.935616 32508 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.936919 32560 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:12:55.937259 28127 leader_election.cc:336] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.650s (SENT)
I20260709 13:12:55.937728 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:55.936213 28235 rpcz_store.cc:276] 0709 13:12:54.498749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.934498 (+1435749us) service_pool.cc:224] Handling call
0709 13:12:55.934838 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.935474 (+   636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.935631 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.911325 32471 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:55.915462 32632 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:55.924791 32595 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:55.917200 32574 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:55.939427 28127 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.653s (SENT)
I20260709 13:12:55.972824 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:55.946815 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.971449 28232 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:12:55.963517 28231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2034) took 1695 ms (client timeout 1650 ms). Trace:
W20260709 13:12:55.997320 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57923.5us
I20260709 13:12:55.998606 28153 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:55.929232 32455 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:55.995837 32520 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:55.925860 32465 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.946990 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.891645 32634 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.946403 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.874832 32627 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.974606 32628 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.946199 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:56.005187 28160 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:55.954505 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:55.997792 28231 rpcz_store.cc:269] 0709 13:12:54.267952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.349682 (+ 81730us) service_pool.cc:224] Handling call
0709 13:12:55.963280 (+1613598us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b54003caad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:55.963408 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205696768}
I20260709 13:12:56.009622 28156 raft_consensus.cc:2410] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:12:55.962929 32506 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:55.939970 32560 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:55.970949 32597 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.999071 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2432) took 1500 ms. Trace:
I20260709 13:12:56.013424 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.972306 32471 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:55.941788 32519 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.017633 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:55.939693 32447 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:56.021004 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:55.998528 32474 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:56.005128 31801 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:55.920892 32606 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.031096 28153 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.023008 28232 rpcz_store.cc:276] 0709 13:12:54.498424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.924005 (+1425581us) service_pool.cc:224] Handling call
0709 13:12:55.971126 (+ 47121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.998780 (+ 27654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.999016 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:55.923230 27980 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 8, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:56.013473 32455 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.020251 32532 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.033460 32471 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:12:56.045758 28080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 930) took 2938 ms (client timeout 1985 ms). Trace:
I20260709 13:12:55.999285 32574 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:55.964331 32601 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.020280 32560 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:56.019773 32640 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:56.052124 32634 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.035357 32627 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:56.049453 28080 rpcz_store.cc:269] 0709 13:12:53.095808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.096165 (+   357us) service_pool.cc:224] Handling call
0709 13:12:56.034144 (+2937979us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54002ad354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:56.034295 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816690432}
I20260709 13:12:56.010643 32543 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.057653 32630 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.053083 32455 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:56.061261 32641 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.073s	user 0.001s	sys 0.001s
I20260709 13:12:56.061576 32546 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:56.062278 32639 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:12:56.113147 28158 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 13:12:56.063208 32644 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:12:56.063511 32574 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.068941 32601 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.122376 28760 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:12:56.122500 32455 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.073909 32465 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.026469 32607 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.075167 32471 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.050561 31801 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.085937 32420 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.078428 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2434) took 1579 ms. Trace:
I20260709 13:12:56.032995 32597 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.084426 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.088747 27980 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 10, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 10 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:56.091773 32506 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.132537 28231 rpcz_store.cc:276] 0709 13:12:54.499040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.011091 (+1512051us) service_pool.cc:224] Handling call
0709 13:12:56.011415 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.068170 (+ 56755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:56.068201 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.068244 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:56.068277 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.078078 (+  9801us) spinlock_profiling.cc:243] Waited 56.5 ms on lock 0x7b54003b4554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:56.078163 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.078354 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129909120}
I20260709 13:12:56.105388 28153 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:56.107831 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.135879 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:56.136412 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.136673 28153 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:56.137272 28080 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:56.138336 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.114799 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:12:56.116775 28764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35896 (request call id 921) took 2450 ms (client timeout 1689 ms). Trace:
I20260709 13:12:56.118095 32477 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:12:56.127029 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:56.143597 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.144666 28157 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:56.144914 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.146955 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:56.147750 28083 raft_consensus.cc:2368] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
I20260709 13:12:56.148160 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.128620 28232 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:56.128477 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:56.113801 32634 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:56.092311 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.085826 32503 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.028992 32628 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.138589 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.140394 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.141213 32506 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:12:56.139137 28764 rpcz_store.cc:269] 0709 13:12:53.660947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.661114 (+   167us) service_pool.cc:224] Handling call
0709 13:12:56.111506 (+2450392us) spinlock_profiling.cc:243] Waited 98.4 ms on lock 0x7b54004d21d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:56.111637 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226356096}
I20260709 13:12:56.148458 28153 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:56.150050 28159 raft_consensus.cc:2410] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:12:56.150871 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.106725 32640 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.162097 28231 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:56.162634 28159 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:12:56.152093 32279 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:12:56.164067 32543 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.165045 31801 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.159190 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:56.167430 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.127839 32629 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.158478 28081 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:56.168798 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:56.162412 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:56.187786 28080 raft_consensus.cc:2376] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 12. Re-sending same reply.
I20260709 13:12:56.169512 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.191468 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:56.192448 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.193677 28232 raft_consensus.cc:2410] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:12:56.195708 28157 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:56.199476 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.197412 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:56.203764 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.200006 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.204517 28231 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:56.204723 28232 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:56.204125 28080 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.211809 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.213131 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:56.213709 28764 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.167666 32477 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:56.216841 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.187714 32455 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:56.215422 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.217593 28231 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:56.217742 28230 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:12:56.170020 32465 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:56.221799 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.154038 32614 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.225303 28235 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:56.226560 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2436) took 1485 ms. Trace:
I20260709 13:12:56.226948 28235 rpcz_store.cc:276] 0709 13:12:54.741413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.009445 (+1268032us) service_pool.cc:224] Handling call
0709 13:12:56.029564 (+ 20119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.225721 (+196157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.226324 (+   603us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54001f4ad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:56.226500 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449650560}
I20260709 13:12:56.222115 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:56.228679 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.232532 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:56.233348 32632 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:56.233839 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:56.234033 32630 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.234660 28159 raft_consensus.cc:2368] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9.
I20260709 13:12:56.234874 32448 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.235550 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:56.235463 32448 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.237879 32448 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.238101 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:56.238940 32474 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:56.239567 32604 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.240290 32447 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:56.241273 32569 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.241408 32447 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:56.203007 32310 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:56.223143 31801 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:56.207470 32546 raft_consensus.cc:2764] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:12:56.154515 32634 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.241601 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 853) took 1905 ms. Trace:
I20260709 13:12:56.254426 28229 rpcz_store.cc:276] 0709 13:12:54.336501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.247592 (+911091us) service_pool.cc:224] Handling call
0709 13:12:55.247922 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.279271 (+ 31349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.241380 (+962109us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b5400305554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:56.241546 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70249856}
I20260709 13:12:56.255841 28229 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:56.256065 28764 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:56.222671 28232 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:56.166646 32506 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:12:56.243983 28228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2010) took 2691 ms (client timeout 2015 ms). Trace:
I20260709 13:12:56.203171 32574 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:56.233845 32632 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:56.264884 28228 rpcz_store.cc:269] 0709 13:12:53.551172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.575547 (+ 24375us) service_pool.cc:224] Handling call
0709 13:12:56.242187 (+2666640us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b5400287154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:56.242280 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1278399744}
I20260709 13:12:56.265712 32632 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:56.267234 32550 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 2.686s	user 0.013s	sys 0.032s
I20260709 13:12:56.270455 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:56.271104 28158 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.154234 32560 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.271749 32560 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.274922 32632 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.234618 32630 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.276575 32469 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.277457 32632 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.277851 32630 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.278308 32628 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.246847 32429 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.248155 28159 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:12:56.244439 32310 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.241971 32569 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.284744 32310 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:56.284839 32390 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:56.286135 32525 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:56.286267 32569 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.286600 32525 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:56.287810 28728 leader_election.cc:400] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
I20260709 13:12:56.221877 32606 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.241896 32447 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:56.287842 32525 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.292618 32525 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:56.292853 32634 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:56.251420 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.264768 28081 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:12:56.292259 32447 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.294826 28080 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:56.288956 28728 leader_election.cc:403] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:12:56.295450 28153 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:56.296866 32524 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.298473 28156 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:56.302597 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.306800 28080 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:56.280248 32506 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:56.303475 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:56.308415 28153 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.312492 28158 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:56.314229 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:56.321156 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2421) took 1111 ms. Trace:
I20260709 13:12:56.321731 28154 rpcz_store.cc:276] 0709 13:12:55.210092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.372381 (+162289us) service_pool.cc:224] Handling call
0709 13:12:55.579550 (+207169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.765235 (+185685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.320872 (+555637us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400252e50. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:56.321070 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461798656}
I20260709 13:12:56.326857 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.330322 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.298583 32471 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.339000 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.340678 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.312085 32569 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
W20260709 13:12:56.280408 32474 consensus_queue.cc:1175] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "70996712c3c942468a6006602de6477f" responder_term: 10 status { last_received { term: 8 index: 9 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: [9.10-9.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:56.307689 32629 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.307744 32503 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:56.343894 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 82456.4us
W20260709 13:12:56.372587 28199 leader_election.cc:336] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.788s (SENT)
I20260709 13:12:56.373385 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.374322 32601 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:56.374833 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:56.375941 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:56.376575 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.351547 28153 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 9.
I20260709 13:12:56.325505 32447 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.354975 28154 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:56.379530 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.380342 28228 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:56.342057 32429 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.359134 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.341683 32628 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:12:56.360481 32647 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
W20260709 13:12:56.361310 32648 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:12:56.340848 28158 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:12:56.382336 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.394254 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.394894 28229 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.383179 28228 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:56.396718 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.400137 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:56.383276 32634 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:56.401175 32634 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.401503 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.402702 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.403403 28084 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:56.404120 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.405719 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 933) took 1837 ms. Trace:
W20260709 13:12:56.357205 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.369822 32640 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.409617 28229 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:56.411340 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.411994 28229 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.414757 28233 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:12:56.385890 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.381151 32310 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.385170 32628 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:56.406131 28082 rpcz_store.cc:276] 0709 13:12:54.568337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.569048 (+   711us) service_pool.cc:224] Handling call
0709 13:12:54.569488 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.379909 (+810421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.405478 (+1025569us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54002add54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:56.405660 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1862289664}
I20260709 13:12:56.383842 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.426358 32628 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:56.427457 28083 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: []
I20260709 13:12:56.407253 28084 raft_consensus.cc:2410] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:12:56.429059 32310 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.429224 28084 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:12:56.421828 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:56.416083 28080 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:56.430410 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:56.430897 28082 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:56.382517 32601 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:56.432971 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:56.332332 27980 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:56.436062 28229 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:56.437582 28080 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:56.437494 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.438719 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.440284 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.440986 28229 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:12:56.441344 28228 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:12:56.401549 32634 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.443396 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:12:56.453677 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.442283 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.451468 32429 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:56.455171 28229 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:56.455384 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.457506 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.464632 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.467864 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.442375 32447 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:56.442685 32601 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:56.495096 32601 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:56.497136 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:56.501330 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.466600 32429 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:56.502056 28235 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:56.503415 28755 raft_consensus.cc:606] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.503840 28755 raft_consensus.cc:614] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:56.444044 32560 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:56.487015 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:56.492808 32597 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:56.504420 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.465240 32465 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:56.510138 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:56.511561 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:56.512401 28764 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:56.469511 32569 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:56.503118 32447 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.532306 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:56.506682 32560 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.536612 28235 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:12:56.464790 32603 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.049s
W20260709 13:12:56.541923 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.542676 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.543334 28235 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:56.544263 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.545675 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.546319 28235 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:56.549326 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:56.510335 32465 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:56.551836 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.550717 32465 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.503564 32390 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:12:56.530463 32440 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.538061 32447 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:56.554183 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.561175 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:56.563131 32652 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.149s	user 0.002s	sys 0.000s
I20260709 13:12:56.548264 32560 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.566118 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:56.569396 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.571339 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:56.557273 32279 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:56.540417 32333 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:12:56.574285 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:56.575261 32518 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:12:56.531474 32429 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.577097 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:56.565078 32557 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.000s
I20260709 13:12:56.574285 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:56.592231 28084 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:56.555630 32629 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:12:56.540091 32543 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:12:56.581769 32545 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 3.000s	user 0.015s	sys 0.005s
W20260709 13:12:56.593590 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.594718 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:56.615285 32440 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.592990 32292 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:56.592691 32279 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:56.634218 28084 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:12:56.626789 32440 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.639335 32023 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.628189 32577 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:12:56.640018 32023 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:56.640443 32023 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:56.640862 32023 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.639145 32440 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.593972 32557 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.667636 28160 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:56.668824 32557 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.676116 28153 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.646437 32279 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.650622 28050 leader_election.cc:400] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9.
I20260709 13:12:56.682171 28050 leader_election.cc:403] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:56.624234 32627 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
W20260709 13:12:56.658578 28730 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.232s (SENT) [suppressed 35 similar messages]
W20260709 13:12:56.660898 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.663597 32601 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.664968 32629 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:56.636417 32534 consensus_queue.cc:1059] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 13:12:56.666484 28161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2329) took 2907 ms (client timeout 1786 ms). Trace:
W20260709 13:12:56.696447 28161 rpcz_store.cc:269] 0709 13:12:53.758637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.110800 (+352163us) service_pool.cc:224] Handling call
0709 13:12:56.666231 (+2555431us) spinlock_profiling.cc:243] Waited 90 ms on lock 0x7b54003444d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:56.666405 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207049472}
W20260709 13:12:56.697284 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46829.5us
W20260709 13:12:56.695649 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 377478us
I20260709 13:12:56.702464 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:12:56.675643 32653 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:12:56.704404 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:56.704703 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:56.708324 32614 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.708940 28761 raft_consensus.cc:2410] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:12:56.663645 32023 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.711120 32023 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.711584 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:56.713130 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:56.714020 28763 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:56.716837 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.717361 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:56.718062 28762 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.718431 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:56.719975 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:56.717586 28232 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:56.720932 28228 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.721177 27980 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:56.722674 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:56.723397 32579 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.725348 28760 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 8 LEARNER]: Advancing to term 9
I20260709 13:12:56.729264 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:56.731320 28153 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:56.746817 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:56.748008 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:56.748678 28159 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:12:56.745299 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:56.756572 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.757040 28158 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:56.757483 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:56.758513 28159 raft_consensus.cc:2368] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9.
W20260709 13:12:56.760833 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 60850.5us
I20260709 13:12:56.753909 32339 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:56.761667 32477 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:12:56.764806 32281 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:56.761910 32413 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.767141 32413 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.756946 32484 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.769037 32484 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.769404 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:56.769726 32413 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.770044 28158 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:56.771399 32484 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.763522 32339 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:56.772446 32628 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.730901 32534 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9.
I20260709 13:12:56.773780 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.773404 32339 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:56.774542 28231 raft_consensus.cc:2368] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
I20260709 13:12:56.774820 32628 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.775547 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.776156 28231 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:56.756134 32579 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:56.767225 32506 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.778623 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.781661 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:56.782588 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.783907 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.777063 32579 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:56.784655 28231 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:56.786710 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.787835 28231 raft_consensus.cc:2410] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 11 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:12:56.784641 28760 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 9 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:56.787994 28158 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:12:56.791837 28125 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:56.792318 28232 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:56.757681 32546 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:12:56.794955 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 199628us
I20260709 13:12:56.795449 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:56.798465 28232 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:56.798413 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:56.799750 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:56.774853 32339 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:56.802206 28229 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:56.796552 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:56.805495 32587 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:12:56.807062 32587 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:56.804354 32484 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:12:56.808109 32634 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.808501 32413 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:56.808246 32587 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.809062 32413 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:56.804275 32627 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:12:56.785748 32579 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:56.812065 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.761898 32626 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:12:56.798847 32546 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.810107 32413 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.814385 32634 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.814509 32413 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:56.815397 32534 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:56.815933 32601 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.816031 32534 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:56.816839 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.817163 32534 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.818151 32560 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.805248 32429 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:56.819532 32429 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:56.819975 32429 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.820341 32560 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:56.813561 32546 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:56.822878 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:56.804812 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:56.785707 32023 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:56.803462 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:12:56.864662 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 58402.9us
I20260709 13:12:56.813058 32375 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:56.723681 27980 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:56.805209 32489 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:12:56.798110 32628 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:56.899188 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 94482.4us
I20260709 13:12:56.857812 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:56.900573 28152 raft_consensus.cc:2368] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
W20260709 13:12:56.861989 32525 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.002s
W20260709 13:12:56.864815 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61425.5us
I20260709 13:12:56.853174 28125 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:56.870618 28228 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:12:56.856799 32465 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.911468 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:56.866075 32535 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:12:56.914112 28730 leader_election.cc:400] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11.
I20260709 13:12:56.913604 32447 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.915119 28760 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
W20260709 13:12:56.917333 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:56.852449 32560 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:56.924706 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2020) took 3333 ms. Trace:
W20260709 13:12:56.916699 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48965.2us
I20260709 13:12:56.916992 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:12:56.917227 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 91775.3us
I20260709 13:12:56.918474 28762 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:56.919312 28232 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:56.920079 28152 raft_consensus.cc:2410] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:12:56.923079 28760 pending_rounds.cc:85] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 9
I20260709 13:12:56.915987 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.865622 32375 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:56.931996 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:56.898819 32519 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:56.928359 28760 pending_rounds.cc:107] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:12:56.932268 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 1992) took 1569 ms. Trace:
I20260709 13:12:56.926721 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:56.915216 32601 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:56.934937 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.936314 28760 raft_consensus.cc:2889] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:56.937400 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:12:56.947724 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.925138 28226 rpcz_store.cc:276] 0709 13:12:53.582885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.827782 (+244897us) service_pool.cc:224] Handling call
0709 13:12:53.828103 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:54.038659 (+210556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.916257 (+2877598us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54003c9e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:56.916428 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273570176}
I20260709 13:12:56.951547 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:56.950225 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:56.927976 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:56.956987 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.957716 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.920974 32222 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:56.931450 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:56.960455 28233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 850) took 3119 ms (client timeout 2232 ms). Trace:
W20260709 13:12:56.962136 28233 rpcz_store.cc:269] 0709 13:12:53.841529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:53.922167 (+ 80638us) service_pool.cc:224] Handling call
0709 13:12:56.960272 (+3038105us) spinlock_profiling.cc:243] Waited 2.31 s on lock 0x7b54001f5ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:56.960384 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5315814656}
I20260709 13:12:56.960336 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:56.963755 28158 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:56.936391 28077 rpcz_store.cc:276] 0709 13:12:55.362815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.836227 (+473412us) service_pool.cc:224] Handling call
0709 13:12:55.836985 (+   758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.837443 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:12:55.852165 (+ 14722us) log.cc:844] Serialized 14 byte log entry
0709 13:12:56.717517 (+865352us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b1800390160. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a 00007f59ce681a2a
0709 13:12:56.931916 (+214399us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:56.931969 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:12:56.932013 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:56.932096 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:56.932212 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7348,"prepare.queue_time_us":14399,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":42,"raft.queue_time_us":84,"spinlock_wait_cycles":14176512,"thread_start_us":214173,"threads_started":1}
I20260709 13:12:56.968521 28760 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:12:56.954126 28226 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:12:56.971050 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:56.971781 28152 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:56.923983 32023 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:56.970774 32310 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:56.943269 32477 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:56.961395 32614 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:12:56.965382 32560 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:12:56.971747 32593 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:56.954746 32634 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:56.974076 32658 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:12:56.921872 32498 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:12:56.978425 27980 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 9, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 9 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:12:56.989938 28230 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:56.991398 28230 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:56.944818 32519 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:56.991915 32546 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:56.943427 32423 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:56.994371 32546 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:56.964958 32508 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:56.994951 32508 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.995018 32546 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:56.995141 32597 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:56.959172 32666 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:56.996325 32666 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:56.965832 32602 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:56.990731 27980 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:56.995466 32508 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:57.000489 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:57.000370 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:57.008160 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:56.999892 32663 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:56.992503 32519 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:57.012410 28230 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:12:57.015043 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:57.017247 28084 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:57.020560 28230 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:57.022571 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:57.009374 32663 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:56.996665 32666 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:57.028505 28081 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:57.031670 28226 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: [8.6-9.7]
I20260709 13:12:57.021936 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:57.032286 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:57.033011 28233 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:57.024910 32628 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:12:57.054459 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.064793 32538 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:57.065490 32538 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:57.066371 32538 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:57.064922 32663 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:57.069217 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:57.070080 28158 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:57.070443 28226 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:57.025473 32560 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 13:12:57.075726 32610 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.706s	user 0.006s	sys 0.000s
I20260709 13:12:57.073521 32515 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:57.074018 32506 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:56.998696 27980 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:57.090937 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:57.096969 28158 raft_consensus.cc:2410] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:12:57.073447 32469 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:57.074194 32628 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:57.050892 32413 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:12:57.077342 32546 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:57.099654 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:57.110260 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:57.112169 28153 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:57.113691 28152 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:12:57.087464 32536 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.118940 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:57.119129 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2342) took 1115 ms. Trace:
I20260709 13:12:57.122632 28156 rpcz_store.cc:276] 0709 13:12:56.004185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.024406 (+ 20221us) service_pool.cc:224] Handling call
0709 13:12:56.024830 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.306096 (+281266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.117599 (+811503us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400317ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:57.119067 (+  1468us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628813184}
I20260709 13:12:57.123296 28232 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:57.122637 28158 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:57.126204 28158 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:12:57.124599 28232 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:57.127920 28158 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:12:57.128104 28232 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:57.128458 28158 pending_rounds.cc:85] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 5
I20260709 13:12:57.129153 28158 pending_rounds.cc:107] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:12:57.129657 28158 raft_consensus.cc:2889] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:57.124580 28156 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:12:57.131127 28156 pending_rounds.cc:85] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 8
I20260709 13:12:57.131587 28156 pending_rounds.cc:107] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:12:57.132035 28156 raft_consensus.cc:2889] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:57.116020 28153 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:57.073249 32023 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:57.141618 28156 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:12:57.146293 32503 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:57.113399 32469 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:57.164601 32469 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:57.125730 32536 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:57.160493 32429 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.166664 32536 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:57.166797 32429 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:57.161173 27980 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:57.169111 27980 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:57.171051 27980 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 10, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 10 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:57.171648 32628 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:57.173195 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.175184 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2343) took 1170 ms. Trace:
I20260709 13:12:57.174789 27980 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 5 to 6, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 6 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:12:57.177265 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 852) took 2885 ms. Trace:
I20260709 13:12:57.177670 28234 rpcz_store.cc:276] 0709 13:12:54.292393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.125493 (+833100us) service_pool.cc:224] Handling call
0709 13:12:55.251627 (+126134us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:55.296411 (+ 44784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:55.296459 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:55.337786 (+ 41327us) log.cc:844] Serialized 26 byte log entry
0709 13:12:55.504565 (+166779us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:55.504640 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:55.504780 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:55.676906 (+172126us) raft_consensus.cc:1685] finished
0709 13:12:55.676929 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.177013 (+1500084us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54003b3b50. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:57.177210 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2180,"raft.queue_time_us":4704,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7615,"spinlock_wait_cycles":289235200,"thread_start_us":103463,"threads_started":1,"wal-append.queue_time_us":316412,"wal-append.run_cpu_time_us":2062,"wal-append.run_wall_time_us":1002094}
I20260709 13:12:57.181335 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2336) took 1235 ms. Trace:
I20260709 13:12:57.181792 28157 rpcz_store.cc:276] 0709 13:12:55.945635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.945804 (+   169us) service_pool.cc:224] Handling call
0709 13:12:57.181165 (+1235361us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400375fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:57.181276 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319215488}
I20260709 13:12:57.183619 28154 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:57.185925 28154 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:12:57.186770 32475 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.187321 32475 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:57.187605 32508 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.175581 28160 rpcz_store.cc:276] 0709 13:12:56.005440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.007067 (+  1627us) service_pool.cc:224] Handling call
0709 13:12:57.174975 (+1167908us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b5400255154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:57.175122 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1462741760}
I20260709 13:12:57.171978 32465 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:57.194049 28160 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:12:57.194165 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 199441us
I20260709 13:12:57.196241 28232 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:12:57.200910 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 179058us
W20260709 13:12:57.188508 32671 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:12:57.196794 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:57.153185 32503 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:57.205130 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:57.204277 28228 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:57.217727 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:57.218200 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2392) took 4895 ms. Trace:
W20260709 13:12:57.220625 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.220968 28762 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:12:57.218672 28155 rpcz_store.cc:276] 0709 13:12:52.319160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:52.756412 (+437252us) service_pool.cc:224] Handling call
0709 13:12:55.010706 (+2254294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:55.022910 (+ 12204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.214649 (+2191739us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b540010bb50. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:57.214786 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5183460224}
I20260709 13:12:57.222195 28155 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:12:57.219882 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 934) took 2647 ms. Trace:
I20260709 13:12:57.223376 28155 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:57.229904 32393 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:57.230194 32498 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:57.231735 32393 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:57.232115 32393 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:57.232944 32663 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:57.233187 32393 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.234575 32393 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:57.218360 32339 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.236093 32339 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:57.239082 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.222915 28086 rpcz_store.cc:276] 0709 13:12:54.568641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.568794 (+   153us) service_pool.cc:224] Handling call
0709 13:12:54.593457 (+ 24663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:54.854932 (+261475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:54.854962 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:54.855002 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:12:54.855029 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.215622 (+2360593us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54000e1854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:57.215666 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.215779 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598601472}
I20260709 13:12:57.253721 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:57.254283 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 57609.9us
I20260709 13:12:57.252588 32465 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:57.258327 32470 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:57.263258 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:57.263949 28155 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:57.259917 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:57.260890 32470 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:57.262326 32640 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:12:57.244658 32679 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:12:57.271874 32607 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:57.258713 32503 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.298169 28760 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:12:57.304940 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:57.305619 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:57.274932 32536 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:57.277046 32470 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:57.349160 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.350112 32556 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:57.350765 32556 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.320206 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:57.352779 28762 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:57.354871 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49994.5us
I20260709 13:12:57.357831 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:57.360440 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:12:57.325083 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.329561 28155 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:57.403204 28198 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:12:57.419344 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59027.6us
I20260709 13:12:57.420547 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:57.320144 32560 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.307356 32508 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:57.432442 28199 leader_election.cc:400] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 9 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10.
I20260709 13:12:57.322052 32339 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:57.302512 32626 consensus_queue.cc:1175] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "11f10c12c03e47dcac88b050d4c1431b" responder_term: 13 status { last_received { term: 12 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:12:57.323104 32475 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:57.319860 32498 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 13:12:57.418027 32536 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:57.424599 32676 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:57.455763 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:57.433641 32506 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:57.426033 32681 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:57.439184 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:57.460914 28756 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:12:57.423642 32526 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:57.443100 32546 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:57.468832 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 439644us
I20260709 13:12:57.419673 32666 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.486008 28762 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
W20260709 13:12:57.489908 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50808.9us
I20260709 13:12:57.490677 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:57.452899 32557 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.499404 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:57.457345 32475 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.501556 28155 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:12:57.507153 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:57.509330 28155 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:12:57.465693 32498 consensus_queue.cc:1243] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:12:57.448311 32560 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:12:57.474812 32676 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:57.489476 32506 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:57.525796 28155 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:12:57.508580 32399 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:57.498102 32579 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:57.531873 32399 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:57.527945 32526 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:57.553517 32399 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:57.554610 32526 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.533999 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:57.519948 32556 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:57.567893 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:57.508704 32557 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:57.516803 32475 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:57.495833 32681 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.566448 32498 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:57.623523 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:57.672082 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 320089us
W20260709 13:12:57.672914 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:57.619971 32680 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:12:57.671627 32515 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:57.702101 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.553426 32676 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.719686 28153 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:57.674510 32498 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.734513 32339 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:57.734963 32557 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:57.735253 32339 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:57.672474 32506 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:57.763064 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.777755 32597 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:57.787510 32597 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:57.788092 32023 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:57.788586 32399 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:57.790124 32023 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:57.790794 32399 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:57.790271 32597 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:57.794641 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2060) took 1161 ms. Trace:
I20260709 13:12:57.798208 32375 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.714569 32515 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.806861 32498 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:57.795245 28230 rpcz_store.cc:276] 0709 13:12:56.633061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.023916 (+390855us) service_pool.cc:224] Handling call
0709 13:12:57.024209 (+   293us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:57.024914 (+   705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:57.024944 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:57.025506 (+   562us) log.cc:844] Serialized 52 byte log entry
0709 13:12:57.212714 (+187208us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:57.212794 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.212886 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:57.794323 (+581437us) raft_consensus.cc:1685] finished
0709 13:12:57.794350 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:57.794558 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":345,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3826,"thread_start_us":186834,"threads_started":1,"wal-append.queue_time_us":250933}
W20260709 13:12:57.810336 32695 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:12:57.792876 32666 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:57.810216 32688 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 13:12:57.809832 32685 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.260s	user 0.002s	sys 0.000s
W20260709 13:12:57.810003 32693 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:12:57.806849 32515 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:57.820197 32684 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.266s	user 0.002s	sys 0.000s
W20260709 13:12:57.831375 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 262114us
I20260709 13:12:57.808884 32469 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:57.824800 32560 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:57.839140 28728 leader_election.cc:400] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9.
I20260709 13:12:57.840961 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 2449) took 1415 ms. Trace:
I20260709 13:12:57.850533 28728 leader_election.cc:403] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:12:57.850668 28229 rpcz_store.cc:276] 0709 13:12:56.425446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.457128 (+ 31682us) service_pool.cc:224] Handling call
0709 13:12:57.836578 (+1379450us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54002dafd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:57.840883 (+  4305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":777062656}
I20260709 13:12:57.852638 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:57.855225 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:57.855492 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:57.856287 28077 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:12:57.856794 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:57.857365 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:57.858897 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:57.858310 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:57.861512 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:57.861552 28077 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:57.862363 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:57.863734 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:57.867832 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:57.869971 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 943) took 1771 ms. Trace:
I20260709 13:12:57.870266 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:57.870572 28082 rpcz_store.cc:276] 0709 13:12:56.099273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:56.423486 (+324213us) service_pool.cc:224] Handling call
0709 13:12:56.430555 (+  7069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:56.431307 (+   752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.865237 (+1433930us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b54002ad350. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:57.869911 (+  4674us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15545728}
I20260709 13:12:57.869944 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:57.872118 28085 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:57.873633 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 197401us
I20260709 13:12:57.861016 32666 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.878715 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:57.879619 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:57.861958 32569 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9.
I20260709 13:12:57.865372 32699 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.865181 32023 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:57.867707 32339 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:57.851809 32469 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:57.895087 32554 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:57.863557 32393 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:12:57.867978 32510 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:57.866456 32475 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:57.885075 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:57.871764 32484 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:57.892381 28161 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:57.901059 28085 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:57.904191 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:57.904981 32393 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:57.910228 32569 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
W20260709 13:12:57.912657 32538 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.846s	user 0.012s	sys 0.012s
I20260709 13:12:57.930155 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:57.930821 28232 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:12:57.914352 32519 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.886s	user 0.011s	sys 0.003s
I20260709 13:12:57.914716 28077 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:12:57.916551 32469 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:57.917620 28158 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:57.908524 32470 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.559s	user 0.001s	sys 0.010s
I20260709 13:12:57.881458 32498 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:12:57.928012 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:57.929774 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:57.931116 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:57.885210 32666 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:57.892536 32699 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:57.961272 32687 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.142s	user 0.001s	sys 0.001s
I20260709 13:12:57.984061 28085 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:57.989868 28232 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:57.943472 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:57.931686 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 144016us
I20260709 13:12:57.932785 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:58.021540 28230 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
W20260709 13:12:58.022282 28152 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.001s	sys 0.032s
I20260709 13:12:58.022830 28157 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:57.994030 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:57.932876 28231 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:12:57.967195 28756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 751) took 2117 ms (client timeout 1645 ms). Trace:
W20260709 13:12:58.032652 28152 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.001s	sys 0.033s
W20260709 13:12:58.032392 28756 rpcz_store.cc:269] 0709 13:12:55.850099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:55.854698 (+  4599us) service_pool.cc:224] Handling call
0709 13:12:57.966941 (+2112243us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400436554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:57.967108 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3668247296}
I20260709 13:12:58.023424 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:58.018302 28229 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:58.021440 28161 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:58.034063 28230 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:12:58.028172 28234 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:57.998176 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:57.973449 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:47466 (request call id 6) took 3353 ms. Trace:
W20260709 13:12:58.035669 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 158014us
I20260709 13:12:58.035691 28082 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:12:58.037002 28728 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:58.035991 28755 rpcz_store.cc:276] 0709 13:12:54.612712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.612862 (+   150us) service_pool.cc:224] Handling call
0709 13:12:57.965672 (+3352810us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54004461d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce671853 00007f59d363f60a 00007f59ccbc57d4 00007f59ccbc5762 00007f59ccbc5691 00007f59ccbc560c 00007f59ccbc48c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:57.965842 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4186779904}
I20260709 13:12:57.973817 32503 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:57.967495 32520 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.957271 32681 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:57.968870 32597 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.048040 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:58.063616 28161 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:57.968163 32510 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.059650 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:57.932920 32538 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.067142 28158 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:12:58.067092 28233 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:58.049314 32518 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:12:58.060627 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 193478us
I20260709 13:12:58.061764 32702 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:58.071369 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:12:58.077327 28234 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:58.077788 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.027640 32519 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.081266 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2454) took 1128 ms. Trace:
I20260709 13:12:58.081843 32526 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.091378 32526 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:58.091969 28086 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:58.062525 32520 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:58.092883 28232 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:12:58.094234 32520 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:58.091188 28156 rpcz_store.cc:276] 0709 13:12:56.952485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.151403 (+198918us) service_pool.cc:224] Handling call
0709 13:12:57.190741 (+ 39338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:57.193476 (+  2735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:57.193519 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:57.202750 (+  9231us) log.cc:844] Serialized 26 byte log entry
0709 13:12:57.472352 (+269602us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:57.474874 (+  2522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.474954 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:57.788490 (+313536us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b480024c258. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:12:58.079956 (+291466us) raft_consensus.cc:1685] finished
0709 13:12:58.079983 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:58.081151 (+  1168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5192,"raft.queue_time_us":2335,"raft.run_cpu_time_us":10186,"raft.run_wall_time_us":589407,"spinlock_wait_cycles":6637440,"thread_start_us":269278,"threads_started":1,"wal-append.queue_time_us":520867}
I20260709 13:12:58.094830 32520 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:12:58.059811 32568 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:12:58.095504 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:58.048054 32339 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:12:58.069399 32597 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.083230 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:58.096987 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:58.097638 28234 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:12:58.095538 32520 consensus_queue.cc:1243] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:12:58.098281 32683 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.749s	user 0.003s	sys 0.002s
I20260709 13:12:58.098441 28156 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:58.099522 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:58.071877 32498 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:12:58.070887 32510 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:58.101054 28156 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:12:58.102627 32510 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:58.102728 28156 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:58.103217 32510 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:58.059741 32703 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:58.104166 32703 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:58.104601 32703 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:58.098323 32520 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:12:58.106297 32699 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:58.071446 32518 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:58.086621 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:58.109366 28231 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:58.109413 32518 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:58.110651 32518 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:58.087308 28235 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:58.112098 28235 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:58.084039 32666 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:58.048483 32708 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:12:58.113315 32666 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:12:58.113677 32666 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:58.092492 32519 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:58.096781 32526 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:58.082604 28230 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:58.110718 28231 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:58.114455 32666 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:58.115684 32666 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:58.025853 32470 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.118537 32470 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:58.119279 28228 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:12:58.120136 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.121762 32538 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:58.122373 28762 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: [12.12-14.13]
I20260709 13:12:58.125429 32476 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:58.131573 32700 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.332s	user 0.004s	sys 0.000s
I20260709 13:12:58.078768 28159 raft_consensus.cc:2376] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12. Re-sending same reply.
W20260709 13:12:58.143174 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 100224us
I20260709 13:12:58.144049 28161 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
W20260709 13:12:58.147439 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 67489.9us
I20260709 13:12:58.152869 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 831) took 1054 ms. Trace:
I20260709 13:12:58.158552 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:58.125998 32476 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.162518 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:58.170117 32568 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:58.171060 32508 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:58.171326 32568 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:58.171496 32508 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:58.171821 32568 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:58.152210 32699 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.173017 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:58.176618 28157 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:58.180058 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:58.180763 28762 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:12:58.180985 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:58.181759 28755 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:58.183166 28227 raft_consensus.cc:2368] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:12:58.182147 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:58.231104 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 49022.2us
W20260709 13:12:58.232055 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:58.228976 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 45605.8us
W20260709 13:12:58.237871 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.240409 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:58.241070 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:58.241461 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:58.157392 32393 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:12:58.243149 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.244180 28077 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:12:58.157004 32547 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:12:58.245270 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:58.245822 28083 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:58.246441 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.248382 28078 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:12:58.248867 28077 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:12:58.250020 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.198035 32503 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:58.251485 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:58.253253 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:58.254801 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:58.255033 28081 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:12:58.255580 28079 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:58.251320 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:58.193210 32023 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:58.261024 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:12:58.233920 32716 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
W20260709 13:12:58.262400 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.243572 32597 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:58.264567 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 192290us
I20260709 13:12:58.237692 32628 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:58.169657 32710 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:12:58.262245 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:58.268442 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:58.271319 32614 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:58.256526 32554 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:58.259850 32023 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:58.258113 32568 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.232836 32508 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.251279 32503 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:58.235505 32469 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:12:58.229929 32703 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.275866 32459 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:58.269277 32547 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:12:58.281316 32709 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:12:58.263161 32470 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:58.285876 28762 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:58.284708 32023 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:58.286026 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:58.286580 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.281158 32597 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:58.296886 32519 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:12:58.297530 32686 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.727s	user 0.002s	sys 0.000s
I20260709 13:12:58.297935 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:58.305387 32503 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:58.300843 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:58.184073 32699 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.309208 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:58.364430 28755 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:12:58.312124 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:58.276197 32628 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:58.315409 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:58.266748 32526 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:12:58.321033 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:58.282902 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:58.283990 32447 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:58.362545 28077 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:12:58.313314 32459 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:58.319545 32469 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:58.318478 32508 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:58.367301 32568 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:58.314117 32547 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:12:58.432593 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:58.439109 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 156281us
I20260709 13:12:58.336814 32519 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:58.336476 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:12:58.447934 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 111525us
I20260709 13:12:58.449258 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:58.445709 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:58.450846 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.259136 32506 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:12:58.366804 32628 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:58.365185 32470 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:58.377485 32459 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:58.472297 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 830) took 1394 ms. Trace:
I20260709 13:12:58.472960 28160 rpcz_store.cc:276] 0709 13:12:57.077538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.194990 (+117452us) service_pool.cc:224] Handling call
0709 13:12:57.195278 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:57.201547 (+  6269us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:57.201586 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:57.202021 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:12:57.428475 (+226454us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:57.428546 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.428672 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:58.296995 (+868323us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:58.447207 (+150212us) raft_consensus.cc:1685] finished
0709 13:12:58.447231 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:58.472197 (+ 24966us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5863,"raft.queue_time_us":7048,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":891,"spinlock_wait_cycles":5334784,"thread_start_us":226038,"threads_started":1,"wal-append.queue_time_us":317252}
I20260709 13:12:58.455982 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:58.438030 32469 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:58.447467 32614 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:58.509490 32721 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:12:58.509742 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:58.510836 32678 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.084s	user 0.009s	sys 0.004s
I20260709 13:12:58.510914 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.508306 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:58.512041 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 942) took 1490 ms. Trace:
I20260709 13:12:58.513091 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.513439 28761 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 9 LEARNER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:58.513767 28083 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:58.513971 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.514679 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.514843 28085 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:58.515116 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1012) took 1259 ms. Trace:
I20260709 13:12:58.515275 28083 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:12:58.515504 28761 rpcz_store.cc:276] 0709 13:12:57.256125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.256299 (+   174us) service_pool.cc:224] Handling call
0709 13:12:58.513233 (+1256934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:58.513896 (+   663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:58.514945 (+  1049us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54008da250. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:58.515064 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2889799040}
I20260709 13:12:58.512403 28757 rpcz_store.cc:276] 0709 13:12:57.021282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.023992 (+  2710us) service_pool.cc:224] Handling call
0709 13:12:57.024239 (+   247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:57.025430 (+  1191us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:12:57.025459 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:57.025819 (+   360us) log.cc:844] Serialized 26 byte log entry
0709 13:12:57.257440 (+231621us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:12:57.257480 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.257542 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:58.510542 (+1253000us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b24000c93c8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4109c4 00007f59cb40eacb 00007f59cb5c9dc9 00007f59cb5c9a55 00007f59cb5c909d 00007f59ce1fb10e 00007f59ce21288c 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221
0709 13:12:58.511724 (+  1182us) raft_consensus.cc:1685] finished
0709 13:12:58.511749 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:58.511967 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":809,"raft.queue_time_us":2822,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":1218,"spinlock_wait_cycles":43722496,"thread_start_us":231344,"threads_started":1,"wal-append.queue_time_us":276929}
W20260709 13:12:58.521476 28227 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2035) took 4253 ms (client timeout 1653 ms). Trace:
W20260709 13:12:58.521884 28227 rpcz_store.cc:269] 0709 13:12:54.268222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:54.362359 (+ 94137us) service_pool.cc:224] Handling call
0709 13:12:58.521263 (+4158904us) spinlock_profiling.cc:243] Waited 3.81 s on lock 0x7b5400305f54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:58.521408 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8752761728}
I20260709 13:12:58.525205 32470 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:58.527830 32200 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.528451 32200 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.454089 32447 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:12:58.531874 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.538882 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:58.541306 28226 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:12:58.542156 28226 pending_rounds.cc:85] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 11
I20260709 13:12:58.542477 28226 pending_rounds.cc:107] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:12:58.542706 28226 raft_consensus.cc:2889] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:58.543624 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:58.545678 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:58.546636 28757 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:12:58.531082 32200 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:58.496340 32469 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:58.580271 28757 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:12:58.583676 28757 pending_rounds.cc:85] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 11
I20260709 13:12:58.584069 28757 pending_rounds.cc:107] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:12:58.586563 28757 raft_consensus.cc:2889] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:58.557098 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.489578 32607 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:12:58.550411 32547 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:58.602319 32547 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:58.542665 32702 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:58.550758 32506 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:58.582871 32676 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:58.576491 32614 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:12:58.595249 32447 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:58.631068 32725 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:58.631716 32725 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:58.632133 32725 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:58.552091 32519 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:58.640139 32722 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:12:58.644543 32708 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:58.645262 32708 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:58.646342 32708 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:58.651392 32689 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.970s	user 0.006s	sys 0.006s
I20260709 13:12:58.647989 32708 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:58.628201 32703 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:58.620044 32676 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:58.662681 32692 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.855s	user 0.000s	sys 0.009s
I20260709 13:12:58.682976 32676 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:58.669694 32597 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:58.685115 32597 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:58.686245 32597 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:58.670847 32569 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:58.688870 32569 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:12:58.688153 32597 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:58.675211 32519 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:58.718266 32666 consensus_meta.cc:287] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.602s	user 0.011s	sys 0.001s
I20260709 13:12:58.618185 32702 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.740412 32702 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:58.742995 32628 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.758491 32725 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:58.834512 28077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:12:58.837055 28077 raft_consensus.cc:606] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:58.838056 28077 raft_consensus.cc:614] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:12:58.838274 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.838697 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:58.840090 28079 raft_consensus.cc:2376] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11. Re-sending same reply.
I20260709 13:12:58.840534 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.841957 28078 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:12:58.843650 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.843940 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:58.844722 28081 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:12:58.846787 28085 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:12:58.847330 28085 pending_rounds.cc:85] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 6
I20260709 13:12:58.847770 28085 pending_rounds.cc:107] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:12:58.848151 28085 raft_consensus.cc:2889] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:58.851619 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 953) took 1365 ms. Trace:
W20260709 13:12:58.852167 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 402845us
I20260709 13:12:58.852377 28086 rpcz_store.cc:276] 0709 13:12:57.485833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.486267 (+   434us) service_pool.cc:224] Handling call
0709 13:12:57.506809 (+ 20542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:58.092424 (+585615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:58.850753 (+758329us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54002968d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:58.851317 (+   564us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1345225088}
I20260709 13:12:58.854300 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:58.868150 28082 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:12:58.870026 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2457) took 1916 ms. Trace:
W20260709 13:12:58.867941 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 342080us
I20260709 13:12:58.880923 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2453) took 1928 ms. Trace:
I20260709 13:12:58.881325 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 863) took 2180 ms. Trace:
I20260709 13:12:58.881570 28153 rpcz_store.cc:276] 0709 13:12:56.952148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.136460 (+184312us) service_pool.cc:224] Handling call
0709 13:12:57.136833 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:57.720206 (+583373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:58.880087 (+1159881us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b54001969d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:58.880642 (+   555us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1339857280}
W20260709 13:12:58.895679 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 413541us
I20260709 13:12:58.907480 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 859) took 2196 ms. Trace:
I20260709 13:12:58.870755 28154 rpcz_store.cc:276] 0709 13:12:56.953544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.186934 (+233390us) service_pool.cc:224] Handling call
0709 13:12:57.187296 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:57.187622 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:57.187649 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:57.188060 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 13:12:57.469927 (+281867us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:57.470433 (+   506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:57.471807 (+  1374us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:58.660846 (+1189039us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:12:58.869025 (+208179us) raft_consensus.cc:1685] finished
0709 13:12:58.869080 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:58.869681 (+   601us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":600,"raft.queue_time_us":1362,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":7839,"spinlock_wait_cycles":5150976,"thread_start_us":279823,"threads_started":1,"wal-append.queue_time_us":557881}
I20260709 13:12:58.153538 28152 rpcz_store.cc:276] 0709 13:12:57.098690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.198754 (+100064us) service_pool.cc:224] Handling call
0709 13:12:57.217079 (+ 18325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:57.217305 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:12:58.033493 (+816188us) log.cc:844] Serialized 14 byte log entry
0709 13:12:58.152323 (+118830us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:58.152408 (+    85us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:12:58.152478 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:58.152598 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:58.152797 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3322,"prepare.queue_time_us":576917,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":6015,"raft.queue_time_us":3454,"thread_start_us":909968,"threads_started":2}
I20260709 13:12:58.881778 28226 rpcz_store.cc:276] 0709 13:12:56.700933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.081775 (+380842us) service_pool.cc:224] Handling call
0709 13:12:58.531372 (+1449597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:58.543339 (+ 11967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:58.880768 (+337429us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400305f50. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:58.881083 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3288596736}
I20260709 13:12:58.908195 28233 rpcz_store.cc:276] 0709 13:12:56.686838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.034421 (+347583us) service_pool.cc:224] Handling call
0709 13:12:57.839322 (+804901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:58.068935 (+229613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:58.426550 (+357615us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54002dafd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:58.882588 (+456038us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54002dafd0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:58.882949 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2279939712}
I20260709 13:12:58.615193 32506 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:58.947636 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:58.948199 32726 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:58.948832 28227 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:58.949162 32726 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:58.958446 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.960649 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:58.962458 28226 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:58.960904 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 421958us
I20260709 13:12:58.970469 32727 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:12:58.972940 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.980792 32506 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:58.986443 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:58.987778 28232 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:12:58.991075 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:58.991892 28232 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:58.993503 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 380372us
I20260709 13:12:58.997270 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:58.997535 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:58.999076 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:58.999642 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:59.000089 28229 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:12:59.000805 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:12:59.002166 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.002763 28231 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
W20260709 13:12:58.974627 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.005093 32498 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.005381 28233 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:12:59.002723 28052 connection.cc:256] RPC call timeout handler was delayed by -1.0792273740000837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18200000000000002s for a response
I20260709 13:12:58.960867 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:58.975211 32733 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:59.005998 32498 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.008388 28086 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:12:59.003527 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:59.010339 28229 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:12:59.000820 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 878) took 1506 ms. Trace:
W20260709 13:12:59.013039 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.014568 32733 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:59.014914 28229 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:59.015384 32676 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:59.016127 32676 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 12 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:59.018708 28229 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:59.011488 28228 rpcz_store.cc:276] 0709 13:12:57.494018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.724215 (+230197us) service_pool.cc:224] Handling call
0709 13:12:57.726256 (+  2041us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:58.119704 (+393448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:58.911641 (+791937us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54001c75d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:12:59.000610 (+ 88969us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":903251840}
I20260709 13:12:59.016129 32733 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.028509 28232 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:12:59.029529 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2360) took 1152 ms. Trace:
I20260709 13:12:59.030200 28158 rpcz_store.cc:276] 0709 13:12:57.876700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:57.879341 (+  2641us) service_pool.cc:224] Handling call
0709 13:12:59.029000 (+1149659us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b54000a2dd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:12:59.029193 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81196800}
W20260709 13:12:59.035115 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:59.035384 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.010900 32739 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.037756 32739 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.010824 32498 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:59.037102 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:59.053205 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53761.5us
I20260709 13:12:59.055446 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:59.064349 32457 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:59.065124 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:59.065634 32520 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:12:59.062796 28762 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:12:59.072845 32476 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.073274 28198 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:59.073705 28226 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:59.074132 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:59.075297 28763 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:12:59.024518 32676 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.097126 32731 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:59.104113 32731 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.065272 32457 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.117208 32733 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.121930 32733 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.127194 32733 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.128116 32733 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.129801 32739 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.131688 32739 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:59.133121 32743 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:59.132416 32739 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.134171 32743 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.136312 32739 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:59.130234 32731 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:59.137935 28085 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:12:59.140046 32023 consensus_meta.cc:287] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.773s	user 0.006s	sys 0.012s
I20260709 13:12:59.130991 32708 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:59.110064 32520 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.115973 32727 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:59.115526 32726 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:59.133265 32733 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.124408 32476 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:59.174297 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:59.140646 32023 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.171013 32476 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:59.182807 32730 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:59.183463 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:59.188915 32520 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:59.190449 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 117296us
I20260709 13:12:59.192922 32708 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:59.195312 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 105918us
I20260709 13:12:59.195763 32457 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.170888 32727 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:12:59.197180 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:59.182890 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:59.204777 32470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.008s	sys 0.034s
I20260709 13:12:59.205399 32750 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:59.205921 32731 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:59.206416 32628 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.216660 28085 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:59.237742 28085 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:59.239679 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:12:59.239171 28085 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.221698 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:59.241320 28152 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:12:59.241550 32720 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:59.242569 32720 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:59.242792 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:59.223804 32727 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:12:59.212042 32470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.010s	sys 0.035s
I20260709 13:12:59.225786 32750 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.226825 32731 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.227622 32520 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:59.228150 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.229087 28082 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:12:59.229457 28157 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:59.230918 32375 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:59.231531 32459 consensus_meta.cc:287] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.737s	user 0.005s	sys 0.013s
I20260709 13:12:59.232452 28156 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:59.234066 28155 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: [7.7-10.9]
I20260709 13:12:59.233901 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:12:59.234700 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 62829.3us
I20260709 13:12:59.234768 32708 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:12:59.235586 32508 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.725s	user 0.008s	sys 0.008s
I20260709 13:12:59.236127 28125 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:59.206441 32739 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:59.176913 32743 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.182868 32547 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:12:59.203908 32476 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:59.253718 32526 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.269351 32730 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:12:59.254716 28198 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:59.255664 32726 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:59.277647 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:59.287032 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:59.289469 32724 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.289719 32739 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.290263 32724 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.294103 32739 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.296190 28159 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:12:59.296291 28761 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:12:59.296797 32469 consensus_meta.cc:287] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.007s
I20260709 13:12:59.296909 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:59.297173 32554 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:59.298969 32731 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.300082 28157 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:12:59.302328 28085 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:59.317904 28152 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:12:59.314651 28156 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
W20260709 13:12:59.315809 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 79962.1us
I20260709 13:12:59.325654 32676 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:59.325852 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:59.326261 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 71717.1us
I20260709 13:12:59.326874 28759 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 14 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:12:59.326542 32676 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.328811 32614 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:59.329764 32614 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.282140 32506 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.351023 28234 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:12:59.334882 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:12:59.340804 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.334869 28759 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:59.319403 32708 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:12:59.357149 32506 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:59.358737 32676 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:59.339567 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:59.333449 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:12:59.348857 32747 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.234s	user 0.000s	sys 0.002s
I20260709 13:12:59.275553 32459 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.341481 32470 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:12:59.334054 32597 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:12:59.379659 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.341176 32733 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.332717 32614 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:12:59.340410 32503 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.963s	user 0.011s	sys 0.008s
I20260709 13:12:59.341531 32724 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:59.347067 32750 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.346860 32699 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.365816 32547 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:12:59.384995 28159 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:59.380010 32676 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.378280 32506 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.322423 32469 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.341008 32554 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:12:59.413816 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 79097us
I20260709 13:12:59.414266 32547 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:59.414830 32547 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:59.415238 28730 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:59.337545 32730 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.345868 32508 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
W20260709 13:12:59.384017 32470 raft_consensus.cc:1068] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:12:59.349953 32526 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.427443 28760 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:59.411253 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:59.411446 32740 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:59.412184 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:59.412272 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73053.2us
I20260709 13:12:59.412947 32200 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:59.380838 32597 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:59.444377 28227 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:59.420116 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:59.477324 28760 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:12:59.478703 28086 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:12:59.421578 28080 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:12:59.404068 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:59.439800 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:59.417145 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:12:59.440096 28230 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:59.425510 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:59.420886 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:59.426142 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:59.484004 28085 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:12:59.394429 28125 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:59.413985 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:59.424809 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:59.440543 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:59.340564 32569 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.402547 32614 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.417833 32393 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:59.416255 32547 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.422959 32708 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.480322 28159 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 6.
I20260709 13:12:59.497958 28083 raft_consensus.cc:2368] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:12:59.418717 32607 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:59.529201 32607 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.531767 32607 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.405004 32023 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.484831 28085 pending_rounds.cc:85] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 7
I20260709 13:12:59.506454 28229 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:59.417479 32676 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.533010 28085 pending_rounds.cc:107] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:12:59.534394 28085 raft_consensus.cc:2889] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:59.534271 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:12:59.536022 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 56702.7us
I20260709 13:12:59.416318 32554 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.389838 32503 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.537112 32720 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:59.539486 32503 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:59.540292 32699 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:59.506827 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:12:59.549480 28230 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:59.481710 32597 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:59.465005 32200 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:12:59.507774 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 67389.9us
I20260709 13:12:59.508322 32614 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.509972 32547 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:12:59.510223 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 81851.9us
I20260709 13:12:59.510535 32708 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.511265 32726 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:59.422238 32520 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:59.517132 32569 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:59.467880 32459 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.517168 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:12:59.518692 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:12:59.417711 32447 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:59.471722 32506 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.477059 32740 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.509987 28755 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:12:59.548187 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:59.552431 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 187105us
I20260709 13:12:59.593099 28125 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:12:59.536218 32676 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:59.594331 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:59.594949 28226 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:12:59.595593 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.520957 32476 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.596998 32750 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:59.597431 32750 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:59.597775 32750 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:59.599123 32752 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:12:59.520716 32754 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.283s	user 0.000s	sys 0.002s
I20260709 13:12:59.536567 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:12:59.603374 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 66973.7us
W20260709 13:12:59.565964 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.533083 32706 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.606067 32706 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.518358 32469 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:59.577795 28086 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:59.608187 28764 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:59.563482 32200 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:59.608892 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:12:59.580324 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.564304 32743 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:59.560087 32628 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:59.506367 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:12:59.588884 28232 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:12:59.622542 32393 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:59.623706 32475 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:59.624907 32716 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:12:59.624933 32724 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:59.566607 32554 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:59.628690 32755 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.404s	user 0.001s	sys 0.000s
W20260709 13:12:59.629196 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.634182 28230 raft_consensus.cc:2410] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:12:59.636308 32725 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.637557 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:12:59.638659 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:12:59.640064 32614 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.646574 32708 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:59.651402 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 132909us
I20260709 13:12:59.652347 32510 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:59.652664 32718 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:12:59.655995 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 240932us
I20260709 13:12:59.596341 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:12:59.686647 28080 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:59.687013 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 90846.3us
W20260709 13:12:59.657577 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 49721.7us
I20260709 13:12:59.688805 32706 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:59.689334 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 97741.7us
I20260709 13:12:59.690043 32476 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.690215 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:59.690340 32676 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:59.691109 32676 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.692909 32413 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:59.692715 32716 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.693588 32413 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.695716 32716 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.696060 32743 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:59.696195 32413 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:59.617949 32526 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.666033 28761 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:59.667568 28233 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:12:59.578001 32520 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:59.586987 32447 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:59.676636 28085 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:12:59.660288 32628 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:59.662019 32475 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:12:59.694135 32763 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.080s	user 0.001s	sys 0.002s
W20260709 13:12:59.734294 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.716699 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:12:59.702843 32765 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:12:59.734319 28082 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:12:59.675858 32726 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:12:59.717283 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:59.707428 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:12:59.697765 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:59.702709 32740 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:12:59.719724 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:59.664525 32720 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 13:12:59.769865 28755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:12:59.696584 32743 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:59.771855 28755 raft_consensus.cc:606] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:59.775686 28755 raft_consensus.cc:614] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:59.685876 32718 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:12:59.762972 32447 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:59.701653 32554 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.762658 32628 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:59.726758 32506 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:59.714732 32676 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:59.697000 32699 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:59.696415 32716 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
W20260709 13:12:59.816057 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 127379us
I20260709 13:12:59.735481 32750 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.701242 32724 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.701313 32023 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:12:59.718858 32476 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:59.767553 32708 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:12:59.785611 32752 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:59.819147 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.859462 32725 raft_consensus.cc:2764] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 13:12:59.896569 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 81097.4us
W20260709 13:12:59.900238 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 183106us
I20260709 13:12:59.890710 32699 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:12:59.815809 32720 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:59.789284 32607 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:59.887503 32476 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:12:59.869096 32413 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:12:59.872159 32726 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.891549 32739 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:59.924285 28080 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
W20260709 13:12:59.924340 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 204735us
I20260709 13:12:59.892490 32469 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:59.931327 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 233709us
I20260709 13:12:59.918442 32708 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.938520 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:12:59.939309 32724 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:12:59.939942 32724 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:59.940609 32724 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:12:59.912955 32718 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.942541 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:12:59.947016 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:12:59.925736 32476 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:59.905401 32752 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.946261 32750 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:12:59.965976 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:12:59.924518 32503 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:12:59.963997 32375 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:12:59.957731 32200 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.983158 32200 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:12:59.984038 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.926931 32475 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:12:59.969470 32718 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:59.926774 32520 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:12:59.967453 32676 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:59.956383 32733 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:12:59.970296 32393 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:13:00.004272 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 61993.4us
I20260709 13:12:59.997500 32708 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:00.001230 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 63492.5us
W20260709 13:13:00.018677 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 76214.8us
I20260709 13:13:00.018865 28755 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:00.022127 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:00.010921 32393 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.016198 32750 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:00.029554 28728 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:00.033885 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:00.032629 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 70027.6us
I20260709 13:13:00.018447 32699 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.025588 32475 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:00.058097 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.032878 32393 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:00.061936 28125 leader_election.cc:400] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:00.062871 28125 leader_election.cc:403] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:13:00.035616 32750 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:00.065532 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:00.035229 32739 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:00.073843 32726 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:00.074337   312 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:00.073272 32750 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:00.050071 32554 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:00.076959   306 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:13:00.081285 32767 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.115s	user 0.001s	sys 0.001s
I20260709 13:13:00.083227 32752 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:00.084857   314 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:00.087082   314 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:00.047982 32708 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:00.068881 32508 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:13:00.093734 32739 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.094466 32413 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:00.073135 32597 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.032763 32725 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 13:13:00.106874 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 73434.9us
W20260709 13:13:00.108793 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 74986.5us
I20260709 13:13:00.072374 32520 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:00.117067 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 112831us
I20260709 13:13:00.091243 32375 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
W20260709 13:13:00.117170   316 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:13:00.118853   303 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:13:00.108835 32413 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:00.120913   305 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:13:00.120644   313 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:00.124931 32607 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:00.125643 32607 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.128230 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:00.128549 32607 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:00.129726   310 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:13:00.131517 32547 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
I20260709 13:13:00.132284 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:00.133633 32726 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:00.134845 32724 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:00.137653 28757 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:00.140018 32724 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:00.141551 32750 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.142990 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:00.163905 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.164251 28757 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:00.164637 28760 raft_consensus.cc:2368] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:00.165268 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.143657 32676 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:00.117674 32708 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.149466 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:00.144047 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:00.168761 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53289us
I20260709 13:13:00.170842 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:00.171504 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.111001   312 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:13:00.151309 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.151104 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.172989 28756 raft_consensus.cc:2410] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:13:00.105230 32554 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:00.173727 28755 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: [12.12-14.13]
I20260709 13:13:00.173852 32554 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.175246 32413 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.154798 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.177399 32469 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:00.161372 32519 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:00.178140 32469 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:00.178427 32519 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:00.179435 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.179544 32519 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.180233 28156 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:00.182550 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.183542 28082 raft_consensus.cc:2410] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:13:00.184710 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.185765 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.186062 32413 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:00.187083 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:00.187795 28083 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:00.188999 32519 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:00.189132 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.189531 32628 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:13:00.192257 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 71684.5us
I20260709 13:13:00.165374 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.167780 32750 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:00.197954 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.151531 32706 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:00.175726 32743 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.176316 32554 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:00.200390 32706 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:00.153795 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.191586 28761 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:00.162467 32597 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:00.210567 28085 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:13:00.209088 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.210711 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.226188 28760 raft_consensus.cc:2410] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:13:00.216091 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 60313.9us
I20260709 13:13:00.230880 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:00.225646 28081 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:00.235291 28757 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:00.242853 28083 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:00.247787 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.253799 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.254380 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.255079 28159 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:00.255527 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:00.256090 28152 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:13:00.258394 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:00.260569 28160 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:00.233560 32200 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:13:00.262049 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 107753us
I20260709 13:13:00.197402 32708 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:00.269625 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.208639 32743 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:00.275712 28152 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
W20260709 13:13:00.262177 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 92163.2us
I20260709 13:13:00.272153 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.212256 32733 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.224507 32547 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:00.289125 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:00.282933 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 258726us
I20260709 13:13:00.212836   313 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:13:00.212867   314 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:00.286912 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:00.308346 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:00.311681 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:00.312361 28157 raft_consensus.cc:2410] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:13:00.315778 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.286731 32547 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:00.289202 32200 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:00.320690 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 89961.9us
I20260709 13:13:00.302099   312 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:00.313759 32597 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:00.332713   319 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:00.333349   319 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.333745   319 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:00.337008   320 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:00.337690   320 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:00.338581 32666 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.057s	user 0.014s	sys 0.006s
W20260709 13:13:00.339084 32666 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.014s	sys 0.006s
I20260709 13:13:00.341113   313 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:00.341327 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.341626   313 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:00.342590   313 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.343626 28082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:00.344236 28082 raft_consensus.cc:606] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:00.344547 28082 raft_consensus.cc:614] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:00.346418 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:00.318626 32733 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.353005 32743 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:00.353574 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.354370 28082 raft_consensus.cc:2368] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:00.353734 32743 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:00.313138 32554 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:13:00.327628 32569 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.359223 32569 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:00.374749 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 109607us
I20260709 13:13:00.324033 32676 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:00.379132 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:00.380117 28082 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:00.319830 32727 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:13:00.381088 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:00.381645 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:00.385453   323 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.385316 32547 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.386173   323 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.386895 32547 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:00.389349 28079 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:00.390940 32676 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.355325   312 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:00.332715   321 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:00.361359 32733 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:00.398114 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 76514.6us
I20260709 13:13:00.369961 32597 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:00.405385 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:00.406203 28232 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
W20260709 13:13:00.397388 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 162731us
I20260709 13:13:00.405827 28124 leader_election.cc:400] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:00.408457 28124 leader_election.cc:403] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:00.392658 32200 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.410209   313 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:00.413255 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:00.414947 28230 raft_consensus.cc:2368] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:00.418784 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:00.418165 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:00.418782 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:00.417913 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:00.416649 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:00.409371 32447 consensus_meta.cc:287] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.004s
I20260709 13:13:00.425086 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:00.425428 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.427191 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:00.437387 28229 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:00.433822 28084 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:00.406096 32607 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.439273 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:00.401646 32727 consensus_queue.cc:1243] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:13:00.372715 32393 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.440899 28080 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:00.449934 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:00.424010 32628 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:00.421592 32733 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:00.476610 28154 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:00.439785 32475 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.480921 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:00.474037 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:00.463388 28231 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:00.463653 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 45849.5us
I20260709 13:13:00.484148 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:00.486099 28759 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:00.462275 32750 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:00.450668   318 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:00.497610 32569 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:00.436806 32554 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:13:00.440501 32447 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.496852 28082 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:13:00.506083 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.506247 28157 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:13:00.438866 32708 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:13:00.510502 28154 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:00.510967 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:00.523262 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.524981 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:00.485494 32503 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.521800 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:00.494513 32520 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:00.537303 32520 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:00.537874 32520 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:00.533061 32476 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.568s	user 0.010s	sys 0.005s
I20260709 13:13:00.540860 32503 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.541234 32476 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:00.542966   327 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:13:00.543445 32503 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:00.543720 32476 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:00.544549 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:00.546207 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:00.546720 32554 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:00.547238 32554 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:00.547621 32554 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:00.533738 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:00.547497 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:00.551671 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:00.508562   337 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.558720 28227 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:00.511442 32708 consensus_queue.cc:1243] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:13:00.561190 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:00.453835 32607 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.568660 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:00.569782 28154 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:00.463168 32743 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.534349 32750 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:00.561661 32413 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.503840 32475 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.577531 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:00.578935 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 114053us
I20260709 13:13:00.531462   323 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:00.593336 28231 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:13:00.581121 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.534960 32676 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:00.582443 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.596087 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.561733 32726 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:00.480506   320 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:00.566107 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69284.5us
I20260709 13:13:00.599217 28757 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:00.561568 32519 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:13:00.527930   318 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.605695 28152 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:00.588615 32727 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.585532 32750 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:00.607303 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 100453us
W20260709 13:13:00.609887 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 56598.6us
I20260709 13:13:00.613549 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.608157 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:00.533095 32597 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.610895 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:00.619747 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:00.610226 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.619527 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.621690 28160 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:00.619518 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.559574   337 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.508288   312 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:00.628907 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:00.598677 32666 raft_consensus.cc:2764] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:13:00.634728 28084 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:00.637436 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:00.620249 32750 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:00.615474 32726 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:00.624276 32739 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:00.598858 32413 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:00.650801 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 97438.3us
I20260709 13:13:00.654256 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.650817 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.627628 32393 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:00.754133 28160 raft_consensus.cc:2410] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:13:00.636642   335 raft_consensus.cc:2764] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:13:00.759780 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.775732 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.642939   323 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:00.764159 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 144481us
W20260709 13:13:00.778172   332 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 13:13:00.775473 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.662685 32739 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:00.806005 28161 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
I20260709 13:13:00.846485 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:00.632771 32727 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:00.879653 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 263889us
I20260709 13:13:00.857843 28157 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
W20260709 13:13:00.883255 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 80897.5us
I20260709 13:13:00.884251 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:00.885514 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:00.882642 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.845412 32666 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:00.889740 28760 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:00.772698 32520 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.903018 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:00.861126 32739 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:00.907194 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:00.899781   338 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:13:00.841118   323 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.917397 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:00.925326 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:00.925732   334 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.280s	user 0.000s	sys 0.000s
I20260709 13:13:00.879464 32023 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:13:00.928874 28160 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:00.975100 32475 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:00.939358 32706 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:00.990224 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 236030us
I20260709 13:13:00.926945 32708 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:13:00.910300 32726 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:00.995535 28154 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:00.918390 32607 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:01.005688 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 84379.3us
I20260709 13:13:01.007967 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:00.997777 32718 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.994s	user 0.015s	sys 0.000s
I20260709 13:13:01.021106 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:01.023330 28762 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:01.002539   336 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.393s	user 0.001s	sys 0.002s
W20260709 13:13:01.002748   342 log.cc:927] Time spent T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.108s	user 0.001s	sys 0.001s
I20260709 13:13:01.027858 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:01.003006   347 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:13:01.031456 32520 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.032645 28226 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:01.037678   328 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.677s	user 0.002s	sys 0.002s
I20260709 13:13:01.039580 28078 raft_consensus.cc:2410] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 13:13:01.057475 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 146758us
I20260709 13:13:01.058844 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:01.003674   319 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.002s
I20260709 13:13:01.044442 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.045512 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:00.999698   355 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:01.010262 32023 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:01.060622 28233 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:13:01.040365 32706 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:01.025216 32475 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:01.078665 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:01.084872 28234 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:01.051172 32413 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.110670 28157 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:13:01.087574 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.057139   337 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:01.091727   313 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.681s	user 0.007s	sys 0.005s
I20260709 13:13:01.094640 28226 pending_rounds.cc:85] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 3
I20260709 13:13:01.117969 28226 pending_rounds.cc:107] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:13:01.118268 28226 raft_consensus.cc:2889] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:01.082401   350 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:13:01.097270   354 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 13:13:01.097698 32556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.003s	sys 0.021s
I20260709 13:13:01.099478 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:01.101226 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:01.070359 32023 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:01.137934 28758 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:01.122331 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.156435 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:01.092329 32750 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.117525   355 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.167888 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.166429 28760 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:01.070616   319 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.173841 28157 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:01.123267 32718 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:01.178989 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 97641us
I20260709 13:13:01.151417 32023 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:01.164323   347 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:13:01.136487   350 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:01.184722 28153 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:01.168525 32727 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:01.198231 28757 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:13:01.200974 28762 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
W20260709 13:13:01.198715 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 147002us
I20260709 13:13:01.174612 32724 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:01.199229 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.206179 28157 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:01.135455   353 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:13:01.205520 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:01.148881 32475 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.216172 32475 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:01.163709   351 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:13:01.219429   319 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:01.220829 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.221021   314 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:01.221638   314 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:01.217446 32547 consensus_meta.cc:287] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.830s	user 0.014s	sys 0.000s
I20260709 13:13:01.222189   314 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:01.221740 28127 leader_election.cc:400] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:01.225317 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:01.222522 32699 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.226470 32699 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.217904 32750 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:01.228649 32699 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:01.229344 32750 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:01.230612   348 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:13:01.181301 32666 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:01.250711 28083 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:01.251178 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:01.286024 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.287048 28157 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:01.194680 32718 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.189958   355 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.299993 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:01.300079 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:01.302522 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:01.261790 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.254981   313 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.303436 28234 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:01.304669 28232 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:01.245369 32554 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.697s	user 0.007s	sys 0.007s
I20260709 13:13:01.344928 28154 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:13:01.346094 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:01.305394   323 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.307940 28157 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:01.309762 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.311728 28079 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:13:01.352984 28079 pending_rounds.cc:85] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 6
I20260709 13:13:01.353246 28079 pending_rounds.cc:107] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:13:01.353472 28079 raft_consensus.cc:2889] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:01.258461 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:01.259827 28156 consensus_meta.cc:287] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.638s	user 0.007s	sys 0.004s
I20260709 13:13:01.354579 28156 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:01.295985   364 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.297376 32666 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.366323 32727 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:01.367326 32727 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:01.367699 32727 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:01.368616 32727 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.370249 32727 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:01.372852 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.378173 28081 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:01.305250   350 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:01.383848   358 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:13:01.391395   355 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.338891   313 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.403067 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:01.404856 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 179633us
I20260709 13:13:01.406067 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:01.407236 28156 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:01.407428 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:01.347805 32554 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.410737 28231 raft_consensus.cc:2368] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:01.410830 28081 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:01.381445 32520 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.411412 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:01.363871 32607 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:13:01.417447 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:01.426477 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 120831us
I20260709 13:13:01.434702 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:01.394395   359 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:13:01.444159 28232 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:01.444598 28127 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:01.447322 32510 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:01.449136 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:01.467587 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.468084 28231 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:01.453557   373 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:01.454123 32554 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:01.454871 32718 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.396231   364 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.456386 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 971) took 1164 ms. Trace:
I20260709 13:13:01.438257   355 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:01.458566 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.461884 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.463181 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:01.479925 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:01.465447 28161 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:01.417730 32475 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:01.465714 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.413223   323 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:01.461886 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:01.474977 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.476769 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:01.475699 28078 rpcz_store.cc:276] 0709 13:13:00.291628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.293091 (+  1463us) service_pool.cc:224] Handling call
0709 13:13:01.456097 (+1163006us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54000c80d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:01.456278 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661544320}
I20260709 13:13:01.483821 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:01.485070 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.434376 32520 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.469250 32510 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:01.485953 28077 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:01.475796   365 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.491182 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:01.500767 32554 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:01.500907 32739 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.000s
I20260709 13:13:01.487710 32743 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.502236   364 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:01.502756 28234 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.524212 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.525071 28081 raft_consensus.cc:2410] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:13:01.469373   366 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:01.503242 32475 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:13:01.529393 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.510047 28153 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:01.511020 28762 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:01.489894 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:01.506503 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:01.518599 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.520226 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.522127 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:01.527143 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 92554.3us
I20260709 13:13:01.529299 28078 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:13:01.531741 28125 leader_election.cc:400] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:01.531847 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:01.532264 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:01.510792 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:01.534916 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:01.538057 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.538846 28077 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:01.484268 32726 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:13:01.519939 32547 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:01.541658 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.489825   355 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.528281 32607 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:01.533133   361 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:01.535514   323 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:01.528441   375 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:01.535970   364 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:01.554994 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:01.559914 28078 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.543308 32699 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:01.540046 32498 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.525728   365 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.561465   356 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:01.561986   374 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:01.562561   355 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.542474   366 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.563827 32510 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.624984   374 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:01.625486   374 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.565294   314 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.627934 28156 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:01.523943 32739 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.630348   366 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:01.630426 32739 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.556311 32554 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:13:01.567202 32520 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.548643   363 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:01.572633 28080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2357) took 1391 ms (client timeout 1538 ms). Trace:
W20260709 13:13:01.631974 28080 rpcz_store.cc:269] 0709 13:13:00.181152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.184409 (+  3257us) service_pool.cc:224] Handling call
0709 13:13:01.572427 (+1388018us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54002ace54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:01.572563 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587469184}
I20260709 13:13:01.633188 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:01.634719 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.635831 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.635936 32743 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:01.636400 28080 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:01.577142   323 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:01.581276 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.600900 28231 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:01.573110 32607 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.637032 32666 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:01.604600 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:01.639514   350 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:01.643150   363 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:01.646683   374 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:01.650697 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.602506 32699 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.559585   364 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:01.654193   380 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:13:01.655089 28158 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:01.655443   356 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.694514   356 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:01.695077   374 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.665134 32498 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.697312 28077 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:01.697767 32498 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.689116   314 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.655974   313 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.656792 28234 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:01.660617 28759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 765) took 1516 ms. Trace:
W20260709 13:13:01.662200   379 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:13:01.557928 32547 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:01.673555 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 131318us
W20260709 13:13:01.674388 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 167990us
W20260709 13:13:01.624172   378 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:13:01.708405 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 839) took 1522 ms. Trace:
I20260709 13:13:01.711303 28759 rpcz_store.cc:276] 0709 13:13:00.143820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.149553 (+  5733us) service_pool.cc:224] Handling call
0709 13:13:00.149916 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:00.489858 (+339942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.660237 (+1170379us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b5400654354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:01.660503 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754668928}
I20260709 13:13:01.660144 32023 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.687647 28233 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:01.689536 32699 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:01.692343   364 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:01.687286 32607 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.700683 28077 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:01.710790   320 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.703516 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.701787   365 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.738629 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 955) took 1182 ms. Trace:
I20260709 13:13:01.715344 28078 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:01.721841 28127 leader_election.cc:400] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:01.746860 28127 leader_election.cc:403] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:13:01.747977 28077 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:01.721851   313 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.716243 28155 rpcz_store.cc:276] 0709 13:13:00.186037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.186152 (+   115us) service_pool.cc:224] Handling call
0709 13:13:00.186481 (+   329us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:00.186863 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:00.186910 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:00.187500 (+   590us) log.cc:844] Serialized 52 byte log entry
0709 13:13:00.284958 (+ 97458us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:00.285504 (+   546us) log.cc:844] Serialized 14 byte log entry
0709 13:13:00.303122 (+ 17618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:00.303199 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:00.978866 (+675667us) spinlock_profiling.cc:243] Waited 9.81 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:01.702077 (+723211us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b24001b8a68. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4109c4 00007f59cb40eacb 00007f59cb5c9dc9 00007f59cb5c9a55 00007f59cb5c8fb0 00007f59ce23f488 00007f59ce23fbe8 00007f59ce217398 00007f59ce21276a 00007f59ce211bb6
0709 13:13:01.708208 (+  6131us) raft_consensus.cc:1685] finished
0709 13:13:01.708229 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:01.708353 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26071,"prepare.queue_time_us":201,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":20645,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":4178,"spinlock_wait_cycles":31626368,"thread_start_us":97039,"threads_started":1,"wal-append.queue_time_us":172323}
I20260709 13:13:01.733947 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:01.750303   313 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.751246 28762 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:01.708142 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 960) took 1071 ms. Trace:
W20260709 13:13:01.738284 28152 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2385) took 1159 ms (client timeout 1538 ms). Trace:
I20260709 13:13:01.753026   387 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
W20260709 13:13:01.753227 28152 rpcz_store.cc:269] 0709 13:13:00.580791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.582176 (+  1385us) service_pool.cc:224] Handling call
0709 13:13:01.737998 (+1155822us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b54001969d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:01.738200 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19873664}
I20260709 13:13:01.753901 32547 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:01.754535 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.755326 28156 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:01.755681 32607 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:01.757234 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:01.760070 28156 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:01.752722 28757 rpcz_store.cc:276] 0709 13:13:00.636634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.636866 (+   232us) service_pool.cc:224] Handling call
0709 13:13:00.646103 (+  9237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:01.200888 (+554785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.707828 (+506940us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54004d21d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:01.708066 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1268951808}
I20260709 13:13:01.763811 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.764483 28159 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:01.752537   313 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:01.766113 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.766721 28159 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:01.734802 32750 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:01.782238 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 244682us
I20260709 13:13:01.783289 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:01.785290 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:01.801550 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.802484 28154 raft_consensus.cc:2410] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:13:01.804261 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:01.808548   333 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.448s	user 0.016s	sys 0.004s
W20260709 13:13:01.809142   333 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.449s	user 0.016s	sys 0.004s
W20260709 13:13:01.809342   345 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.016s	sys 0.012s
I20260709 13:13:01.743353 28764 rpcz_store.cc:276] 0709 13:13:00.556559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.556718 (+   159us) service_pool.cc:224] Handling call
0709 13:13:00.568385 (+ 11667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:00.636036 (+ 67651us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:00.651185 (+ 15149us) log.cc:844] Serialized 14 byte log entry
0709 13:13:01.044668 (+393483us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:01.044741 (+    73us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:01.044823 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.738287 (+693464us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b5400563954. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:01.738381 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:01.738557 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14958,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":6386,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":9527,"spinlock_wait_cycles":140003200,"thread_start_us":268809,"threads_started":1,"wal-append.queue_time_us":468714}
I20260709 13:13:01.805150 28159 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
W20260709 13:13:01.810849 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 293155us
I20260709 13:13:01.812683 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.813408 28154 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:01.818390 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:01.822497 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:01.823196 28161 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:01.822546 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:01.825752 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:01.822409 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:01.827716 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.827984 28757 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
W20260709 13:13:01.811251   345 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.017s	sys 0.012s
W20260709 13:13:01.824294 28728 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.102s (SENT) [suppressed 9 similar messages]
I20260709 13:13:01.827498   313 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:01.832116 28160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33788
I20260709 13:13:01.837555 28160 raft_consensus.cc:606] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Received request to transfer leadership
I20260709 13:13:01.837927 28160 raft_consensus.cc:614] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:01.831660 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.842133 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:01.843366 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.844992 28230 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
W20260709 13:13:01.849370 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 316524us
I20260709 13:13:01.850069 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:01.850555 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:01.853012 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.851799 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:01.854209 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:01.867657 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.873315 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.875600 28154 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:01.835680   313 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:01.818836 32475 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:13:01.831633   388 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:01.863039 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:01.837200 32607 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:01.884478 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.836433 32726 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:13:01.889314 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.857504   387 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:01.856993   335 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:01.880134 32739 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:01.860198 32743 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:01.890053 28154 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:01.840103 32597 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:13:01.853032   359 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:13:01.905599 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:01.831692   363 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:13:01.896106 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 71835.9us
I20260709 13:13:01.894960   386 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:01.909778   386 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.910859 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.898245   395 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:01.898699   349 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:01.885677   313 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:01.897924 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:01.883599   388 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:01.903774 28050 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:01.907837 32023 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:01.912241 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.901144   312 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:01.928671 28082 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:01.929766 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.930430 28082 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
I20260709 13:13:01.931226 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:01.931792 28082 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:01.925398 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:01.932519 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:01.945735 32706 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:01.946972 32498 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:01.947563 32498 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:01.895612   351 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 13:13:01.949810 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51982.4us
I20260709 13:13:01.939409   388 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:01.912184   386 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:01.951776 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:01.952347   345 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:01.984508   345 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.984990   345 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:01.985277 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:01.985900 28082 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:01.958082   359 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:01.958246 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54571.8us
I20260709 13:13:01.988962   359 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:01.989360   359 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:01.989907 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:01.961169 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:01.961764 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:01.965485   400 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:01.991427 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.993435 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:01.994566 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.995592 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:01.996497 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:01.999150 28161 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
W20260709 13:13:02.001979 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.003281 28761 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:01.924202   349 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:01.994690   364 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.006775 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:02.009863 28161 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
W20260709 13:13:01.977280 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 100841us
I20260709 13:13:02.010131 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:02.005816 28082 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:02.011241 28125 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:02.013731 28082 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:13:02.027077 28082 pending_rounds.cc:85] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 8
I20260709 13:13:02.036314 28082 pending_rounds.cc:107] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:13:02.036686 28082 raft_consensus.cc:2889] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:02.025584 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:02.019888 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:01.948885 32498 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:01.992224   400 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.061040 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:02.023330 32023 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:02.020918   390 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:13:02.012323   349 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:02.037395 28154 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
W20260709 13:13:02.064953 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 63406.1us
I20260709 13:13:02.085326 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:02.089619 28161 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:02.012039   318 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:02.092633 28161 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:02.037784   387 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:02.057754 32739 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:02.061203   351 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:02.108038 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:02.069360   320 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:02.112280 32498 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:02.062969   395 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:02.060822   408 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:02.059520 32743 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:02.121841 28161 raft_consensus.cc:2376] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14. Re-sending same reply.
I20260709 13:13:02.116520 28154 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:02.131486 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:02.132213 28161 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:02.131958 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:02.132819 28154 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:02.133329 28161 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:02.133335   333 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.128247   395 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:02.144466   411 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:02.144783   333 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.147300   333 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:02.093559   321 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:02.058564 32699 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.147925   321 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:02.149530 32699 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:02.149919   387 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.150455 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:02.151130 28762 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:02.151353 32739 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.128399   408 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:02.152690 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:02.153265   408 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.143540 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:02.153935 28764 pending_rounds.cc:85] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 9
I20260709 13:13:02.157766 28764 pending_rounds.cc:107] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:13:02.158071 28764 raft_consensus.cc:2889] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:02.153290 28755 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:02.159528 28080 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:02.159829   359 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.160518 28152 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:02.144819   395 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:02.156447 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2393) took 1154 ms. Trace:
I20260709 13:13:02.165879 28158 rpcz_store.cc:276] 0709 13:13:01.002593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.002753 (+   160us) service_pool.cc:224] Handling call
0709 13:13:02.156215 (+1153462us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b54001973d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:02.156372 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1397592960}
I20260709 13:13:02.152208   400 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:02.153748   356 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.167846 28231 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:02.140151 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:02.168943   356 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:02.148454 28229 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:13:02.171196 28229 pending_rounds.cc:85] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 6
I20260709 13:13:02.171485 28229 pending_rounds.cc:107] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:13:02.171789 28229 raft_consensus.cc:2889] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:02.175897 28154 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 7.
I20260709 13:13:02.156903   388 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:02.130263   382 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.497s	user 0.005s	sys 0.000s
I20260709 13:13:02.087276 32023 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:02.292330 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 148902us
I20260709 13:13:02.289451 28755 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:02.293439 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:02.293219 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:02.297173 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 157166us
I20260709 13:13:02.298321 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:02.298609 28077 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 13:13:02.301973 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.141968   366 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.303470 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:02.301422 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 264061us
I20260709 13:13:02.320981 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:02.320624 28080 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:02.404549 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 83696.9us
I20260709 13:13:02.130558   313 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.405505 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:02.305353   333 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:02.408867 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.303383 32706 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:02.324965 32508 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:02.471174   415 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.472173   400 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:02.473073 32699 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:02.472874   400 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.473551 32699 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:02.474018 32699 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:02.475764   400 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:02.472874 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 67459.6us
I20260709 13:13:02.477404 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:02.478447 28125 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:02.471874   415 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.485801   364 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:02.488544   411 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:02.491644 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:02.492569 28080 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:02.495710   411 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:02.496529   411 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:02.426206   409 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:02.441475 32706 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:02.509081   371 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.853s	user 0.002s	sys 0.008s
I20260709 13:13:02.512408   388 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:02.303587   345 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.342895   366 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:02.515038 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 661492us
W20260709 13:13:02.515913 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 222760us
I20260709 13:13:02.517182 28125 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:02.511476 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2490) took 1417 ms. Trace:
I20260709 13:13:02.524920 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:02.485679   400 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:02.528262 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:02.536207 28759 raft_consensus.cc:2368] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:02.503690   409 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:02.514060 32706 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:02.539923 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:02.526594 28226 rpcz_store.cc:276] 0709 13:13:01.093879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.094004 (+   125us) service_pool.cc:224] Handling call
0709 13:13:01.094401 (+   397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:01.137056 (+ 42655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.137098 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:01.147291 (+ 10193us) log.cc:844] Serialized 52 byte log entry
0709 13:13:01.402684 (+255393us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:01.417270 (+ 14586us) log.cc:844] Serialized 14 byte log entry
0709 13:13:01.435068 (+ 17798us) log.cc:844] Serialized 14 byte log entry
0709 13:13:01.435308 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.435408 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:01.653302 (+217894us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b180031c840. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:13:01.656144 (+  2842us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b180031c840. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:13:02.508499 (+852355us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:02.511105 (+  2606us) raft_consensus.cc:1685] finished
0709 13:13:02.511152 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:02.511384 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18400,"prepare.queue_time_us":264582,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":77,"raft.queue_time_us":32281,"raft.run_cpu_time_us":520,"raft.run_wall_time_us":26542,"spinlock_wait_cycles":48098816,"thread_start_us":206920,"threads_started":1,"wal-append.queue_time_us":289867}
I20260709 13:13:02.540176 28156 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:02.544778 28229 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:02.467482   333 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.509153   415 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:02.404721   349 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.544018 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:02.548122 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:02.554737 28198 connection.cc:256] RPC call timeout handler was delayed by -1.2278286890000345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1755s for a response
W20260709 13:13:02.555421 28198 connection.cc:256] RPC call timeout handler was delayed by -1.0984744400000181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
I20260709 13:13:02.557039 28159 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:02.558235 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:02.558957 28156 raft_consensus.cc:2376] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 14. Re-sending same reply.
I20260709 13:13:02.560403 28125 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:02.563602 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 250641us
I20260709 13:13:02.565501 28229 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:02.546052   409 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:02.563609 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:02.568257 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:02.569952 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.485149 32743 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.558264 32676 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.582871 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:02.575484 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:02.583413 28198 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:02.550909   418 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:02.540959 32607 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 13:13:02.589293 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 291076us
I20260709 13:13:02.594172 28198 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:02.574011 32597 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:02.570767   359 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:02.575634   400 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:02.598870 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:02.603880 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:02.584782 32724 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:02.607002 28082 raft_consensus.cc:2410] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:02.596704 32676 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.588214   418 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:02.621639 28229 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:13:02.592433   320 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.566154   373 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:13:02.622735 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:02.566543   417 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:13:02.600852 32597 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:02.634167 28082 raft_consensus.cc:2410] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 12 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:13:02.599769   394 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:02.592083 32200 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:02.615128   363 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:02.615635   345 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:02.622591 32724 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:02.624449   400 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:02.628978   418 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:02.635672   364 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:02.600703   388 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:13:02.642638 32597 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:02.641589 32676 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:02.657815   340 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.015s	user 0.008s	sys 0.018s
W20260709 13:13:02.698256   340 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.008s	sys 0.018s
I20260709 13:13:02.652245 32724 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:02.610222   395 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.635777   320 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.655647   321 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.703831   364 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:02.676261 32023 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:02.735559 32498 consensus_meta.cc:287] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.609s	user 0.010s	sys 0.000s
I20260709 13:13:02.716588   349 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:02.738883   320 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:02.729664   366 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:02.719002   321 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:02.748610   333 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:02.800990 28763 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:02.778688   319 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:02.754206 32699 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.748178   411 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:02.817152 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:02.749855   364 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:02.818555 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:02.757211   400 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:02.889612 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 295501us
I20260709 13:13:02.892799 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:02.891745 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 316310us
I20260709 13:13:02.894285 28198 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:02.900908 28198 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:02.903380 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:02.898972 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:02.906985 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:02.908627 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:02.911796 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2083) took 2389 ms. Trace:
I20260709 13:13:02.886569   333 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.848760   356 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:02.888947   321 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.919643 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:02.912364 28227 rpcz_store.cc:276] 0709 13:13:00.513986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.514145 (+   159us) service_pool.cc:224] Handling call
0709 13:13:00.536197 (+ 22052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:00.561888 (+ 25691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:02.900507 (+2338619us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b54000e9054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:02.903235 (+  2728us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43254656}
W20260709 13:13:02.926260 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 132715us
I20260709 13:13:02.927608 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:02.895888   349 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:02.906322 32730 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:02.950975 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:02.947808 28198 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:02.901513   320 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:02.917135   318 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:02.956068 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:02.947113 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:02.959296 28728 leader_election.cc:336] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.739s (SENT)
I20260709 13:13:01.617132 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:02.965137 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:02.968014 28152 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:02.964527 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:02.970436 28158 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:02.967424 28728 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:02.969583 28198 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:02.933852   356 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:02.910313 32676 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:13:02.975250 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:02.965752   318 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:02.933084   333 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:02.971980   395 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:02.971623   394 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:02.951977   321 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.004969 28124 leader_election.cc:400] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14.
I20260709 13:13:02.987390 32727 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:02.995141   318 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:03.049657 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 90406.4us
I20260709 13:13:03.010356 32498 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:03.011458 32743 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:03.056371 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:03.056700 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 85335.6us
I20260709 13:13:03.062119 28152 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:13:02.993700   320 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:13:03.075549 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:03.076248 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:03.078364 28152 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:03.097836 28161 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:03.055823   353 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.053109 32727 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:03.100051   420 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:03.101423 32498 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:03.073943   356 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.117246 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 896) took 2691 ms. Trace:
W20260709 13:13:03.119552 28761 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35896 (request call id 959) took 2503 ms (client timeout 2023 ms). Trace:
I20260709 13:13:03.132916 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2383) took 1318 ms. Trace:
W20260709 13:13:03.100363   416 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:13:03.125531 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:03.138258 28077 rpcz_store.cc:276] 0709 13:13:01.814387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.119779 (+305392us) service_pool.cc:224] Handling call
0709 13:13:02.134818 (+ 15039us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:02.302630 (+167812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.132614 (+829984us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400207e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:03.132853 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375832448}
I20260709 13:13:03.125826 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:03.122314 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2099) took 1272 ms. Trace:
I20260709 13:13:03.196978 28227 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:03.200304 28757 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:13:03.137267 28761 rpcz_store.cc:269] 0709 13:13:00.616125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.624644 (+  8519us) service_pool.cc:224] Handling call
0709 13:13:03.119359 (+2494715us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540043ab54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:03.119464 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3114062080}
W20260709 13:13:03.113283   423 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 13:13:03.112699   437 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:13:03.135179 28235 rpcz_store.cc:276] 0709 13:13:00.425843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.436971 (+ 11128us) service_pool.cc:224] Handling call
0709 13:13:00.473274 (+ 36303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:00.521473 (+ 48199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:00.521507 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:00.553486 (+ 31979us) log.cc:844] Serialized 26 byte log entry
0709 13:13:02.659630 (+2106144us) spinlock_profiling.cc:243] Waited 6.23 ms on lock 0x7b18002ef9c0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:13:02.897143 (+237513us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:02.897203 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:02.897307 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.116804 (+219497us) raft_consensus.cc:1685] finished
0709 13:13:03.116825 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.117068 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28662,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":4432,"spinlock_wait_cycles":14323072,"thread_start_us":235752,"threads_started":1,"wal-append.queue_time_us":2430343}
I20260709 13:13:03.127552 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2499) took 1324 ms. Trace:
I20260709 13:13:03.202477 32666 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:03.130815   376 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.499s	user 0.003s	sys 0.007s
I20260709 13:13:03.112849   433 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:13:03.128069   372 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.670s	user 0.000s	sys 0.008s
I20260709 13:13:03.122483   421 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:03.111881   420 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:03.125805   356 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:03.133592   427 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:13:03.237524 32706 consensus_meta.cc:287] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.004s
I20260709 13:13:03.115412   411 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:03.242241 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:03.197618 28231 rpcz_store.cc:276] 0709 13:13:01.850078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.170191 (+320113us) service_pool.cc:224] Handling call
0709 13:13:02.170587 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:02.562613 (+392026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:02.562666 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:02.568602 (+  5936us) log.cc:844] Serialized 52 byte log entry
0709 13:13:02.844608 (+276006us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:02.844666 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:02.844779 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.122074 (+277295us) raft_consensus.cc:1685] finished
0709 13:13:03.122094 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.122257 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":386065,"raft.queue_time_us":391976,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7894,"thread_start_us":244872,"threads_started":1,"wal-append.queue_time_us":418170}
I20260709 13:13:03.129400   426 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.113469 32498 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.107262 32447 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.264590 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:03.302425 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 370745us
I20260709 13:13:03.266239   321 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.246438 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:03.236081 32708 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:03.133618   424 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 13:13:03.108321   353 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.247120 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:03.223958 28232 rpcz_store.cc:276] 0709 13:13:01.803446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.015783 (+212337us) service_pool.cc:224] Handling call
0709 13:13:02.016605 (+   822us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:02.544317 (+527712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:02.544345 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:02.544933 (+   588us) log.cc:844] Serialized 52 byte log entry
0709 13:13:02.781291 (+236358us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:03.000477 (+219186us) log.cc:844] Serialized 14 byte log entry
0709 13:13:03.004983 (+  4506us) log.cc:844] Serialized 14 byte log entry
0709 13:13:03.014644 (+  9661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.014739 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.127279 (+112540us) raft_consensus.cc:1685] finished
0709 13:13:03.127301 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.127493 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":540327,"prepare.queue_time_us":270784,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":605689,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":25625,"thread_start_us":404647,"threads_started":3,"wal-append.queue_time_us":533527}
I20260709 13:13:03.133605   438 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:03.133790   436 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:03.239356 32666 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:03.265429 32724 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.000s
I20260709 13:13:03.132758   435 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.306s
I20260709 13:13:03.236145   449 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.201s
I20260709 13:13:03.110795 32727 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.268460 32510 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:03.131065   360 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:03.268807   312 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:03.269263 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 971) took 2169 ms. Trace:
I20260709 13:13:03.267531   421 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:03.246392   420 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:03.308830 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 973) took 1170 ms. Trace:
I20260709 13:13:03.133543   434 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:03.293897   452 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:03.240218 32597 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.532s	user 0.005s	sys 0.005s
I20260709 13:13:03.334623 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:03.370170 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:03.335197 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:03.371551 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.332868 32724 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:03.336057 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 363147us
I20260709 13:13:03.348140   430 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.342509 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:03.284778 32706 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.326989   455 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:03.350570 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:03.312289   414 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:03.351254 28764 rpcz_store.cc:276] 0709 13:13:02.138878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.140929 (+  2051us) service_pool.cc:224] Handling call
0709 13:13:02.153722 (+ 12793us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:02.295884 (+142162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:02.295921 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:02.296574 (+   653us) log.cc:844] Serialized 52 byte log entry
0709 13:13:02.768431 (+471857us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:02.768491 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:02.768595 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.308534 (+539939us) raft_consensus.cc:1685] finished
0709 13:13:03.308562 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.308770 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133387,"raft.queue_time_us":140768,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":6376,"thread_start_us":356851,"threads_started":1,"wal-append.queue_time_us":564053}
I20260709 13:13:03.361457 28082 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
W20260709 13:13:03.363489 28234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2093) took 2050 ms (client timeout 2037 ms). Trace:
I20260709 13:13:03.312898   451 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:03.366320 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:03.245617   433 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.335515   394 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.374542 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2372) took 2117 ms. Trace:
I20260709 13:13:03.374956 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:03.311764 32708 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.336046   333 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.367123   421 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.348500 28756 rpcz_store.cc:276] 0709 13:13:01.099386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.099534 (+   148us) service_pool.cc:224] Handling call
0709 13:13:01.099915 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:01.134420 (+ 34505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.134452 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:01.165415 (+ 30963us) log.cc:844] Serialized 26 byte log entry
0709 13:13:01.394471 (+229056us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:01.394516 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.394589 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.127282 (+1732693us) spinlock_profiling.cc:243] Waited 9.43 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:03.268932 (+141650us) raft_consensus.cc:1685] finished
0709 13:13:03.268957 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.269176 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17153,"raft.queue_time_us":17393,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":17571,"spinlock_wait_cycles":21686016,"thread_start_us":191464,"threads_started":1,"wal-append.queue_time_us":266285}
I20260709 13:13:03.305123   426 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.350545   312 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:03.382638 28232 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:03.384390 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:03.408543 28077 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:03.382066 32676 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:03.385205 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:03.385284 28234 rpcz_store.cc:269] 0709 13:13:01.296721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.325659 (+ 28938us) service_pool.cc:224] Handling call
0709 13:13:03.347394 (+2021735us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400286c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:03.347509 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358216448}
I20260709 13:13:03.330466   438 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.384845   411 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:03.375955   436 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:03.403751 28758 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:03.363924   387 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:13:03.397485 28086 rpcz_store.cc:276] 0709 13:13:01.250281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.250440 (+   159us) service_pool.cc:224] Handling call
0709 13:13:01.294663 (+ 44223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:01.309066 (+ 14403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.309100 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:01.329630 (+ 20530us) log.cc:844] Serialized 26 byte log entry
0709 13:13:01.491557 (+161927us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:01.491598 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:01.491661 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.130117 (+1638456us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:03.362870 (+232753us) raft_consensus.cc:1685] finished
0709 13:13:03.362904 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.367680 (+  4776us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37663,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":8899,"spinlock_wait_cycles":98072832,"thread_start_us":127341,"threads_started":1,"wal-append.queue_time_us":210823}
I20260709 13:13:03.413831   421 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.413590   438 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:03.379128   435 consensus_queue.cc:1059] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:13:03.362576   360 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:03.410034   409 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.819s	user 0.012s	sys 0.002s
I20260709 13:13:03.439432 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:03.372979 32597 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:03.450054 28153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2390) took 2590 ms (client timeout 2759 ms). Trace:
W20260709 13:13:03.452418 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 77584.2us
I20260709 13:13:03.419294   411 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:03.463982   411 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:03.460394 28198 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:03.451092 28153 rpcz_store.cc:269] 0709 13:13:00.850146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.850377 (+   231us) service_pool.cc:224] Handling call
0709 13:13:03.439713 (+2589336us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b5400342bd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:03.439865 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686064000}
I20260709 13:13:03.459895 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:03.441140   418 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.732s	user 0.005s	sys 0.009s
I20260709 13:13:03.373128   452 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.461845 32726 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:03.461804 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:03.462819 32706 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:03.463425 32708 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.460668 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:03.479272 28086 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:13:03.476101 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 75853.5us
I20260709 13:13:03.477546 28152 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:03.479300 28082 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:13:03.481006 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:03.481679 28230 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:03.481729 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:03.482471 28234 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:03.459215   436 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:03.386719   455 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.396831   451 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.484787 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:03.511606 28153 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:03.395565   430 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.482697   435 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:03.486760   436 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:03.414742   394 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.518852   333 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:03.512564 32739 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.544170 28086 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:03.544486 28077 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:03.472048   409 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.546100 28077 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:03.537191   459 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:13:03.476972   418 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.583786 28227 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:03.537432   333 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:03.599289 28077 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:03.598239 28158 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:13:03.481730   312 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:03.552421   467 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:13:03.538014   436 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:03.603956   460 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:03.604645   465 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
W20260709 13:13:03.605121   468 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:13:03.610911   319 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:03.619495   319 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:03.620262   360 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:03.625412   458 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:13:03.627318   452 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:03.628654 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:03.629328 28763 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:13:03.590337   395 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:13:03.610800 32708 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:13:03.597728 32447 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:03.637143 32447 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:03.607896 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 222754us
W20260709 13:13:03.639307   463 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:13:03.547351   361 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:13:03.641445   361 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:03.641831   426 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.642019   361 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:03.643006   361 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.644284   361 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:03.533198   335 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:13:03.646162   335 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:03.646545   335 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:03.647739   335 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.656790 32597 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:03.657495   335 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:03.592566 32666 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:03.641462 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2508) took 1041 ms. Trace:
I20260709 13:13:03.648455 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:03.646587   319 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.602232   333 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:03.653540   466 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:13:03.588979   318 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 13:13:03.591878   435 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:03.597613   438 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.618907   394 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:03.665913 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:03.598807   353 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.664845 28229 rpcz_store.cc:276] 0709 13:13:02.599944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.957946 (+358002us) service_pool.cc:224] Handling call
0709 13:13:02.958295 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:02.970932 (+ 12637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:02.970964 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:02.981108 (+ 10144us) log.cc:844] Serialized 52 byte log entry
0709 13:13:03.333726 (+352618us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:03.348916 (+ 15190us) log.cc:844] Serialized 14 byte log entry
0709 13:13:03.349213 (+   297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.349310 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.641125 (+291815us) raft_consensus.cc:1685] finished
0709 13:13:03.641156 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.641366 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12068,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":7263,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":17943,"spinlock_wait_cycles":1909632,"thread_start_us":290688,"threads_started":1,"wal-append.queue_time_us":445471}
I20260709 13:13:03.553520 32739 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.682067 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:03.682977 28077 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:03.690768 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:03.698464 28154 raft_consensus.cc:2368] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:03.664827 32724 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:03.679987   388 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:03.706331 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:03.661098   460 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.710287   460 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.717175 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:03.719023 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:03.720116 28152 raft_consensus.cc:2368] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:03.622035   420 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.723076   420 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.729205 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:03.730507 28077 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:03.730599   418 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.731858   430 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:03.732530   433 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:03.732785 32739 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:03.702730   438 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.733570   411 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.699335 32597 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:03.743091   411 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:03.729364   435 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.745613   455 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:03.682288   318 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.738782   427 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
W20260709 13:13:03.752243 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 202473us
W20260709 13:13:03.755748 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.699844 32726 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:03.759415   426 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:03.705591   394 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:03.760703   456 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:03.760785   394 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.761240   456 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:03.762475   438 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.762297   456 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.763617   456 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:03.770475 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:03.773712 32597 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:03.774816 32597 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:03.775918 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:03.778155 28757 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:03.754488 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:03.783782 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:03.734863   387 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.795338 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:03.795529 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:03.796543 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:03.797015 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:03.737161   319 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.772972 28199 leader_election.cc:400] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:03.801689 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:03.796538   361 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:03.802596   395 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:03.799517 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 327867us
I20260709 13:13:03.763427   394 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:03.915838 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.979485 28084 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:13:03.980203 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.792272   318 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:03.797897   433 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.990626 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:03.922737 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 149972us
I20260709 13:13:03.992121 28199 leader_election.cc:400] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:03.992523 28199 leader_election.cc:403] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:13:03.923919 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:03.800402 32666 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:03.770000   420 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.139s
I20260709 13:13:03.803823 32510 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:03.951606 28757 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:03.967041 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:03.968863 28081 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2380) took 2660 ms (client timeout 2840 ms). Trace:
I20260709 13:13:03.919639   323 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.975633 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:03.754483   388 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:03.976706 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:04.135905 28077 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:04.136998 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:04.137735 28077 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:13:03.976625 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:03.803609   387 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:03.985809 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:03.989439 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:03.960614   411 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.153680 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.154135   411 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.155786   408 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:04.156453 28155 raft_consensus.cc:2410] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:13:04.156997 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:04.158573 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.158757 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.159201 28234 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:04.159757 28231 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
W20260709 13:13:03.990245 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 83544.1us
I20260709 13:13:04.160422 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.163817 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:04.164884 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.165266 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:04.165853 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.166616 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:04.169250   453 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:04.169766   453 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:04.170310   453 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:04.175269 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:04.169772   488 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:13:04.186808 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:04.187105 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:04.187647   436 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.002s
I20260709 13:13:04.187893 28159 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:04.188150   436 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.192046 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2375) took 2940 ms. Trace:
I20260709 13:13:04.192700 28083 rpcz_store.cc:276] 0709 13:13:01.251709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.251861 (+   152us) service_pool.cc:224] Handling call
0709 13:13:01.286349 (+ 34488us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:01.468024 (+181675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.468058 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:01.468655 (+   597us) log.cc:844] Serialized 52 byte log entry
0709 13:13:01.523553 (+ 54898us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:01.526215 (+  2662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.191519 (+2665304us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54003a93d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:04.191582 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.191655 (+    73us) raft_consensus.cc:1685] finished
0709 13:13:04.191669 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.191978 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65508,"raft.queue_time_us":37157,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6161,"spinlock_wait_cycles":335444352,"thread_start_us":32128,"threads_started":1,"wal-append.queue_time_us":71814,"wal-append.run_cpu_time_us":1863,"wal-append.run_wall_time_us":1016175}
W20260709 13:13:04.196704   472 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.590s	user 0.002s	sys 0.002s
I20260709 13:13:04.078747 32554 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:04.024830 32676 consensus_meta.cc:287] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.602s	user 0.012s	sys 0.000s
I20260709 13:13:04.035868   484 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:04.004076 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.078694   473 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.997054   353 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.095201 28152 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:04.031817   312 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:04.029851 32447 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:04.074062 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:04.214367 28759 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:03.995982 32666 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.004096 32510 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:04.221920 32730 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:03.979054   395 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:04.215770   373 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:04.064597   480 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:03.997148   319 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
W20260709 13:13:04.092110   479 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:13:03.995059   433 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.001535 32726 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:04.077534   412 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.653s	user 0.008s	sys 0.008s
I20260709 13:13:03.982703   361 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:04.128512 28081 rpcz_store.cc:269] 0709 13:13:01.300179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.438631 (+138452us) service_pool.cc:224] Handling call
0709 13:13:03.953460 (+2514829us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b54001ae5d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:03.960225 (+  6765us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102397312}
I20260709 13:13:04.142539   388 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.274228   353 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.274611   436 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:04.140007   323 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.299723   353 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:04.299885 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2070) took 1480 ms. Trace:
I20260709 13:13:04.139078   452 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.143918 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:04.145478 28230 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:04.150820 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2416) took 1204 ms. Trace:
W20260709 13:13:04.296468   470 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.691s	user 0.004s	sys 0.000s
I20260709 13:13:04.023358   333 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.197708 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:04.198279 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:04.198463 28229 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:04.199656 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:04.199848   320 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:04.200397 32699 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:04.201325 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.201444   456 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.201932 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:04.025064 32498 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:04.213694 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:04.231881 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:04.204368 28763 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:04.038388 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:04.239745 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:04.252138 28084 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:04.214552 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:04.206349   363 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:13:04.349823 28050 connection.cc:256] RPC call timeout handler was delayed by -1.0106044030001158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
I20260709 13:13:04.350829 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:04.268478 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:04.259398 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.233517   411 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:04.203670 28764 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:13:04.357990 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 158853us
I20260709 13:13:04.271410 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:04.272495 28235 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:04.270288   395 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:04.269932   484 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.270545   373 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:04.243263 28077 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:04.206349   349 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:04.212991   386 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.270795 32730 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:04.211588   409 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:04.202816   335 consensus_meta.cc:287] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.003s
I20260709 13:13:04.291081 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2089) took 2999 ms. Trace:
W20260709 13:13:04.205837   485 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:13:04.266461   473 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.227208 27984 catalog_manager.cc:5697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:04.271446 32554 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.286194 32676 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:04.301265 28160 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:04.300308 28080 rpcz_store.cc:276] 0709 13:13:02.819825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.307214 (+487389us) service_pool.cc:224] Handling call
0709 13:13:03.330783 (+ 23569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:03.347223 (+ 16440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:03.347261 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:03.362060 (+ 14799us) log.cc:844] Serialized 26 byte log entry
0709 13:13:03.475757 (+113697us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:03.475814 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.475916 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.195901 (+719985us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:04.292876 (+ 96975us) raft_consensus.cc:1685] finished
0709 13:13:04.292908 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.299818 (+  6910us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4650,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":12774,"spinlock_wait_cycles":8856448,"thread_start_us":88392,"threads_started":1,"wal-append.queue_time_us":227715}
I20260709 13:13:04.291535   480 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.386988 28080 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:04.307361   433 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:04.388485 32554 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:04.308007   464 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.301011   361 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:04.388312   480 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:04.287173 32739 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:04.313095 28758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 775) took 1774 ms (client timeout 1779 ms). Trace:
W20260709 13:13:04.402768 28758 rpcz_store.cc:269] 0709 13:13:02.527084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.532983 (+  5899us) service_pool.cc:224] Handling call
0709 13:13:04.301428 (+1768445us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b5400562cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:04.301575 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1951762688}
I20260709 13:13:04.267956 32510 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.404302   498 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:04.404995 32739 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.268478 32666 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:04.406579   498 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.407030   498 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:04.402859   464 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.318207 28158 rpcz_store.cc:276] 0709 13:13:02.945998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.986137 (+ 40139us) service_pool.cc:224] Handling call
0709 13:13:03.003398 (+ 17261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:03.600825 (+597427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.150487 (+549662us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b5400375fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:04.150745 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1367363328}
I20260709 13:13:04.409674 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:04.195649   492 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:04.326198 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:04.410250   464 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:04.410667   492 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:04.411679   492 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.322568 28756 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:13:04.412066 27984 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:04.341193 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:04.413623 28081 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:04.343703 28083 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:04.348990 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 117207us
I20260709 13:13:04.352408 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:04.353112 32498 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:04.414866 32498 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:04.415269 32498 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:04.415938 28086 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:04.338732 28761 raft_consensus.cc:2368] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:13:04.353734 28079 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:04.354774 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:04.354974   333 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:04.363245 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:04.364069 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 187423us
I20260709 13:13:04.370205 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:04.374851 28077 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:13:04.380137   323 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:04.351114 32607 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:04.375864 28082 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:04.377357 28232 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:04.377390 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:04.328135   452 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.433357 28234 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:04.390173 28231 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:04.390355 28226 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:04.381109 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.310123   453 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.371271   349 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:04.392835   473 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:04.393301   373 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.464577   373 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:04.394536   484 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:04.350332   456 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.466854 32720 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:04.468078   456 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:04.382241   386 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.469058 28760 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:04.470189   386 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:04.410818 28153 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:04.471195 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:04.359637 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 317300us
I20260709 13:13:04.417675 28152 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:04.417796 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:04.418378 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:04.427222 28083 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:04.410288 28158 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
W20260709 13:13:04.431433 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 68330.1us
I20260709 13:13:04.434926 28229 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:04.430229 32200 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:13:04.438288 27984 catalog_manager.cc:5697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 5 to 6, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 6 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:04.417408   335 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.432705   480 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:13:04.442479 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:04.443236 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:04.443980 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:04.444592 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:04.446493   453 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:04.447254 28159 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:04.464581   349 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:13:04.450803 27980 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 191) took 21548 ms (client timeout 15000 ms). Trace:
I20260709 13:13:04.452069   323 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.454905   498 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.458072   395 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.458425 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:04.459339   451 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:04.462518   452 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:04.401688   499 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.432523 32607 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:04.471390 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:04.380797 28228 rpcz_store.cc:276] 0709 13:13:01.291513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.539533 (+248020us) service_pool.cc:224] Handling call
0709 13:13:01.547498 (+  7965us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:01.596761 (+ 49263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.596806 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:01.682587 (+ 85781us) log.cc:844] Serialized 52 byte log entry
0709 13:13:01.826190 (+143603us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:01.840412 (+ 14222us) log.cc:844] Serialized 14 byte log entry
0709 13:13:02.514218 (+673806us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b1800466b40. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:13:02.514418 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.290724 (+1776306us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b54000dd4d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:04.290780 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:04.290832 (+    52us) raft_consensus.cc:1685] finished
0709 13:13:04.290845 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.291013 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28301,"prepare.queue_time_us":20202,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":59,"raft.queue_time_us":712578,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":15673,"spinlock_wait_cycles":114667264,"thread_start_us":125774,"threads_started":1,"wal-append.queue_time_us":199003,"wal-append.run_cpu_time_us":2383,"wal-append.run_wall_time_us":1013748}
W20260709 13:13:04.623291 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.486286 28234 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:04.728991 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:04.492594 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:04.493914 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2074) took 1146 ms. Trace:
I20260709 13:13:04.736079 27984 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 8 to 12, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 12 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:04.502588 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 860) took 1396 ms. Trace:
I20260709 13:13:04.456426 32726 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:04.733790 32706 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.744513 32706 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.744841   499 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.493237   484 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:04.746196   484 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:04.746609   484 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:04.746923 32706 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:04.689497 32676 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:04.748150 32676 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.749064   349 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:04.723053 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.494474 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.737838 28227 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:04.737244 28152 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
W20260709 13:13:04.474056 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 55745.3us
I20260709 13:13:04.754861 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:04.741827 28161 rpcz_store.cc:276] 0709 13:13:03.105819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.106297 (+   478us) service_pool.cc:224] Handling call
0709 13:13:03.106783 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:03.413008 (+306225us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:13:03.532368 (+119360us) log.cc:844] Serialized 14 byte log entry
0709 13:13:03.680392 (+148024us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:03.680466 (+    74us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:03.680533 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.501681 (+821148us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b18003b3ce0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419
0709 13:13:04.502278 (+   597us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54000a32d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:04.502334 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:04.502514 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1163,"prepare.queue_time_us":190436,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":121,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":14449,"spinlock_wait_cycles":706690048,"thread_start_us":265308,"threads_started":2,"wal-append.queue_time_us":452532}
I20260709 13:13:04.756968 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:04.730206 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 312419us
I20260709 13:13:04.737496 28077 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:04.474028 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:04.758572 28230 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:04.740833 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:04.736810 28086 rpcz_store.cc:276] 0709 13:13:03.347870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.545536 (+197666us) service_pool.cc:224] Handling call
0709 13:13:03.545976 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.416314 (+870338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.493587 (+ 77273us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b54001aead4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:04.493824 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1999914624}
I20260709 13:13:04.734428 28084 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 7.
I20260709 13:13:04.757736 28161 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:04.757023 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:04.760838 28158 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:04.763763 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:04.764415 28154 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:04.765600 28050 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:04.767539 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.770850 28161 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:04.767434 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.690271   364 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:04.798900 28081 raft_consensus.cc:2368] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:13:04.799181 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:04.799757 28080 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:04.798712 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:04.777383 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.703732   464 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:04.502516   456 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:04.806094 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:04.767347   451 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:04.808408   351 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:04.808851   351 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:04.809053 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:04.809788 28757 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:04.809870   351 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.811172 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:04.793020 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.812227 28761 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:04.778997 28158 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:04.774420 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:04.496940   313 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:04.802651 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:04.808099   451 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.798600 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:04.796911 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.505383   415 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:04.436460   312 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:04.820622 28230 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:04.821359 28235 raft_consensus.cc:2410] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:04.822049 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:04.822926 28230 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:04.862793 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:04.772826   353 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:04.861856 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47095.3us
I20260709 13:13:04.866714 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.866809 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:04.705152 32666 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:04.868650 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:04.870304 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 507094us
I20260709 13:13:04.751137   335 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.743752 32498 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.578670 32730 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.864537 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:04.928460 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:04.933779 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:05.045444 28756 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:13:04.939947 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 498060us
I20260709 13:13:04.929862 27984 catalog_manager.cc:5697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:04.813439   360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.006s	sys 0.017s
I20260709 13:13:04.817031   386 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:05.054678 28229 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:04.945343 28085 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:04.947100 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:04.909665 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:04.763294   452 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:05.018262   361 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.587s	user 0.005s	sys 0.005s
I20260709 13:13:05.057828 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:05.016156   498 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:04.935690 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:04.811039 32510 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:05.064260 28154 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:05.083761 32510 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.084249 32510 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:04.872846 32200 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:05.086014 32200 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.086525 32200 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.087529 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:04.868175   320 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:05.089231 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:05.090600 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:05.091408 28763 raft_consensus.cc:2368] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:13:04.756126   323 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.082000   506 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:13:05.092249 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:05.092783 28763 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:05.093344   323 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:05.095913   351 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:05.096889 28153 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:05.097137 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:05.097541 28758 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:04.952266 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:05.003985   491 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.864s	user 0.005s	sys 0.011s
I20260709 13:13:04.871690 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:05.107402 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:05.109125 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:04.919926 28084 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:05.059924 28228 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:05.059370 28158 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:05.053249   492 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:05.141793 28763 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:05.137010 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:04.932379 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.143234 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.930287 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:05.099447 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:05.054733   360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.006s	sys 0.027s
W20260709 13:13:04.950495 32706 raft_consensus.cc:670] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:13:04.948063   426 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:05.101931 28078 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:05.047863 28230 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:05.105657 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:05.194657 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:05.195211 28153 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
W20260709 13:13:05.106475 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 61413.4us
I20260709 13:13:05.198694 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:05.107482 28233 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:05.051213 28086 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:05.124712 28234 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:04.964273   409 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.003s	sys 0.071s
I20260709 13:13:04.870703   353 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:13:05.045487 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 111810us
I20260709 13:13:05.211598 28050 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:05.001468 32676 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:05.147178 28762 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:05.070741 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:05.143247 28158 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:13:04.804150   364 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.150141 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:05.152021 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:05.152091 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:05.216037 28232 raft_consensus.cc:2376] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11. Re-sending same reply.
I20260709 13:13:05.220621 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:04.994427   451 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.158159 32200 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:05.158525 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:05.083341   496 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:13:05.161274   505 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 13:13:05.164836 28226 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:05.089231   320 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.168805   333 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:13:05.141922   492 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.142376   510 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:13:05.097371   361 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.152309   455 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.171614 32510 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:05.202992 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.203850   319 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:05.189311 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:05.207716 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:04.818893   464 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:04.871115   415 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.210865 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:05.205641   409 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.003s	sys 0.087s
W20260709 13:13:05.305605 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.219688 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:05.180485   499 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.235173 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:05.251410 28228 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 13:13:05.311599 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 236709us
I20260709 13:13:05.137673 32708 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:13:05.256844 28226 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:05.313709 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:05.314498 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:05.210464   351 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:05.256026 28229 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:05.315626 28084 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:05.260068 28078 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:05.125434   452 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.217228   373 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.219488   353 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:04.946638 32607 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.264040 28233 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:13:05.283082 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.310839   506 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:05.210075   386 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:13:05.191861   426 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:05.305739   319 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:05.318620 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.319597 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:05.211153   484 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.261211 32706 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:05.257222   496 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:05.257571   312 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:13:05.358760 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.325664 28235 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:05.360529   364 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.234627   323 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:13:05.262341   492 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:05.328308 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 282514us
W20260709 13:13:05.366436 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.262034 32666 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:05.331501 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:05.340374   506 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.307093   313 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.336995 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:05.341089 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:05.306085 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 116904us
W20260709 13:13:05.342945 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 478485us
I20260709 13:13:05.310873   455 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.374047 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:05.375574 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:05.375834 32666 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:05.231276   498 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:05.376451 32666 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:05.377573 28226 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:05.377813   455 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.350457 28161 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:13:05.353969 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:05.378947 28233 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:05.330881   453 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:05.379861   364 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.380378   453 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:05.380501   364 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.381887   453 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.383041   364 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:05.378971 32666 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:05.385221   364 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:05.376717 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:05.346192 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:05.364940 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:05.344005   360 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:05.397054   360 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:05.397755   360 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:05.332350 28230 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:05.372380 28763 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:05.351161 32607 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:05.430529 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:05.429849 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45054.8us
I20260709 13:13:05.435636 28049 leader_election.cc:400] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:13:05.436236 28049 leader_election.cc:403] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:05.350185 32706 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:05.337283   373 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.440981 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:05.442665 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 207542us
I20260709 13:13:05.447899 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:05.454164 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:05.455044 28230 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:05.374852   506 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:05.456202 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:05.465571 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:05.472040 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 125911us
I20260709 13:13:05.445833 32706 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:05.440294   510 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:05.454571 32498 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:05.453397   395 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.453789   387 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:05.457655   386 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:05.461231 32730 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.484457   320 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:05.475140   349 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.725s	user 0.013s	sys 0.000s
I20260709 13:13:05.487715   420 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:05.439288   498 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:13:05.493968   386 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.509579   386 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:05.460722 32708 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:05.510572 32708 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:05.511045 32708 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:05.462842   409 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.513020   409 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.514849   350 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:05.515316   350 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:05.515424   409 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:05.516129   409 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.489351 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:05.391243   313 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.458619   426 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.535218 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 996) took 1370 ms. Trace:
I20260709 13:13:05.474393 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:05.539639 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 65448.7us
I20260709 13:13:05.541553 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:05.538415 28078 rpcz_store.cc:276] 0709 13:13:04.164636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.352089 (+187453us) service_pool.cc:224] Handling call
0709 13:13:05.535031 (+1182942us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54003aa2d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:05.535153 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1580388736}
I20260709 13:13:05.496641 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:05.461637   456 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:05.494571 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:05.552868 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 58407.9us
I20260709 13:13:05.557507 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:05.504997 28228 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:13:05.500824   526 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:05.507097 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 993) took 1343 ms. Trace:
I20260709 13:13:05.508881   395 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.542804 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:13:05.482014 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 262386us
I20260709 13:13:05.616664 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:05.544395 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47807.5us
I20260709 13:13:05.572921 28080 rpcz_store.cc:276] 0709 13:13:04.163562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.387933 (+224371us) service_pool.cc:224] Handling call
0709 13:13:04.388236 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.801229 (+412993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.506798 (+705569us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54001af4d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:05.507026 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":945950080}
I20260709 13:13:05.621338 28084 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:05.567143   511 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.563s	user 0.001s	sys 0.003s
I20260709 13:13:05.491487   430 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:05.569366 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2103) took 1095 ms. Trace:
I20260709 13:13:05.631769 28227 rpcz_store.cc:276] 0709 13:13:04.474196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.474645 (+   449us) service_pool.cc:224] Handling call
0709 13:13:04.737672 (+263027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.752006 (+ 14334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.569061 (+817055us) spinlock_profiling.cc:243] Waited 5.88 ms on lock 0x7b54000fb9d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:05.569267 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13513984}
I20260709 13:13:05.570252 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2108) took 1065 ms. Trace:
W20260709 13:13:04.737855 27980 rpcz_store.cc:269] 0709 13:12:42.902007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.902313 (+   306us) service_pool.cc:224] Handling call
0709 13:13:04.450547 (+21548234us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:58.267730 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:58.272572 (+  4842us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:58.272612 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:12:58.272633 (+    21us) tablet.cc:662] Decoding operations
0709 13:12:58.374586 (+101953us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:58.374656 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:58.374682 (+    26us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:12:58.654632 (+279950us) tablet.cc:701] Row locks acquired
0709 13:12:58.655587 (+   955us) write_op.cc:260] PREPARE: finished
0709 13:12:58.660436 (+  4849us) write_op.cc:270] Start()
0709 13:12:58.690630 (+ 30194us) write_op.cc:276] Timestamp: P: 1783602778659072 usec, L: 0
0709 13:12:58.690674 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:12:58.691859 (+  1185us) log.cc:844] Serialized 7877 byte log entry
0709 13:12:58.786088 (+ 94229us) op_driver.cc:464] REPLICATION: finished
0709 13:12:58.788493 (+  2405us) write_op.cc:301] APPLY: starting
0709 13:12:58.788604 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:59.075927 (+287323us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:59.076134 (+   207us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:59.519052 (+442918us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:59.519562 (+   510us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:12:59.519638 (+    76us) write_op.cc:312] APPLY: finished
0709 13:12:59.520386 (+   748us) log.cc:844] Serialized 180 byte log entry
0709 13:12:59.520642 (+   256us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:59.521101 (+   459us) write_op.cc:454] Released schema lock
0709 13:12:59.521206 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:59.589991 (+ 68785us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:59.593044 (+  3053us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3820568,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":2094,"apply.run_cpu_time_us":43410,"apply.run_wall_time_us":773124,"cfile_cache_hit":78,"cfile_cache_hit_bytes":81783,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6224,"cfile_init":2,"lbm_read_time_us":9234,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":11,"num_ops":20,"prepare.queue_time_us":260637,"prepare.run_cpu_time_us":16981,"prepare.run_wall_time_us":514375,"raft.queue_time_us":16471,"raft.run_cpu_time_us":1049,"raft.run_wall_time_us":3332,"replication_time_us":95217,"thread_start_us":265574,"threads_started":4,"wal-append.queue_time_us":73901,"wal-append.run_cpu_time_us":5075,"wal-append.run_wall_time_us":1756812}]]}
I20260709 13:13:05.617920 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2117) took 1110 ms. Trace:
I20260709 13:13:05.564798 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2059) took 4082 ms. Trace:
I20260709 13:13:05.635538 28232 rpcz_store.cc:276] 0709 13:13:04.507763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.218281 (+710518us) service_pool.cc:224] Handling call
0709 13:13:05.231013 (+ 12732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.231226 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:05.444079 (+212853us) log.cc:844] Serialized 14 byte log entry
0709 13:13:05.617472 (+173393us) log.cc:844] Serialized 14 byte log entry
0709 13:13:05.617598 (+   126us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.617634 (+    36us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:05.617674 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.617740 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.617854 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":360782,"threads_started":2}
I20260709 13:13:05.633468 28234 rpcz_store.cc:276] 0709 13:13:04.505102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.738212 (+233110us) service_pool.cc:224] Handling call
0709 13:13:05.570025 (+831813us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b5400305f54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:05.570189 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819186560}
I20260709 13:13:05.637221 28125 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:05.636986 28079 rpcz_store.cc:276] 0709 13:13:01.482697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.703670 (+220973us) service_pool.cc:224] Handling call
0709 13:13:03.953681 (+2250011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.417360 (+463679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.323215 (+905855us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54001ae5d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:05.564492 (+241277us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b54001ae5d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:05.564696 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6093295872}
I20260709 13:13:05.641981 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:05.533780   320 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:05.504179 32666 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:05.653033 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:05.518726 32510 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:05.649156 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:05.649308 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 207758us
I20260709 13:13:05.572242   498 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:05.522588   373 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.669109 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2519) took 1341 ms. Trace:
I20260709 13:13:05.667399 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2521) took 1308 ms. Trace:
I20260709 13:13:05.670998 28231 rpcz_store.cc:276] 0709 13:13:04.359134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.359291 (+   157us) service_pool.cc:224] Handling call
0709 13:13:04.359586 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.434132 (+ 74546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.667166 (+1233034us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b54003bfbd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:05.667331 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69872512}
I20260709 13:13:05.521070   415 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.667398 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:05.529130   452 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.500639   484 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.490648   527 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:13:05.554841   507 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:05.644310 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:05.520241   464 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.635025   430 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:05.661312 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:05.661466 28232 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:05.572403   526 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.538039   513 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:05.666097 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:05.666806 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:05.612234   524 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:05.667912 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:05.670281 28760 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 1049) took 1456 ms (client timeout 1846 ms). Trace:
I20260709 13:13:05.672194 28764 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:05.653224 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:05.707566 28077 raft_consensus.cc:2410] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:13:05.701812 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:05.666256 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 275111us
I20260709 13:13:05.669622 28153 rpcz_store.cc:276] 0709 13:13:04.327997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.471156 (+143159us) service_pool.cc:224] Handling call
0709 13:13:05.668895 (+1197739us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b540033ed54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:05.669039 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":834516864}
I20260709 13:13:05.679986 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:43180 (request call id 14) took 1572 ms. Trace:
I20260709 13:13:05.681496 28079 raft_consensus.cc:2368] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:05.684655 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:05.711753 28086 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:05.516414   350 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.643757   426 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:05.706115 28760 rpcz_store.cc:269] 0709 13:13:04.214129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.214260 (+   131us) service_pool.cc:224] Handling call
0709 13:13:05.670058 (+1455798us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54005f0fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:05.670208 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286913280}
I20260709 13:13:05.689682 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:05.734416 28757 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:13:05.690869 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49881.8us
I20260709 13:13:05.735437 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:13:05.693167 28159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 866) took 1505 ms (client timeout 1883 ms). Trace:
I20260709 13:13:05.523509   360 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.661795   420 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:05.700368 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:05.701061 28756 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:05.686690   451 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:05.492357   496 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.703729 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:05.714078 28083 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:05.627780   349 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 13:13:05.687597 28085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2082) took 1896 ms (client timeout 1521 ms). Trace:
W20260709 13:13:05.736861 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.817723 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:05.516685   409 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:05.698220   498 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:05.743762 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:05.669483   320 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:05.638598   395 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.491364   510 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:05.784416 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49070us
I20260709 13:13:05.784901   420 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.786298   541 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.789037   464 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:13:05.648623   456 consensus_queue.cc:1175] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0faaace2b36c4bcebf2fa6835235e92b" responder_term: 7 status { last_received { term: 6 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:05.793421 28756 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:05.710563 28157 rpcz_store.cc:276] 0709 13:13:04.089705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.089925 (+   220us) service_pool.cc:224] Handling call
0709 13:13:05.661283 (+1571358us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400237654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67d735 00007f59d363dd29 00007f59ccbbc1d4 00007f59ccbbc162 00007f59ccbbc091 00007f59ccbbc00c 00007f59ccbbb2c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:05.661474 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326692224}
I20260709 13:13:05.686400   523 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:05.818419 28083 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 9.
W20260709 13:13:05.818419 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 275654us
I20260709 13:13:05.792145   492 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:05.846894 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:05.702544   452 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:05.848047 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:05.882441 28079 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
W20260709 13:13:05.736382 28159 rpcz_store.cc:269] 0709 13:13:04.187322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.188574 (+  1252us) service_pool.cc:224] Handling call
0709 13:13:05.692922 (+1504348us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400254c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:05.693087 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285696256}
I20260709 13:13:05.847720 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:05.734556 32607 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.135s
I20260709 13:13:05.670158 32666 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.705165   484 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:05.701099   526 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:05.705050   524 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:05.696364   373 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.150s
I20260709 13:13:05.703342   415 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.151s
I20260709 13:13:05.755561   532 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:05.757347   527 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:05.816223   350 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:05.872557 28761 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
W20260709 13:13:05.873592 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 176279us
W20260709 13:13:05.878492 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 171590us
W20260709 13:13:05.817128 28085 rpcz_store.cc:269] 0709 13:13:03.775008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.954248 (+179240us) service_pool.cc:224] Handling call
0709 13:13:04.143734 (+189486us) spinlock_profiling.cc:243] Waited 958 us on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:13:05.671119 (+1527385us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b54002ace54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:05.671292 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444687104}
I20260709 13:13:05.878049 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:05.884428 28077 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:05.881834   452 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:05.890254 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:05.890470   498 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:05.891345   521 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:13:05.872159   349 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:05.848675   529 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:05.890321 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:05.853981   451 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:05.872679   420 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:06.020560   506 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.002s
I20260709 13:13:06.060273   480 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:06.061034   480 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.061426   480 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:06.061623   452 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:06.056699   456 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.063798   535 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:06.063791   456 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:06.001525 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:05.992971 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.954752   386 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.069687   480 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.005455   395 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.069865   451 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.070534 28086 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:05.999563   361 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:05.990221   360 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.068410   456 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:05.918236   353 consensus_meta.cc:287] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.000s
I20260709 13:13:06.004664   549 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:05.972762 32666 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:05.970103 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.101923   353 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.102988 28078 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:06.103971 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.071679   506 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.923514   320 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.072368   386 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:05.951730   551 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.108953 28078 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:06.074244 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:05.965677   319 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.110162 28085 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
W20260709 13:13:06.107504 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 240270us
I20260709 13:13:06.111191 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.112054 28085 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:05.975561 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.965615 32739 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:05.961547   499 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:05.890282   492 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.075500 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:05.946924 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.926519   312 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:06.133123 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.134802 28761 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:06.135294 28762 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:06.135528 28761 raft_consensus.cc:606] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:06.135934 28761 raft_consensus.cc:614] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:05.915148   544 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:05.905536   538 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:05.906144 28082 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:06.060160 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 182222us
I20260709 13:13:06.077821 28083 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:06.140180 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.141880   480 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:06.142500 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.144039 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:05.901165 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 190882us
I20260709 13:13:06.061440 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.145040 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.145401 28077 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:06.145320 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:06.146333 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.061344 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:06.051219 32708 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.001s
I20260709 13:13:06.147758 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.148185 32708 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.148977 28079 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:13:06.149111 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.149703 28757 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:13:06.150779 32708 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.151103 28764 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:13:06.151803 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.152706   546 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:05.974083   503 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.971s	user 0.000s	sys 0.012s
I20260709 13:13:06.084705 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:06.084900 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 381258us
I20260709 13:13:06.155139 28081 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:06.040931   543 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.147794 28077 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:13:06.155886   543 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.883117   541 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.122355   499 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:06.136538   312 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:06.157907   499 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.996574   335 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.159036 28083 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:06.159233   535 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:06.021848   550 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:13:06.160180   499 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:05.965034   525 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:13:05.913516 32706 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:06.092394 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0432718239999303s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:13:06.085587   373 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:13:06.085939   349 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.173547 28759 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:06.174971 28235 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:06.104635   353 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.114158 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:06.155049 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:05.895800   484 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.156392 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2409) took 3337 ms. Trace:
I20260709 13:13:06.158327   543 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.059334   529 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.156810 28077 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:06.158246   312 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:05.990754   409 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.159538 28758 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:06.159931   535 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.196036   535 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:05.899024   524 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:05.898175   540 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.201148   540 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.202585   529 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.203819   540 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.158636   541 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:06.118559   526 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.206459 28079 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:06.207195   526 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.156793 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:06.084338   426 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:13:06.155861 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.098594   395 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:06.165578 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:06.087497 28152 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.017s	sys 0.008s
W20260709 13:13:06.228711 28152 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.018s	sys 0.008s
W20260709 13:13:06.168241 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54572.4us
I20260709 13:13:06.228883 28228 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:06.173678   456 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:06.177834 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 93190.3us
I20260709 13:13:06.177538 28199 leader_election.cc:400] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
W20260709 13:13:06.237895 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 60559.1us
I20260709 13:13:06.239838 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:06.181167 28085 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:06.240995 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:06.184911 28762 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:06.185971 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:06.176717 28156 rpcz_store.cc:276] 0709 13:13:02.818572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:02.818703 (+   131us) service_pool.cc:224] Handling call
0709 13:13:02.819042 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:02.948771 (+129729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:02.948861 (+    90us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:04.283944 (+1335083us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b2c000b54b8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:13:04.290589 (+  6645us) log.cc:844] Serialized 52 byte log entry
0709 13:13:04.433035 (+142446us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:04.433079 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:04.433659 (+   580us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:05.973487 (+1539828us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:06.156139 (+182652us) raft_consensus.cc:1685] finished
0709 13:13:06.156161 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.156298 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7293,"raft.queue_time_us":151133,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":8325,"spinlock_wait_cycles":19445248,"thread_start_us":192747,"threads_started":2,"wal-append.queue_time_us":664050}
I20260709 13:13:06.188758 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:06.246040 28230 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:06.188220   538 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:06.117995   551 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.059948   532 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.164420   319 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:06.209151 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 52471.5us
I20260709 13:13:06.260039 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:06.210047 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 55158.3us
I20260709 13:13:06.261562 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:06.208855   547 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:13:06.184835   544 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:06.218044   312 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:06.186676   484 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:06.122342   549 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:13:06.220852   506 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.226517   349 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.178088 32739 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:13:06.227424 32706 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.247882 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:06.248522 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:06.235396   426 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:06.172477 28082 raft_consensus.cc:2410] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:13:06.229060   541 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:06.212227   492 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:13:06.251575 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 65730.1us
I20260709 13:13:06.249293 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:06.266410 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.303403   323 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:06.270260 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:06.307766 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.250587   529 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:06.322942   529 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.323392   529 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:06.323498 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.324806 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.200582   420 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.250759   535 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:06.314419 32607 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:13:06.317021 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 69224.2us
W20260709 13:13:06.330909 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 60754.9us
I20260709 13:13:06.329622 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2429) took 1273 ms. Trace:
W20260709 13:13:06.310529   351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.018s	sys 0.008s
W20260709 13:13:06.318301 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 69844.8us
I20260709 13:13:06.258498   480 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:06.290612   320 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:06.320416 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 71199.2us
W20260709 13:13:06.278277 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.320557 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:06.326802   553 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:06.247290   456 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.253553   373 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:13:06.311615 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.341110 28757 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:06.331243 32607 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:06.342067 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.342787 28757 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:06.343232   420 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.342818 32607 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.343833 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.344319 32607 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:06.331707 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:06.257932   538 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:06.319794   349 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:06.315755   426 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:06.355487   364 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:06.353591   544 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.334714 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:06.358024   538 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.359241 28125 leader_election.cc:400] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:13:06.359653 28125 leader_election.cc:403] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:06.360292   415 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:06.360765   415 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.361311   415 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:06.363238 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.363982   426 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.365509   426 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:06.365729 28759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1059) took 1029 ms. Trace:
I20260709 13:13:06.366173 28759 rpcz_store.cc:276] 0709 13:13:05.335986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.336641 (+   655us) service_pool.cc:224] Handling call
0709 13:13:05.349763 (+ 13122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.174015 (+824252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.365504 (+191489us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b5400722254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:06.365671 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1872499584}
I20260709 13:13:06.367479 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:06.355878   363 raft_consensus.cc:1068] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Replica 11f10c12c03e47dcac88b050d4c1431b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:13:06.370337 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:06.370553 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:06.371336   453 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:06.373054   453 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:13:06.374143   526 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.374687   526 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.376910   350 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.377104 32739 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:06.352443   553 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.377511   350 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.378041   553 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:06.379714   350 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.381076   335 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:06.383529   409 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:06.384012   409 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:06.384379   532 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.385066   554 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:06.386502 28761 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:06.332254 28152 rpcz_store.cc:276] 0709 13:13:05.056393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.058030 (+  1637us) service_pool.cc:224] Handling call
0709 13:13:05.094168 (+ 36138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.094364 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:05.393876 (+299512us) log.cc:844] Serialized 14 byte log entry
0709 13:13:06.328328 (+934452us) log.cc:844] Serialized 14 byte log entry
0709 13:13:06.329224 (+   896us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:06.329291 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:06.329349 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.329446 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.329570 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":299844,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":73240,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":1231036,"threads_started":4,"wal-append.queue_time_us":361638}
I20260709 13:13:06.355369 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:06.333897 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65466.9us
I20260709 13:13:06.336091 28756 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:06.321563   540 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:06.236425   395 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:06.334692   557 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.098s	user 0.000s	sys 0.002s
I20260709 13:13:06.344539 28757 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:06.345062   564 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:06.345778   549 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:06.337412   492 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
W20260709 13:13:06.330572 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 97056.1us
W20260709 13:13:06.335848   351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.018s	sys 0.008s
I20260709 13:13:06.331408   535 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:06.333863   451 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:06.391860   540 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.364492 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.390760 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:06.385892   554 consensus_queue.cc:1243] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:06.334712 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:06.392663 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:06.397382 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.392330   530 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:06.416679   484 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.391396   350 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:06.410848 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.364387   430 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.376802   551 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:06.356972   364 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.430564 28757 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:13:06.432353 28760 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:06.434887 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44180.6us
I20260709 13:13:06.406515   564 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:06.437831 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 45229.2us
I20260709 13:13:06.413724   526 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.447988   451 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:06.447230   554 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:06.390276   480 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:13:06.446447 28086 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:06.409647 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.483536 28080 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:06.419768   351 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:06.424650   535 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:06.392009   452 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.397034   353 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
W20260709 13:13:06.456060 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.428104   456 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:06.435643   540 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:06.470731 28762 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:13:06.406769   549 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:06.477113 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:06.439003   532 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:06.479717 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:06.447599   350 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:06.471670   364 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:06.473614   430 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:06.513367 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.515138 28757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
W20260709 13:13:06.526731 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 47096.2us
I20260709 13:13:06.436069   409 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.471370   564 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:06.524855 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.479032   451 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:06.527050 28757 raft_consensus.cc:606] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:06.527096 28756 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:06.533664 28757 raft_consensus.cc:614] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:06.535256 28761 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:13:06.535737 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:06.536262 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.473459   530 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:13:06.473865   363 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 13:13:06.549260 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 214562us
I20260709 13:13:06.539211 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:06.550302 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.482221 32676 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:13:06.547868 28756 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:13:06.550946 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:06.502136   554 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:06.531592   455 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:06.527091   526 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:06.499831   415 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.559409 28756 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:06.534924   480 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:06.527402   350 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:06.562915 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:06.563136 28757 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:06.542074   451 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:06.574172 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.577463 28756 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:06.547354   453 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
W20260709 13:13:06.582180 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.584877 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:06.585239 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:06.586330 28084 raft_consensus.cc:2410] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:13:06.587532 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:06.559605   492 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:06.588318 28084 raft_consensus.cc:2376] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12. Re-sending same reply.
I20260709 13:13:06.561273   532 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.593737 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.566548   526 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:06.595192 28086 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:06.580303   363 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:06.597674 28084 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:06.598796 28032 auto_rebalancer.cc:893] Removing replace marker failed: Replica 611ec64ec58244fe82058d6130e4e6f3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:13:06.592593 32739 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:06.595101   409 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:06.605616 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.610749 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:06.611440 28032 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:13:06.636431 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:06.597183   395 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.597760   363 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:06.601105   453 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:06.635198 28032 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:13:06.647868   524 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.595974   532 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.694957   532 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:06.696336   576 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:13:06.697558 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2396) took 1030 ms. Trace:
I20260709 13:13:06.697719   452 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.580019   549 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:06.678035   574 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:13:06.678818   580 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:06.680595   570 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:13:06.656086   526 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.683056 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.712301 28078 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:06.657013   484 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:06.715829   395 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.717662 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:06.717856   526 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:06.689361   363 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.691259 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:06.700516 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.721153 28763 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:06.646574 32739 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:13:06.700664 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 115708us
I20260709 13:13:06.698007 28082 rpcz_store.cc:276] 0709 13:13:05.667256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.672840 (+  5584us) service_pool.cc:224] Handling call
0709 13:13:06.697334 (+1024494us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54000e1d54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:06.697494 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":345345792}
I20260709 13:13:06.705974 28760 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:06.646114   456 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:13:06.592123   351 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.733510 32739 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:06.734602 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.736850   484 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:06.737855   452 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:06.738102   484 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:06.739655   484 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:06.740780   579 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.741040 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.741370   579 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.742605   456 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:06.743029   456 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:06.743940   579 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:06.725306 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:06.717996   395 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.745069 28078 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:06.745266   415 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.746467   395 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.747830   349 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:06.739379   452 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:06.698273   524 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.741963 28081 raft_consensus.cc:2410] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:13:06.730221 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:06.750519 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:06.752329 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.613617   554 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.753240 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.713970   549 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:06.761828 28082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:06.761965 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.762651 28082 raft_consensus.cc:606] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:06.757612 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.737457 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:06.763031 28082 raft_consensus.cc:614] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:06.744576   579 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.754307 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:06.763796 28756 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:06.765772 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.768453 28198 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:06.769845 28198 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:06.771450 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:06.771864 28758 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:06.765847 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:06.773203 28757 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:06.773247 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.775913 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:06.777420 28763 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:06.779978 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:06.781527 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.782233 28156 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:13:06.763098   312 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:06.785738 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:06.763002   363 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:06.811139 28198 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:06.804401 28156 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:06.809778 28153 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:06.787603 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:06.714972   580 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.783730   522 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:06.791103 28159 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 6. Current term is 8. Ops: []
I20260709 13:13:06.790762 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.790073 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.848798 28161 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:06.811364 28082 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:13:06.816951 28757 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 8.
I20260709 13:13:06.811990 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:06.818496 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.818667 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.776185 32676 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:06.820341   349 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:06.820873   547 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:06.739840   364 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.779829   524 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:06.866832   553 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.867995   373 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:06.868491   373 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:06.824949   312 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:06.879614 28155 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 16: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:06.779000   581 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:06.825582   480 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:06.813172   452 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:06.817241   583 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:06.827508   510 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:06.831108   564 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.833299 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.838466   456 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:06.844367 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:06.777218   319 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:06.845489 28756 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:13:06.847038 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:06.954788 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.872567   455 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:06.854420 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.961395 28157 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:06.856218 28124 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:06.962715 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 106586us
I20260709 13:13:06.860107 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.965744 28161 raft_consensus.cc:2410] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:13:06.854066   349 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.861325 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.863416   364 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:06.880039 28759 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:06.811431   551 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:06.926460 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.981349 28154 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:13:06.840330   540 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:06.930799 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:06.983876 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.777545   579 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.986624 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:06.987207 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 175275us
I20260709 13:13:06.987922 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.949272 28160 raft_consensus.cc:2410] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:13:06.857261   430 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:06.791117 28764 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:06.852633 28153 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:06.970942   535 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:06.971685   575 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:06.832612   451 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:06.974560 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:06.975458   529 consensus_meta.cc:287] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.000s
I20260709 13:13:06.945900   547 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:06.977710 28078 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:07.082316 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:06.985479   349 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:06.985539   415 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.924785   586 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:06.999871 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:07.005707 28156 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:07.007193 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:06.948895   583 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:07.008980 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.009733 28084 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:07.010805 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.011410 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:07.075719 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.070132 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.089840 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.109431 28232 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:07.108003 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.110396 28154 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:07.110451 28235 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:07.111415 28228 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:06.954181   319 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.948432 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:06.943584 32676 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:13:06.985802   409 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:13:07.083247 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72488.3us
I20260709 13:13:07.087301 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:06.991042 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:06.997885   430 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.958518   452 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:06.998029   313 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:07.065160 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.113282 28159 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:07.112984 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:06.942008   373 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.087766 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.120697   553 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:07.123206   540 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:07.084512   541 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.198874   541 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.201386   541 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:07.201826   585 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:13:07.201913   409 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:07.124665 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.125751   451 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:07.127341   580 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:07.127717   579 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:07.128768   564 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:07.212854 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:06.996570   312 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.129310 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.133059 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.213559 28156 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:07.214149 28227 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:07.007129   456 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:07.216238 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:07.133919 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 185595us
I20260709 13:13:07.217269 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:07.112944 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 253758us
I20260709 13:13:07.133970 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.085035   543 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.137645 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.220189 28228 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:07.161525 28161 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:07.167172 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:07.222210 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.223841 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.204818   535 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:07.086190   547 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:07.171949 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:07.224663 28227 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:07.226907 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.115295   415 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.227559 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.237600 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.088176   581 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.116s
I20260709 13:13:07.178973 28153 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:13:07.258435 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:07.181021 28230 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:07.182264 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.111416   395 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.133311   319 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:07.192876 28160 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:07.149657   532 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.118299   586 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:07.197350 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.206102 28234 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:07.235772   555 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.018s	user 0.000s	sys 0.006s
I20260709 13:13:07.208897 28154 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:07.152491   430 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:07.085197   529 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.115588   583 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:07.203064   409 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.214818 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:07.182437   524 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:07.254817 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 82984us
I20260709 13:13:07.309180 28080 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:13:07.301457 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.309979 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:07.310940 28728 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:07.311534 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:07.311863 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:07.312121 28084 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:07.237847 28756 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:07.255342 28156 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:07.242705 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.313870 28228 raft_consensus.cc:2410] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:07.178349 28231 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:07.263545 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2410) took 1172 ms. Trace:
I20260709 13:13:07.314836 28083 rpcz_store.cc:276] 0709 13:13:06.090731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.139895 (+ 49164us) service_pool.cc:224] Handling call
0709 13:13:07.263311 (+1123416us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b54000ca3d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:07.263469 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41534336}
I20260709 13:13:07.315375 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.315920 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:07.270148 28235 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:07.271643 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.274060 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2401) took 1532 ms. Trace:
I20260709 13:13:07.277874 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:07.317495 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:07.317646 28079 rpcz_store.cc:276] 0709 13:13:05.742068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.149942 (+407874us) service_pool.cc:224] Handling call
0709 13:13:06.150231 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.206898 (+ 56667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.273822 (+1066924us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b54000c8fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:07.273999 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128843520}
I20260709 13:13:07.318318 28083 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:07.318902 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:07.281213 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:07.321530 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.321838 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:13:07.322773 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 108008us
I20260709 13:13:07.286073 28232 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
W20260709 13:13:07.252051   350 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.002s
W20260709 13:13:07.288264 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.322209 28233 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:07.204059 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:07.293545 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.272696   420 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:07.296979 28198 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:07.226028   323 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:07.219121   373 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:07.301976 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:07.306545 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.336440 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:07.337051 28079 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:07.301162 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.247810   538 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:07.337855   538 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:07.338515   538 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.339171   538 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:07.322945 28084 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:07.282366 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:07.323743 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:07.288453   564 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:07.282501 32739 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:07.324290 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.325208 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.323764 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:07.327071 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 123075us
I20260709 13:13:07.329974 28227 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:07.334206 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:07.321748   320 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:07.322021   541 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:07.229627   579 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:13:07.363166 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:07.364013 28232 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:07.340767 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:07.342391 28082 raft_consensus.cc:2410] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:13:07.362104 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:07.340682 28234 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:07.345204 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:07.367280 28160 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:07.373193 28234 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:07.324708   335 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:07.362082 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:07.342948 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:07.349043 28756 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:07.344079   363 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:07.350674 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.351861   590 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:07.287191   395 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.321432   546 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:07.299191   532 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.364075 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:07.293257   586 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:07.375160 28761 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:07.225003   452 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.375528 28084 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
W20260709 13:13:07.418174 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.352820 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:07.298911   580 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:13:07.377341 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:07.422964 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45720.5us
I20260709 13:13:07.423360 28234 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:13:07.360140 32200 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:07.342109   323 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:07.378475 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:07.431663 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.334543   313 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:07.381245 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.299801   364 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:13:07.302312 32676 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:13:07.382347 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.447678 28230 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:07.337218   351 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:07.359025 28079 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
W20260709 13:13:07.420715 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 56700.9us
I20260709 13:13:07.381512   363 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.264092   543 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.425895 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.424290 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:07.427246 32607 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:07.432250 28083 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:13:07.426645   579 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:07.324949   595 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:13:07.435389 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 91252.7us
I20260709 13:13:07.383970   496 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:07.383361 28232 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:13:07.440342 28086 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:07.458822   549 consensus_meta.cc:287] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.001s
I20260709 13:13:07.452584 28234 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:13:07.335736   350 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:07.434477   323 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:07.461180 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:07.373612   541 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.439492   313 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:07.374006   547 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.424784   546 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:07.507808   546 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:07.508013   547 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:07.484104 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.510118   551 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:07.510644   551 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:07.510771 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.511446 28230 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:07.512302 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.514674 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.515419 28230 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:07.486749 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 134022us
I20260709 13:13:07.487792 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.489281 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:07.450275   349 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.495164   313 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:07.497225   350 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:07.497720 28077 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:07.487927   541 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:07.500964 28160 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:07.501425   592 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:13:07.503286 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.517485 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:07.519356 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.519351 28229 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:07.474476   335 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:07.521931 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:07.521121 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:07.529824 28082 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:07.380221   535 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:07.530339 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.523218 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:07.532871 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:07.464075   363 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:07.525575 28229 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:07.520995 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.535154 28756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:07.528445 32708 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:07.531314 28159 raft_consensus.cc:2376] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13. Re-sending same reply.
I20260709 13:13:07.584211 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.584882 28756 raft_consensus.cc:606] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:07.585518 28228 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:07.465807   579 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:07.528463   592 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:07.537595 28232 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:13:07.520323   526 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:13:07.601783 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.602035 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.586136 28756 raft_consensus.cc:614] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:07.584911 28762 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:07.528712   564 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:07.474828 32200 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:07.584973 28159 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:07.588476 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:07.545396   529 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:07.584125   496 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:07.527740   522 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:07.598780   579 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:07.539290   586 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.617774 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.583204   320 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:07.589401 32708 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.529099   335 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.589054   350 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:07.632005 28234 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
W20260709 13:13:07.714345 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.547181 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:07.589293   602 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:13:07.644052 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.615868 28232 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:07.603202   415 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:07.675751 28159 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:07.682945 28230 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:07.631357   532 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:07.634930   395 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:07.729143 28756 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: [12.12-15.14]
I20260709 13:13:07.732064 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:07.662492   456 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:07.661463   530 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:13:07.752841 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.736631 28231 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:07.662914   605 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:07.738716 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:07.713451 32708 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:07.754076 28159 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:07.734038   599 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:13:07.665190   522 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.721287   335 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:07.663700   554 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 13:13:07.721065   351 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:07.726893   364 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.764577   480 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:07.774700 32200 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:07.775379 32607 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:07.758878 28234 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:13:07.777443 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.738993   602 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:07.758423   530 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:07.752007   456 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:07.797735   597 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:13:07.722978   592 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.766317   605 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.769891   526 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:07.776222   426 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:07.766835   323 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.732064   529 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
W20260709 13:13:07.866511 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:07.780596 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 137500us
I20260709 13:13:07.823546 28157 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:07.818202   611 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:07.797725   320 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.822418 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.826889 28086 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:13:07.836302 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 289214us
I20260709 13:13:07.878188 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:07.851882 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73904.4us
I20260709 13:13:07.807097   420 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:07.811627   543 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:07.813287   349 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:07.884521   323 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:07.885406 28161 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:07.853856   499 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:07.830458 32676 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 13:13:07.851087   319 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.549s	user 0.005s	sys 0.005s
I20260709 13:13:07.810151   607 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:13:07.885808 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:07.852674   373 consensus_meta.cc:287] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.000s
I20260709 13:13:07.816576   364 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.886760 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:07.876963   592 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:07.818291 32200 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.837198   456 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:07.901386 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.901422 28762 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:13:07.904187 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.884963   420 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:13:07.815106   541 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.919946   602 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.926244   541 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:07.913528 28159 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:07.889735 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:07.927491 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:07.927855 28232 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:07.928803 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.928844   530 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.930294   530 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:07.931354   363 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.933084   629 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.933363   605 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:07.934005   605 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:07.933672   629 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.934463   605 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:07.935629 28230 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:13:07.935879   629 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:07.936803   526 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.935346   605 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.832572   496 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:07.910784   319 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:07.940409 28761 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:13:07.915406 28153 raft_consensus.cc:2410] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:13:07.896234   499 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:07.899330   312 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:07.930097 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:07.944682 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:07.945338 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:07.913658   620 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:07.964778 28086 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:07.961256 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2442) took 1130 ms. Trace:
I20260709 13:13:07.966046 28159 rpcz_store.cc:276] 0709 13:13:06.830789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.929843 (+1099054us) service_pool.cc:224] Handling call
0709 13:13:07.961172 (+ 31329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:07.967860 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:07.969823 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:07.970830 28234 raft_consensus.cc:2410] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:13:07.969930 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 876) took 1181 ms. Trace:
I20260709 13:13:07.971309 28155 rpcz_store.cc:276] 0709 13:13:06.788432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.788773 (+   341us) service_pool.cc:224] Handling call
0709 13:13:07.969722 (+1180949us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b5400376ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:07.969857 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42941184}
I20260709 13:13:07.979914 28764 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:07.980485 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:07.979848 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 95698.9us
I20260709 13:13:07.872412   351 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.020339 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:08.020853 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.020785 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:07.915495 32607 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:07.976186   625 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:07.915285   623 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 13:13:07.919466   546 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:08.024482 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.952235   618 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:08.024312   586 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:07.964097   312 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:07.952657   373 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:13:07.970628   426 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:08.035776 28234 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:08.036231 28155 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:08.102407 28759 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
W20260709 13:13:08.024606   622 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:13:08.040961   319 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:08.042469   625 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:08.022644   610 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.049569 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:08.117471 28086 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:08.117398   610 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.119854   610 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:08.028908 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:08.122077 28159 raft_consensus.cc:2410] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:13:08.122846 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2549) took 1023 ms. Trace:
I20260709 13:13:08.122994 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:08.123358 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2100) took 1727 ms. Trace:
W20260709 13:13:08.124442 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.126118 28159 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:08.049387   313 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.512s	user 0.002s	sys 0.008s
I20260709 13:13:08.052122   320 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:13:08.133490   526 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.125656   625 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:08.103606   456 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.136711   526 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.139482   456 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:08.108579   524 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.141276   524 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.159013 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:08.159862 28078 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:08.170141 28155 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:08.170854 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2549) took 1041 ms. Trace:
I20260709 13:13:08.171226 28155 rpcz_store.cc:276] 0709 13:13:07.129026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.972478 (+843452us) service_pool.cc:224] Handling call
0709 13:13:08.170779 (+198301us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:08.038934   551 consensus_meta.cc:287] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.002s
I20260709 13:13:07.978431   620 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.074052   312 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:08.124276 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:08.123302 28227 rpcz_store.cc:276] 0709 13:13:07.099064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.362666 (+263602us) service_pool.cc:224] Handling call
0709 13:13:07.418062 (+ 55396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:07.681748 (+263686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:07.689841 (+  8093us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:07.730054 (+ 40213us) log.cc:844] Serialized 26 byte log entry
0709 13:13:07.916335 (+186281us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:07.916380 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.916448 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:08.122590 (+206142us) raft_consensus.cc:1685] finished
0709 13:13:08.122614 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:08.122776 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2892,"raft.queue_time_us":67403,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":29255,"thread_start_us":328880,"threads_started":2,"wal-append.queue_time_us":252365}
I20260709 13:13:08.127923 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:08.123812 28085 rpcz_store.cc:276] 0709 13:13:06.395900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.396410 (+   510us) service_pool.cc:224] Handling call
0709 13:13:07.263654 (+867244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:07.264152 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:07.264192 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:07.294430 (+ 30238us) log.cc:844] Serialized 26 byte log entry
0709 13:13:07.521904 (+227474us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:07.521949 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.522026 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:07.801805 (+279779us) raft_consensus.cc:1685] finished
0709 13:13:07.801826 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:08.123120 (+321294us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54000ca3d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:08.123303 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12925,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":11732,"spinlock_wait_cycles":1993200000,"thread_start_us":175827,"threads_started":1,"wal-append.queue_time_us":385571}
I20260709 13:13:08.036512   541 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:08.133198 28086 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:08.032639   496 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.101794   363 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:08.136304   624 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:13:08.177634 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:08.184985 28155 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:08.185487 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2449) took 1352 ms. Trace:
I20260709 13:13:08.185750 28155 rpcz_store.cc:276] 0709 13:13:06.832721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.177375 (+1344654us) service_pool.cc:224] Handling call
0709 13:13:08.185429 (+  8054us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.179795 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:08.105156   426 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.099418   499 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.076926   618 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.201448 28078 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:08.135766   313 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.207195 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:08.207939 28155 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:08.204969 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:08.212270   605 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:08.213124   605 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:08.214197   620 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:08.214040   605 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.214576   546 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:08.217473   364 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:08.219583   524 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:08.219900 28229 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.225219 28234 raft_consensus.cc:2410] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:13:08.225376 28229 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:08.230129 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:08.230301 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:08.230857 28757 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:08.233117 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:08.232883 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:08.235122 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:08.235615 28155 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:13:08.238168   430 consensus_meta.cc:287] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.924s	user 0.011s	sys 0.002s
I20260709 13:13:08.238623   430 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.241225   564 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:08.241714   564 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:08.241932 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:08.242494   335 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.243254 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:08.243791 28154 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:08.244236   312 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:08.245849 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.246006   313 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:08.246980   351 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.247363   629 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:08.247942   629 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:08.247498   351 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.249207 28226 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:08.249383 28085 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:13:08.249755   351 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:08.180404   551 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.252074 28085 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:08.254200 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:08.256120 28078 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:08.257673 28086 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:13:08.205063   499 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:08.261101   426 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:08.269605 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:08.270579 28155 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:08.260841   430 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:08.261222 28085 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:08.260195 28154 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:08.183912   541 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.245788 32739 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.262117 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:08.270712 28078 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:08.280742 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:08.270851 28079 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:08.281132 28084 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:08.281594 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:08.259482 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:08.272280 28155 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:13:08.274566 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.277875 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:08.283496 28156 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:08.279066 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:08.244616   335 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:08.327242 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.433329 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2443) took 1602 ms. Trace:
I20260709 13:13:08.344408 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:08.353392 28127 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.561s (SENT) [suppressed 28 similar messages]
I20260709 13:13:08.434012 28155 rpcz_store.cc:276] 0709 13:13:06.831074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.271793 (+1440719us) service_pool.cc:224] Handling call
0709 13:13:08.272108 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:08.288131 (+ 16023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.433252 (+145121us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:08.326967 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.259024   524 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.449182   524 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.285328   640 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:08.285173   627 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:08.371698   638 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:08.269313   630 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:13:08.367697 32708 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.604s	user 0.008s	sys 0.002s
W20260709 13:13:08.404464   583 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.082s	user 0.009s	sys 0.007s
I20260709 13:13:08.285303   626 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.567502 28229 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:08.568445 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2144) took 1373 ms. Trace:
I20260709 13:13:08.285785 32739 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:08.264918   629 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.310850 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:08.571696 28079 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
W20260709 13:13:08.285316   600 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.505s	user 0.000s	sys 0.004s
I20260709 13:13:08.432407 28159 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:08.336246 28154 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:13:08.590161 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2550) took 1460 ms. Trace:
I20260709 13:13:08.352196   312 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:08.308465   617 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:08.590553 28159 rpcz_store.cc:276] 0709 13:13:07.129286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.122727 (+993441us) service_pool.cc:224] Handling call
0709 13:13:08.590097 (+467370us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:08.357901   579 consensus_meta.cc:287] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.665s	user 0.007s	sys 0.004s
W20260709 13:13:08.346414 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 64857.8us
I20260709 13:13:08.590734 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2533) took 1791 ms. Trace:
I20260709 13:13:08.591815 28154 rpcz_store.cc:276] 0709 13:13:06.799533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.280351 (+1480818us) service_pool.cc:224] Handling call
0709 13:13:08.335934 (+ 55583us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:08.590457 (+254523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.590684 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.438118 28086 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:08.439175   541 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:08.440090   644 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:13:08.440806   351 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:08.442478 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.596810 28159 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:13:08.449497 28155 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
W20260709 13:13:08.460515 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:08.597967 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.568775 28229 rpcz_store.cc:276] 0709 13:13:07.195431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.443799 (+248368us) service_pool.cc:224] Handling call
0709 13:13:07.466344 (+ 22545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.219499 (+753155us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b4c0043f4c0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cc943a20 00007f59cc954a29 00007f59cc91a644 00007f59cc941f2d 00007f59cc941b93 00007f59ce67218b 00007f59ce672cb5 00007f59ce66df30 00007f59ce66f7de 00007f59ce679b49 00007f59ce67a416
0709 13:13:08.568084 (+348585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.568377 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11050752}
W20260709 13:13:08.533566 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 4c9b061a7ca84a61ac13d29946a28169: Illegal state: Replica 0faaace2b36c4bcebf2fa6835235e92b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:08.574842 28756 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:08.575924 28762 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:08.577116   524 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:08.577790 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 224426us
I20260709 13:13:08.578855 28757 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:08.578971 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:08.579928 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:08.584899   626 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.566727   627 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:08.566536 32708 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:08.577042   645 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:13:08.609112   335 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.638468 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:08.587502   640 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:08.278661   320 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:08.592450 28728 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:08.606464 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2446) took 1774 ms. Trace:
I20260709 13:13:08.345470 28156 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:13:08.606616 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2445) took 1775 ms. Trace:
I20260709 13:13:08.568887   625 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.468398   638 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:08.592787   595 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:13:08.637373 28760 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:08.635233 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2447) took 1803 ms. Trace:
W20260709 13:13:08.637543 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.591063   617 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.714219 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2421) took 1458 ms. Trace:
I20260709 13:13:08.714956 28080 rpcz_store.cc:276] 0709 13:13:07.255571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.256223 (+   652us) service_pool.cc:224] Handling call
0709 13:13:07.303482 (+ 47259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:07.309823 (+  6341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.713879 (+1404056us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b540017c850. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:08.714119 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83373824}
I20260709 13:13:08.686847 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2444) took 1855 ms. Trace:
I20260709 13:13:08.685616 28155 rpcz_store.cc:276] 0709 13:13:06.831597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.447279 (+1615682us) service_pool.cc:224] Handling call
0709 13:13:08.448694 (+  1415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.606363 (+157669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.606563 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.688028   541 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:08.687784 28758 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:13:08.651870 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59489.9us
I20260709 13:13:08.689309 28084 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:08.685556 28159 rpcz_store.cc:276] 0709 13:13:06.831869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.596053 (+1764184us) service_pool.cc:224] Handling call
0709 13:13:08.596486 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:08.605970 (+  9484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.606399 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.690680 32708 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:08.694321   480 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:08.695751 28079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c654c99561449cb823642a1b85879ac"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:08.570225   583 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:08.644848   647 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:13:08.706396 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2105) took 2301 ms. Trace:
I20260709 13:13:08.644202   335 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.705850 28154 rpcz_store.cc:276] 0709 13:13:06.832158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.594755 (+1762597us) service_pool.cc:224] Handling call
0709 13:13:08.634625 (+ 39870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.634859 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:08.634896 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:08.634947 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:08.634987 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.635058 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:08.635175 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.594513   579 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.638834   627 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:08.787922 28155 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
W20260709 13:13:08.755512 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.746457 28158 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:13:08.787206 28079 raft_consensus.cc:606] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:08.806524 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2448) took 1974 ms. Trace:
I20260709 13:13:08.806870 28079 raft_consensus.cc:614] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:08.789675 28154 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:08.757135 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2423) took 1501 ms. Trace:
I20260709 13:13:08.808035 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2539) took 1883 ms. Trace:
I20260709 13:13:08.806989 28158 rpcz_store.cc:276] 0709 13:13:06.832430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.745592 (+1913162us) service_pool.cc:224] Handling call
0709 13:13:08.746082 (+   490us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:08.806193 (+ 60111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.806463 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.808193 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2536) took 1952 ms. Trace:
W20260709 13:13:08.739205 28077 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2101) took 2343 ms (client timeout 1664 ms). Trace:
I20260709 13:13:08.788620 28159 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:08.812760 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2451) took 1935 ms. Trace:
I20260709 13:13:08.808691 28155 rpcz_store.cc:276] 0709 13:13:06.856027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.787433 (+1931406us) service_pool.cc:224] Handling call
0709 13:13:08.787746 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.807906 (+ 20160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.808143 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.813531 28159 rpcz_store.cc:276] 0709 13:13:06.877427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.787832 (+1910405us) service_pool.cc:224] Handling call
0709 13:13:08.788263 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.812469 (+ 24206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.812680 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.814303 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:08.747613 28153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2529) took 1946 ms (client timeout 2069 ms). Trace:
W20260709 13:13:08.815088 28153 rpcz_store.cc:269] 0709 13:13:06.797966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.280935 (+482969us) service_pool.cc:224] Handling call
0709 13:13:08.744415 (+1463480us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b540014f854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:08.744635 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1365966080}
W20260709 13:13:08.807524 28050 connection.cc:256] RPC call timeout handler was delayed by -1.1313837069999408s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
I20260709 13:13:08.816553 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:08.817477 28153 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:08.808210 28083 rpcz_store.cc:276] 0709 13:13:07.256142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.343211 (+ 87069us) service_pool.cc:224] Handling call
0709 13:13:07.343551 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:07.461849 (+118298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.756658 (+1294809us) spinlock_profiling.cc:243] Waited 88.3 ms on lock 0x7b54001ad454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:08.757049 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203165952}
I20260709 13:13:08.716943 28156 rpcz_store.cc:276] 0709 13:13:06.831347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.284396 (+1453049us) service_pool.cc:224] Handling call
0709 13:13:08.345133 (+ 60737us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:08.686479 (+341346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.686787 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.738430 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2140) took 1545 ms. Trace:
I20260709 13:13:08.747479   580 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:08.823172   580 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:08.640915   351 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.823768 28155 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:08.823210 32708 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:08.827890 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:08.821372 28228 rpcz_store.cc:276] 0709 13:13:07.193612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.324638 (+131026us) service_pool.cc:224] Handling call
0709 13:13:08.738092 (+1413454us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54003917d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:08.738316 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514963200}
W20260709 13:13:08.726200   649 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:13:08.824431 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2548) took 1695 ms. Trace:
W20260709 13:13:08.810227 28077 rpcz_store.cc:269] 0709 13:13:06.396263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.397070 (+   807us) service_pool.cc:224] Handling call
0709 13:13:08.738956 (+2341886us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540017c354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:08.739114 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2444030208}
I20260709 13:13:08.832542 28155 rpcz_store.cc:276] 0709 13:13:07.128666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.822449 (+1693783us) service_pool.cc:224] Handling call
0709 13:13:08.822780 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.824189 (+  1409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.824380 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:08.816926 28050 connection.cc:256] RPC call timeout handler was delayed by -1.109873258999869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1947s for a response
I20260709 13:13:08.650784   551 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:08.811444 28154 rpcz_store.cc:276] 0709 13:13:06.924788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.788187 (+1863399us) service_pool.cc:224] Handling call
0709 13:13:08.788504 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.807805 (+ 19301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.807985 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.787608 28081 rpcz_store.cc:276] 0709 13:13:06.405234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.775465 (+370231us) service_pool.cc:224] Handling call
0709 13:13:06.775893 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:06.778127 (+  2234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.778159 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:06.788343 (+ 10184us) log.cc:844] Serialized 26 byte log entry
0709 13:13:07.239052 (+450709us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b1800334a20. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:13:07.464570 (+225518us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:07.464632 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.464749 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:08.163231 (+698482us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:08.706146 (+542915us) raft_consensus.cc:1685] finished
0709 13:13:08.706171 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:08.706330 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1846,"raft.queue_time_us":2100,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":32713,"spinlock_wait_cycles":43943040,"thread_start_us":225283,"threads_started":1,"wal-append.queue_time_us":799564}
I20260709 13:13:08.825383 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:08.707301   638 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:08.650310   312 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:08.759505   626 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:08.805626 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:08.851255 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 216752us
W20260709 13:13:08.851789 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46254.7us
I20260709 13:13:08.739068   564 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.831017 28153 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:08.860419 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:08.838779 28155 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:08.863611 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2554) took 1733 ms. Trace:
W20260709 13:13:08.839762 28050 connection.cc:256] RPC call timeout handler was delayed by -1.0417453399998067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1907s for a response
I20260709 13:13:08.840056 28158 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:08.864184 28155 rpcz_store.cc:276] 0709 13:13:07.130328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.837982 (+1707654us) service_pool.cc:224] Handling call
0709 13:13:08.838537 (+   555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.863271 (+ 24734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.863537 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.842062 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:08.842761 28154 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:13:08.844763   351 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:08.746162   617 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:08.735037   596 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.314s	user 0.006s	sys 0.000s
I20260709 13:13:08.810619   524 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:08.802390   541 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:08.703869   625 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.721208   640 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.830699 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:08.865530 28154 pending_rounds.cc:85] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 5
I20260709 13:13:08.725306   510 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.124s
I20260709 13:13:08.828429 32708 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:08.866590 28155 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:08.881698 28154 pending_rounds.cc:107] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 10.6
W20260709 13:13:08.885015 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.887491 28154 raft_consensus.cc:2889] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:08.868242 28050 connection.cc:256] RPC call timeout handler was delayed by -1.0093453399997543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
I20260709 13:13:08.889695 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 883) took 1674 ms. Trace:
W20260709 13:13:08.889998 28050 connection.cc:256] RPC call timeout handler was delayed by -1.00598214799993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
I20260709 13:13:08.890498 28155 rpcz_store.cc:276] 0709 13:13:07.215374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.865734 (+1650360us) service_pool.cc:224] Handling call
0709 13:13:08.866402 (+   668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.889289 (+ 22887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.889566 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:08.891068 28050 connection.cc:256] RPC call timeout handler was delayed by -1.0000732589999188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
I20260709 13:13:08.891722 28158 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:08.892964 28158 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:08.893916 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2453) took 1618 ms. Trace:
I20260709 13:13:08.892441 28155 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:08.894389 28158 rpcz_store.cc:276] 0709 13:13:07.275782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.892494 (+1616712us) service_pool.cc:224] Handling call
0709 13:13:08.892809 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.893569 (+   760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.893838 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.895479 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2556) took 1634 ms. Trace:
I20260709 13:13:08.896042 28155 rpcz_store.cc:276] 0709 13:13:07.261311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.891597 (+1630286us) service_pool.cc:224] Handling call
0709 13:13:08.892037 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.895141 (+  3104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.895407 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.896809 28158 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:08.894929 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:08.897516 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2558) took 1570 ms. Trace:
I20260709 13:13:08.900009 28158 rpcz_store.cc:276] 0709 13:13:07.327080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.896083 (+1569003us) service_pool.cc:224] Handling call
0709 13:13:08.896601 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.897259 (+   658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.897464 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.869614   524 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:08.889415   580 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.907205 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 868) took 2126 ms. Trace:
W20260709 13:13:08.906560   651 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.268s	user 0.003s	sys 0.000s
I20260709 13:13:08.720121 32739 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.859879 32200 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:08.955993 32200 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:08.907619 28152 rpcz_store.cc:276] 0709 13:13:06.780785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.780957 (+   172us) service_pool.cc:224] Handling call
0709 13:13:06.813495 (+ 32538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:06.813869 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.813902 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:06.859738 (+ 45836us) log.cc:844] Serialized 26 byte log entry
0709 13:13:07.189579 (+329841us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:07.434629 (+245050us) log.cc:844] Serialized 14 byte log entry
0709 13:13:07.435044 (+   415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.435329 (+   285us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:08.734156 (+1298827us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:08.906869 (+172713us) raft_consensus.cc:1685] finished
0709 13:13:08.906972 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:08.907144 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10381,"prepare.queue_time_us":414181,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":9695,"raft.queue_time_us":197,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":13960,"spinlock_wait_cycles":6857472,"thread_start_us":410979,"threads_started":2,"wal-append.queue_time_us":374715}
I20260709 13:13:08.909189 28227 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:13:08.913592 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2555) took 1783 ms. Trace:
I20260709 13:13:08.908397   524 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:08.965088 28154 rpcz_store.cc:276] 0709 13:13:07.130588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.841581 (+1710993us) service_pool.cc:224] Handling call
0709 13:13:08.842165 (+   584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.888344 (+ 46179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.913530 (+ 25186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277120}
W20260709 13:13:08.913076 28161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2552) took 1783 ms (client timeout 1716 ms). Trace:
W20260709 13:13:08.967034 28161 rpcz_store.cc:269] 0709 13:13:07.129811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.237253 (+107442us) service_pool.cc:224] Handling call
0709 13:13:08.912771 (+1675518us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54001973d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:08.912979 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453236608}
I20260709 13:13:08.957335 32200 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:08.883414   499 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.000s
W20260709 13:13:08.908532 28226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 2543) took 1819 ms (client timeout 1608 ms). Trace:
I20260709 13:13:08.869001   430 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.907635   580 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:08.880858   605 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.871759   335 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:08.961613   351 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:08.875129   625 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:08.872954   579 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:08.882251   583 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:08.878242   540 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:08.857926   312 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:08.983968 28226 rpcz_store.cc:269] 0709 13:13:07.089357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.089527 (+   170us) service_pool.cc:224] Handling call
0709 13:13:08.908275 (+1818748us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001c7fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:08.908459 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2619689728}
I20260709 13:13:08.878206   564 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:08.909615   544 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.871068   541 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:08.913865   617 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.886950   546 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:08.982908   602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.056s	user 0.023s	sys 0.049s
W20260709 13:13:09.013265 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.994238 28153 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
W20260709 13:13:08.999050 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.881417 32708 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:08.993975 28157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2440) took 2163 ms (client timeout 1842 ms). Trace:
W20260709 13:13:09.016333 28157 rpcz_store.cc:269] 0709 13:13:06.830243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.962263 (+132020us) service_pool.cc:224] Handling call
0709 13:13:08.993702 (+2031439us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54000a32d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:08.993880 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502848384}
I20260709 13:13:09.001263   351 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:08.989558   430 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.008692   349 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:08.993033   605 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:09.015703   602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.089s	user 0.023s	sys 0.050s
I20260709 13:13:09.012053   312 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:09.005043   540 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.043319 32739 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:09.027570   349 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:09.014739   617 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.056555   540 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:09.036716 32708 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:09.056830   617 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.059650   605 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.036993   579 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.061134   579 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.014045   544 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.015903   546 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.070429 28758 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:13:09.053411   551 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.072382   349 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.072753   551 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.075124 28759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1003) took 1610 ms. Trace:
I20260709 13:13:09.075655 28759 rpcz_store.cc:276] 0709 13:13:07.464924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.465080 (+   156us) service_pool.cc:224] Handling call
0709 13:13:07.483702 (+ 18622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.102856 (+619154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:09.074879 (+972023us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b54006504d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:09.075058 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422357888}
I20260709 13:13:09.051889   607 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.077766   607 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.062125   430 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:09.075218   551 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.090390   526 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:09.093878   618 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.095180   522 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.079741   351 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:09.103328   632 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.068s	user 0.000s	sys 0.003s
I20260709 13:13:09.105662   627 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.107893   349 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:09.108677   629 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:09.110661   638 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.112920   544 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.114352   627 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:09.119079   638 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:09.100378   546 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:09.119534   579 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:08.960457 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65588.8us
I20260709 13:13:09.197240 28156 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:08.992810 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 322502us
W20260709 13:13:09.200623 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.205613 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:13:09.209146 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.209439 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.229817 28758 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:09.232831 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:09.238560 28759 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:13:09.159778   350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.403s	user 0.002s	sys 0.007s
W20260709 13:13:09.240482   350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.488s	user 0.002s	sys 0.008s
I20260709 13:13:09.240855   350 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:09.242746   350 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.251034   579 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.251389 28050 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:09.251950   579 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.255338   579 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:09.256071   579 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.263700   579 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.265892 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 937) took 1226 ms. Trace:
I20260709 13:13:09.262459 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 884) took 2046 ms. Trace:
I20260709 13:13:09.267616 32708 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.269412 32708 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.266374 28234 rpcz_store.cc:276] 0709 13:13:08.038846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.101672 (+ 62826us) service_pool.cc:224] Handling call
0709 13:13:09.265337 (+1163665us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b5400306454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:09.265652 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102671616}
I20260709 13:13:09.269905   607 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:09.266474 28159 rpcz_store.cc:276] 0709 13:13:07.215721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.882233 (+1666512us) service_pool.cc:224] Handling call
0709 13:13:08.882580 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.199068 (+316488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:09.199125 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:09.199248 (+   123us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:09.199397 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:09.261750 (+ 62353us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54000a28d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:09.261839 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:09.262155 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725927168}
I20260709 13:13:09.265904 28756 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:09.266541 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:09.285568   351 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:09.303297 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:09.310524 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2558) took 1420 ms. Trace:
I20260709 13:13:09.311295 28227 rpcz_store.cc:276] 0709 13:13:07.890203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.177297 (+287094us) service_pool.cc:224] Handling call
0709 13:13:08.908847 (+731550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.966533 (+ 57686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:09.309811 (+343278us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54001c7fd0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:09.310157 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653096832}
W20260709 13:13:09.312953   664 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.139s	user 0.001s	sys 0.001s
I20260709 13:13:09.313625 28084 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:13:09.314179 28084 pending_rounds.cc:85] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 7
I20260709 13:13:09.314512 28084 pending_rounds.cc:107] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:13:09.314868 28084 raft_consensus.cc:2889] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:09.315776 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1002) took 1853 ms. Trace:
I20260709 13:13:09.316604 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:09.317785 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66762.9us
I20260709 13:13:09.318039 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 817) took 1782 ms. Trace:
I20260709 13:13:09.299700 28758 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:09.318591 28764 rpcz_store.cc:276] 0709 13:13:07.535960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.536127 (+   167us) service_pool.cc:224] Handling call
0709 13:13:07.536522 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.980425 (+443903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:09.317426 (+1337001us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b5400439ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:09.317818 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1019213184}
I20260709 13:13:09.318209 28081 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:09.322181 28199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.889s	user 0.021s	sys 0.012s
W20260709 13:13:09.322677 28199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.907s	user 0.021s	sys 0.012s
W20260709 13:13:09.322558   671 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:13:09.326239 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:09.327939 28081 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:09.327107 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:09.331346   627 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:13:09.325736 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 2.00175e+06us
I20260709 13:13:09.316308 28761 rpcz_store.cc:276] 0709 13:13:07.462413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.462591 (+   178us) service_pool.cc:224] Handling call
0709 13:13:07.463160 (+   569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:07.954928 (+491768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:09.315126 (+1360198us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b540043ab54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:09.315566 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1096987392}
I20260709 13:13:09.333854 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:09.334666   579 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:09.343842 28231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2129) took 2162 ms (client timeout 2223 ms). Trace:
W20260709 13:13:09.344996 28231 rpcz_store.cc:269] 0709 13:13:07.178389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.315017 (+136628us) service_pool.cc:224] Handling call
0709 13:13:09.340062 (+2025045us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54003bfbd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:09.341006 (+   944us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966552448}
I20260709 13:13:09.355118 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2119) took 1013 ms. Trace:
I20260709 13:13:09.355774 28078 rpcz_store.cc:276] 0709 13:13:08.341405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.341601 (+   196us) service_pool.cc:224] Handling call
0709 13:13:09.354565 (+1012964us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b54003aa7d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:09.354852 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2086042496}
W20260709 13:13:09.355142 28160 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2542) took 2404 ms (client timeout 1943 ms). Trace:
W20260709 13:13:09.358408 28160 rpcz_store.cc:269] 0709 13:13:06.950912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.380141 (+429229us) service_pool.cc:224] Handling call
0709 13:13:09.354623 (+1974482us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b5400317ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:09.354831 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94262144}
I20260709 13:13:09.359658   351 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:09.360316   350 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:09.361733   650 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.056s	user 0.005s	sys 0.000s
W20260709 13:13:09.361753 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.363212 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.364223   350 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.364907 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 882) took 2218 ms. Trace:
I20260709 13:13:09.365382 28153 rpcz_store.cc:276] 0709 13:13:07.146087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.861685 (+1715598us) service_pool.cc:224] Handling call
0709 13:13:08.993896 (+132211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.013822 (+ 19926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:09.364365 (+350543us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54000a32d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:09.364704 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303098624}
I20260709 13:13:09.367362   350 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.367951 28078 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:09.368050 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:09.371140 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:09.371866 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:09.378294 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.389876 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.390587 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.390401 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:09.391705 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:09.392681 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:09.393024 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:09.396075   607 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:09.397878 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:09.399292 28081 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:13:09.400424 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:09.401400 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:09.402957 28078 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:13:09.411012 28078 pending_rounds.cc:85] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 10
I20260709 13:13:09.411273 28078 pending_rounds.cc:107] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:13:09.411494 28078 raft_consensus.cc:2889] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:09.403081 28077 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:09.405198 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:09.406381 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:09.379657 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 58932.2us
I20260709 13:13:09.410634 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:09.415026 28083 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
W20260709 13:13:09.419691 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.421530 28763 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:09.422542 28764 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:09.423293 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:09.423645 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:09.425446 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:09.425608   627 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:09.428153   526 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:09.429032 28760 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:09.432003   541 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:09.433393 28761 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:09.428189 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.429292 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.432514   541 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:09.438117 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.444705   595 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:09.445508 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:09.445603   595 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.446487 28080 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
W20260709 13:13:09.447360 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.448997   579 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:09.449494   579 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:09.449973   579 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:09.450824   387 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:09.452682   387 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:09.456041   541 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.456995   524 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:09.457580   524 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:09.458001   541 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:09.458897   524 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.460142   524 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:09.452140   620 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:09.454169   455 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:13:09.461302   620 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:09.461822   620 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:09.462035   455 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:09.462639   455 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:09.463337 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:09.445318 28761 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:09.463109   620 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.445094 28755 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:09.455149   581 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:09.466193   581 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:09.466584   581 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:09.467820   581 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.469281 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:09.453191   595 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:09.469596 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.470583 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.471259 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.471472 28233 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
W20260709 13:13:09.471678   674 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:13:09.473364   581 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:09.473783 28229 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 13:13:09.481398 28229 pending_rounds.cc:85] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 12
I20260709 13:13:09.481688 28229 pending_rounds.cc:107] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:13:09.481930 28229 raft_consensus.cc:2889] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:09.483153 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.483776 28229 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:09.484627 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:09.485188 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:09.485810 28229 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:09.486038 28233 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:09.486883 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.487864 28229 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:09.489230 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:09.493062   335 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:09.474274 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:09.474709 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.470701 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:09.477439 28084 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:13:09.477787 28230 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:09.478358 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:09.478449 32708 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:09.479419 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 111691us
I20260709 13:13:09.479597 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:09.480759 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.463994   455 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.507272 28232 raft_consensus.cc:2368] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15.
I20260709 13:13:09.501194 28760 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:09.506764 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.444257 28758 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:09.515790 28764 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:09.501204   620 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:09.520972 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 86030.7us
I20260709 13:13:09.521727 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:09.532263 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.530349 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:09.538445 28080 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:09.539319 28229 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:09.540583 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:09.541005 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:09.541601   629 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:09.541868   607 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:09.542253 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:09.542963 28232 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:09.542604   607 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.544801 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:09.547811   312 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:09.548435   312 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:09.549816 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.549747   312 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.552606   607 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:09.552919   670 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.343s	user 0.001s	sys 0.001s
I20260709 13:13:09.557458   455 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:09.560005 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:09.561051 28078 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 8.
W20260709 13:13:09.561901 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 7e272db0632a4262a90eaacd6e6d4378: Illegal state: Replica 0faaace2b36c4bcebf2fa6835235e92b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:09.565026 28235 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:09.566305 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.567117 28235 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:09.567381   335 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:09.568295 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.569413   351 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:09.537927 28234 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:09.541326 28229 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:09.576058 28757 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:09.593364 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:09.594259 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 120163us
I20260709 13:13:09.594416 28234 raft_consensus.cc:2410] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:13:09.573812   351 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:09.597055 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:09.596872   351 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.598934   667 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:09.599429   667 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.599530 28050 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:09.599871   667 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:09.580220 32708 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:09.606586 32708 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.607097 32708 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:09.611470   667 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.615681   667 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.615844   579 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.616796 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:09.617910 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2117) took 1333 ms. Trace:
I20260709 13:13:09.618305 28082 rpcz_store.cc:276] 0709 13:13:08.284532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.284812 (+   280us) service_pool.cc:224] Handling call
0709 13:13:08.309948 (+ 25136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.746570 (+436622us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:09.001287 (+254717us) log.cc:844] Serialized 14 byte log entry
0709 13:13:09.570505 (+569218us) log.cc:844] Serialized 14 byte log entry
0709 13:13:09.581902 (+ 11397us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:09.586430 (+  4528us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:09.590285 (+  3855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:09.603216 (+ 12931us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b18003a0ea0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:13:09.617450 (+ 14234us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b540027bfd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:09.617509 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:09.617695 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15330,"prepare.queue_time_us":391033,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":81,"raft.queue_time_us":69678,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":4295,"spinlock_wait_cycles":1007572096,"thread_start_us":553800,"threads_started":2,"wal-append.queue_time_us":343321}
I20260709 13:13:09.618932   579 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:09.622256   579 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:09.620914 28231 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:09.622991 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:09.623654   615 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:09.623986   595 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:09.624312   615 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:09.633688 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 44541us
W20260709 13:13:09.627967 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet c71e4401306a4e84a111e5443be7652c: Illegal state: Replica 0faaace2b36c4bcebf2fa6835235e92b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:09.641981 28234 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:09.624574 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:09.642846 28235 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:09.585570   581 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.647619 28231 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
W20260709 13:13:09.646708 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.632650 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:09.643111 28233 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:09.633428   526 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:09.652907 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:09.624676   595 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:09.625707 27982 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:09.661731   524 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.661918   618 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:09.662817   618 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.666347   595 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.668854   618 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.634474   350 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:13:09.669768   618 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:09.670564   350 raft_consensus.cc:3055] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:09.671075   350 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:09.655629   526 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.673022 28235 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:13:09.673256   526 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:09.673847   581 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.675933   581 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:09.676997   615 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.656086   335 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:13:09.678548   615 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:09.679042   335 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:09.679486   335 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:09.656338   607 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:09.681731 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.662454   524 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.681363   607 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.633129   313 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:13:09.693761 28229 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:09.694059 28230 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:09.693660 28227 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:13:09.683545 27982 catalog_manager.cc:5697] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 6 to 9, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 9 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
W20260709 13:13:09.697516 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.693401   524 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:09.686743 28757 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
W20260709 13:13:09.699028 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.701251 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.684015   607 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:09.703325 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.662899   452 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.704150 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:09.697175   351 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:09.705467 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:09.711756 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:09.712044 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59557.2us
W20260709 13:13:09.712823 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.712666 28082 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:09.672511   350 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.680632   335 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.734933   350 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:09.735885   335 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:09.737334 28080 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:09.738991 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:09.728844   349 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:09.728904   452 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.741443   313 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:09.742122   313 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:09.743484   620 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.743885 28125 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:09.744328   620 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.746965   620 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.747983   522 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:09.739243 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:09.749083   620 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:13:09.750021 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 125554us
W20260709 13:13:09.749521   680 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 13:13:09.751138 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 175792us
I20260709 13:13:09.748529   522 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.750550   553 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:13:09.754487   629 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:09.756793 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:09.760169 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:09.760649 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:09.743204   452 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:09.755061   629 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:09.743273   313 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.757416   522 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:09.770476 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:09.773437 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:09.764358 28758 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:09.781746 28232 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:09.757382   618 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:13:09.782819   676 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:13:09.810886   618 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.814637   618 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:09.792963   581 raft_consensus.cc:2764] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:13:09.765098   526 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.775369 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:09.778019   452 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.778513   553 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:13:09.780414 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 178690us
I20260709 13:13:09.768020   629 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:09.821094 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.822300 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49023.5us
W20260709 13:13:09.751348   667 consensus_queue.cc:1175] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "70996712c3c942468a6006602de6477f" responder_term: 10 status { last_received { term: 9 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:09.829392 27982 catalog_manager.cc:5697] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 8 to 9, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 9 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:09.832902   524 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.833326 27982 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 5 to 8, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 8 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:09.835634 28153 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:09.837213 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:09.839781 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:09.840772 28156 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:13:09.840823   349 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:09.841441 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:09.842235 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:09.842553 27984 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 276) took 42850 ms (client timeout 15000 ms). Trace:
I20260709 13:13:09.841617   349 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:09.844714 28154 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:13:09.842270 28157 raft_consensus.cc:2368] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14.
I20260709 13:13:09.845338 28154 pending_rounds.cc:85] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 11
I20260709 13:13:09.837231 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:09.846897 28160 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:09.844897   452 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.850514   452 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.851296   452 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:09.854535   526 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.828104   553 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:09.855472   526 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:09.859539 28226 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
W20260709 13:13:09.842994 27984 rpcz_store.cc:269] 0709 13:12:26.987182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:26.987515 (+   333us) service_pool.cc:224] Handling call
0709 13:13:09.836607 (+42849092us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:05.898722 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:05.965622 (+ 66900us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:05.965681 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:13:05.965700 (+    19us) tablet.cc:662] Decoding operations
0709 13:13:05.966420 (+   720us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:05.966504 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:05.966533 (+    29us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:13:05.986623 (+ 20090us) tablet.cc:701] Row locks acquired
0709 13:13:05.986672 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:13:05.986840 (+   168us) write_op.cc:270] Start()
0709 13:13:05.986992 (+   152us) write_op.cc:276] Timestamp: P: 1783602785986776 usec, L: 0
0709 13:13:05.987012 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:13:05.987738 (+   726us) log.cc:844] Serialized 2094 byte log entry
0709 13:13:06.833080 (+845342us) op_driver.cc:464] REPLICATION: finished
0709 13:13:07.203310 (+370230us) write_op.cc:301] APPLY: starting
0709 13:13:07.203413 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:07.733685 (+530272us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:07.761270 (+ 27585us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:07.839180 (+ 77910us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:07.879914 (+ 40734us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:13:07.884721 (+  4807us) write_op.cc:312] APPLY: finished
0709 13:13:07.885014 (+   293us) log.cc:844] Serialized 57 byte log entry
0709 13:13:08.166387 (+281373us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:08.180193 (+ 13806us) write_op.cc:454] Released schema lock
0709 13:13:08.182325 (+  2132us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:08.213698 (+ 31373us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:08.213731 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1121598,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":282226,"apply.run_cpu_time_us":42796,"apply.run_wall_time_us":1018905,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25097,"num_ops":5,"prepare.queue_time_us":490379,"prepare.run_cpu_time_us":22484,"prepare.run_wall_time_us":287329,"raft.queue_time_us":233803,"raft.run_cpu_time_us":55986,"raft.run_wall_time_us":220059,"replication_time_us":845873,"thread_start_us":947361,"threads_started":5,"wal-append.queue_time_us":571527,"wal-append.run_cpu_time_us":22434,"wal-append.run_wall_time_us":2534273}]]}
W20260709 13:13:09.860076 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 234920us
I20260709 13:13:09.845808 28154 pending_rounds.cc:107] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:13:09.862233 28154 raft_consensus.cc:2889] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:09.864354 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:09.833716   524 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.835712 27982 catalog_manager.cc:5697] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 4 to 6, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 6 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:09.863771 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:09.873354 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:09.874332 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.875070 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:09.888289 28080 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:09.889532   349 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.894449 27982 catalog_manager.cc:5697] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:09.897215   553 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.898599   524 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:09.899060   553 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:09.897920 27982 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:09.903535 27982 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 5 to 7, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 7 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:09.905141 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.907439 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 43799.6us
I20260709 13:13:09.904946   607 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:09.915598 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.923935 28758 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:09.881601   351 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:13:09.932300   546 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:09.934466   580 raft_consensus.cc:2764] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:13:09.936040   522 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:09.905257   526 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:09.962080 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:09.962813 28160 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:09.937907 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:09.930992   541 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:13:09.931758   524 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:09.946946 27982 catalog_manager.cc:5697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 7 to 9, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 9 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:09.961637   629 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.921480   607 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:10.003494   607 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:10.004719   522 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:10.004968   667 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:10.005609   522 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:10.005766   667 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.006316   522 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:13:09.946286 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.981372 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.007577   522 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.001962 28153 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:13:10.002791   629 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:10.010249 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:09.945055 32708 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:09.979826   524 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:09.961750   526 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.037693 28084 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:10.050040   526 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:10.053635 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 520476us
I20260709 13:13:10.031472   350 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.035791   546 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:10.064064 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:10.065429 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.064862   581 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.069221   581 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.068984 27982 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 8 to 10, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 10 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:10.069991 32708 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:10.071192 27982 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:10.081696 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:10.082892 28758 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:10.057385   546 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.096004 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:10.097100 28757 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:10.071954   581 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:10.101712 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:10.100473 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 314959us
I20260709 13:13:10.089288   667 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:10.114585 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.117558 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:10.121222   580 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.125309   553 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:10.128293 27982 catalog_manager.cc:5697] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 6 to 7, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 7 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:10.128463 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 305054us
I20260709 13:13:10.063119   350 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.095746   546 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:10.143891 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69737.4us
W20260709 13:13:10.147122 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 217350us
I20260709 13:13:10.149605   667 raft_consensus.cc:2764] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:13:10.170132 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 239551us
I20260709 13:13:10.105345   524 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.180284   620 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:13:10.188539   620 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:10.180344   541 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:13:10.197324 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:10.199496 28728 leader_election.cc:400] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14.
I20260709 13:13:10.199849 28728 leader_election.cc:403] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:13:10.200490   627 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:10.200975   627 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:10.201956   629 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:10.202490   629 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:10.202098   627 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.202888   629 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:10.203792   627 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:10.205703   541 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14.
I20260709 13:13:10.206987   620 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.207759   620 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.210101   541 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:10.218621 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:10.225097   682 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:13:10.225764   541 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:10.226423   541 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:10.229130 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46452.3us
I20260709 13:13:10.229324   627 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:10.238049   524 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:10.230592   350 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:10.239897   695 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:13:10.234130   678 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.232s	user 0.005s	sys 0.000s
I20260709 13:13:10.235286 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:10.234947 27982 catalog_manager.cc:5697] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 6 to 8, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 8 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:10.158156   580 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.239173 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:10.250335   580 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:10.251519 28052 leader_election.cc:400] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15.
I20260709 13:13:10.252002 28052 leader_election.cc:403] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:13:10.252712   350 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:10.253175   350 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:10.245606   618 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:10.248364   335 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.512s	user 0.005s	sys 0.009s
I20260709 13:13:10.254231 27982 catalog_manager.cc:5697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 9 to 10, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 10 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:10.253305 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:10.263532 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 114514us
W20260709 13:13:10.281354 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.256847   350 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:10.258109 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.228070   607 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.256371   541 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:10.224990 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:10.256527   620 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:10.288558 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.264675   580 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.286896 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:10.290774 28082 raft_consensus.cc:2410] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:13:10.295346   607 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:10.295743   697 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
W20260709 13:13:10.297216 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.300513   312 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:10.315814 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:10.316367 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:10.319052 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 65107.2us
W20260709 13:13:10.321485 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:10.325145 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.302949   580 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.332093   580 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
W20260709 13:13:10.334844 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61958.6us
I20260709 13:13:10.334586   700 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:13:10.335745 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:10.336846   684 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:10.341147 27982 catalog_manager.cc:5697] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:10.313160   313 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:10.356149 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:10.317770   312 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:10.360193   710 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:13:10.362815 28199 connection.cc:256] RPC call timeout handler was delayed by -2.1269321330000821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
I20260709 13:13:10.363489   580 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:10.363719 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45360.3us
I20260709 13:13:10.367522 28226 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:10.369746   629 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.400410   629 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:10.371277   700 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:10.401206   700 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:10.381086 28127 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.561s (SENT)
W20260709 13:13:10.379621 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 154262us
W20260709 13:13:10.387564 28199 connection.cc:256] RPC call timeout handler was delayed by -2.005432133000113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
I20260709 13:13:10.388432   667 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:10.390347 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:10.395445 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 75493.2us
I20260709 13:13:10.383399   335 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:10.403600   580 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.404603 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:10.387509   312 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:10.410322   694 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:10.410497   618 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:10.415180   546 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.402201   700 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:10.435847 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45620.3us
I20260709 13:13:10.422605 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:10.431572 28199 connection.cc:256] RPC call timeout handler was delayed by -1.9001321329999428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
W20260709 13:13:10.437139 28199 connection.cc:256] RPC call timeout handler was delayed by -1.6562002640000628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
I20260709 13:13:10.432986   595 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:10.437690 28199 connection.cc:256] RPC call timeout handler was delayed by -1.4843321329999526s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2556s for a response
W20260709 13:13:10.438050 28199 connection.cc:256] RPC call timeout handler was delayed by -1.4105002640001203s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2137s for a response
I20260709 13:13:10.438170 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:10.433955 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:10.437837   595 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:10.438717 28763 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:10.439182   351 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.439677   351 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.441041   692 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:10.441834   351 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:10.441893 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:10.442523   351 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:10.443693   580 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:10.443908   351 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:10.440445   595 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:10.435109 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.429292   667 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:10.438403 28199 connection.cc:256] RPC call timeout handler was delayed by -1.15557118300012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
I20260709 13:13:10.432241   335 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.446724   695 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:10.449972   351 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:10.450477   695 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.452776   695 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:10.453150 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:10.453779 28082 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.449776   667 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:10.451978   524 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:13:10.457101 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 52663.7us
I20260709 13:13:10.459662   524 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:10.460016 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:10.449414   692 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:13:10.465431   700 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:10.461431   351 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.468302 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:10.468940   713 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:10.448407   452 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:10.452325 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:10.434065   694 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:13:10.480501 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 124076us
I20260709 13:13:10.474466 28763 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:10.475409 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:10.482376 28198 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:10.482254   705 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:10.482929   684 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15.
I20260709 13:13:10.484190 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.484884 28079 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
W20260709 13:13:10.485597 28127 leader_election.cc:336] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.509s (SENT)
I20260709 13:13:10.491418 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:10.491855 28083 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:10.491437   695 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.493443 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:10.493784 28127 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.512s (SENT)
I20260709 13:13:10.494234 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:10.489090 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.489852 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.490334 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.490568 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.490837 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.450462   335 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:10.500576 28085 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:10.500655 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:10.505191 28082 raft_consensus.cc:2410] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:10.509027 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:10.510690   705 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:10.510679 28078 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:10.510528 28081 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:10.511785 28080 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
W20260709 13:13:10.514390 28127 leader_election.cc:336] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.664s (SENT)
I20260709 13:13:10.517127 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:10.517971 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.480901   524 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:10.527395 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:10.529009 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 60769.1us
I20260709 13:13:10.532611 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:10.525758 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:10.537145 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:10.538781 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 95632us
I20260709 13:13:10.506215   629 raft_consensus.cc:2764] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:13:10.542127 28232 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:13:10.543337 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:10.561829 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:10.546241 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:10.548082 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:10.516932   581 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:10.571062 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.542665 28232 pending_rounds.cc:85] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 9
I20260709 13:13:10.572448 28232 pending_rounds.cc:107] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:13:10.572911 28232 raft_consensus.cc:2889] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:10.561475 28755 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:10.569142 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:10.575385 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:10.580119 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 129167us
I20260709 13:13:10.530295   524 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:10.542014   684 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:10.587687 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.594857 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:10.595377 28080 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:10.590524 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.586745 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:10.592015 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.541965   580 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:10.596602 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56020.2us
I20260709 13:13:10.596673 28759 raft_consensus.cc:2376] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15. Re-sending same reply.
I20260709 13:13:10.598325 28159 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:10.599658 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57870.8us
I20260709 13:13:10.600263 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.602056 28079 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:10.602345 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:10.575006   695 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:10.606806 28156 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:10.606700 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:10.595858   524 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:10.542488   541 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.594616   706 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.614970 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:10.599118   580 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.619860 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:10.596558   694 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.627969   694 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:10.630831 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.631577 28159 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:13:10.632586 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.633414   723 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:10.634055   723 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:10.633939 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.599078   712 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.200s
I20260709 13:13:10.596719   350 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.612083 32708 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:10.621628 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:10.640798 28154 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:10.594759   629 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:13:10.635313 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.607829   684 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:10.663928 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 57340.5us
W20260709 13:13:10.664575 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 133455us
I20260709 13:13:10.670958 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:10.671695 28156 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:10.671020   684 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.687500 28159 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:10.624665   580 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:10.705919 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:10.634826   714 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:10.705267   351 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.713069 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:10.712984 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.713654 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:10.714048 28228 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:10.714752 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.715183 28152 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:10.643533   697 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:10.718444 28159 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:13:10.725175 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.713122 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:10.731768 28156 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:10.689674   546 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:10.711478   714 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.688548   607 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 13:13:10.749631 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 145548us
I20260709 13:13:10.748605 28085 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:13:10.758227 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:10.760035 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:10.761266 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.763285 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:10.764052 28228 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:10.764443 28226 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:10.765334 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:10.766534 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:10.767288 28226 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:10.769397 28230 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:10.769837 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:10.769455 28232 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:13:10.770836 28232 pending_rounds.cc:85] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 6
I20260709 13:13:10.770836 28233 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:10.771301 28232 pending_rounds.cc:107] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:13:10.771615 28232 raft_consensus.cc:2889] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:10.710481   723 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.773113 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.775245   667 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.776247 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:10.777097 28232 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:10.776587 28079 raft_consensus.cc:2368] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14.
I20260709 13:13:10.722477   720 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.780287   350 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:10.717427   629 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:13:10.784642 28155 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:10.722216 32708 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:10.788335 28077 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:10.784250 28226 raft_consensus.cc:2410] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:13:10.735081   581 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:10.622681   706 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:10.791378 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 52894.2us
I20260709 13:13:10.748934   714 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:10.713002   351 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.803537 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:10.805580 28152 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:13:10.808666 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:10.812494 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.814105 28085 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:10.808298   721 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:10.815551 28158 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:10.820212   721 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:10.752688   697 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.851130 28158 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
W20260709 13:13:10.851287 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65793.6us
W20260709 13:13:10.847240 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 175122us
I20260709 13:13:10.812064   546 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:10.857714 28763 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 9 LEARNER]: Advancing to term 10
I20260709 13:13:10.870782 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:10.825057   721 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:10.877856   718 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.244s	user 0.000s	sys 0.003s
I20260709 13:13:10.882035 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.882805 28158 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:10.883500   607 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:10.885447 32708 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:10.795709   720 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:10.894243   720 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:10.901609 28155 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:10.867416 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:10.909296 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 101828us
I20260709 13:13:10.912151 28081 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:10.912640 28228 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:10.913329 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:10.919516 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:10.920550 28082 raft_consensus.cc:2368] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:10.922070 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.923813 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:10.925450 28085 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:13:10.917496 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.929548 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:10.930126 28763 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 10 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:13:10.930630 28158 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:10.933220 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.941406 28155 raft_consensus.cc:2410] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:13:10.926038 28085 pending_rounds.cc:85] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 8
I20260709 13:13:10.945247 28085 pending_rounds.cc:107] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:13:10.945560 28085 raft_consensus.cc:2889] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:10.946280 28198 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:10.955332 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:10.961822 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:10.965667 28226 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:10.965557 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.952682 28159 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:10.980420 28079 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:10.984333 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 52004.4us
I20260709 13:13:10.941963   541 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:10.959837 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:10.988204 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:10.988976 28227 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:10.989053 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.914647   727 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:13:11.004788 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:10.993924 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:11.005856 28153 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:11.007520 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:11.008659 28159 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:11.013672   629 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:11.014137   351 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:11.021293 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 61534.7us
I20260709 13:13:11.021979 28227 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:11.024242 28160 raft_consensus.cc:2410] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:11.037873 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:11.038846 28226 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:11.051556 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:11.055577 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 151012us
I20260709 13:13:11.057054 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:11.081755 28158 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
W20260709 13:13:11.050393   335 consensus_meta.cc:287] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
I20260709 13:13:11.023468   734 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:11.039817   667 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:11.085646 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:11.067420   687 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:11.092514 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:11.090045 28153 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:11.093436 28158 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:11.090667 28155 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:13:11.079214   349 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:11.104949 28226 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:11.115900 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:11.088979   335 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:11.135844   735 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.136571   735 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.138991   735 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:11.140245   733 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:13:11.141625 28155 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:11.139925   721 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.147125   721 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:11.139304 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:11.165200 28161 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:11.166152   714 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:11.165297   312 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.000s
W20260709 13:13:11.162952   726 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:13:11.175367 28153 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
W20260709 13:13:11.179080 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:11.179239 28127 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.663s (SENT)
W20260709 13:13:11.185982 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 163024us
I20260709 13:13:11.352437   729 raft_consensus.cc:2764] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:13:11.350879   335 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:11.235862 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:11.386518   734 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:11.276168   700 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.809s	user 0.008s	sys 0.008s
W20260709 13:13:11.352582   732 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:13:11.186883   312 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:11.286682   313 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:11.285887   746 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:11.551400 28086 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:11.551784   312 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:11.213435   706 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:11.552661   706 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.553294   706 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.556018   706 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:11.214494 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:11.286486 32708 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:11.353425   723 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:11.236364 28155 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:13:11.240710 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:11.275264   524 consensus_meta.cc:287] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.638s	user 0.012s	sys 0.002s
I20260709 13:13:11.239705 28079 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
W20260709 13:13:11.224798 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55476us
I20260709 13:13:11.348511 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:11.285962   581 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:11.185760   687 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:11.287396 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:11.118743   349 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.282902 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:11.316277 28232 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:11.303103 28158 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:11.289177   697 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:11.410446 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:11.423971 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:11.560194 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:11.573118 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:11.573482 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.573347 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:11.574455 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:11.575219 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:11.387588   695 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.379285   735 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:11.473212 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:11.562207 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2461) took 1016 ms. Trace:
I20260709 13:13:11.576941 28763 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:13:11.474085   737 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.184s	user 0.001s	sys 0.001s
W20260709 13:13:11.476919   750 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:13:11.586699   705 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:11.587204   705 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:11.478137   758 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:13:11.587579   705 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:11.481174 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 424303us
I20260709 13:13:11.403038   350 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 13:13:11.503433   580 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.002s
I20260709 13:13:11.590122   580 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.591897   751 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:11.472404   741 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:11.354406   727 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:11.535197 28127 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:11.542842 28761 raft_consensus.cc:2410] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:13:11.578006 28125 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:11.577219 28157 rpcz_store.cc:276] 0709 13:13:10.545941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.546116 (+   175us) service_pool.cc:224] Handling call
0709 13:13:11.348391 (+802275us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552f8c 00007f59cb552ef6 0000000000375939 00007f59d35c61e0 00007f59d368f0c4 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:13:11.562126 (+213735us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7900288}
I20260709 13:13:11.571318 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:11.581104 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:11.583828 28755 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11. Ops: []
W20260709 13:13:11.569728 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 286913us
I20260709 13:13:11.579872   526 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:11.584939 28760 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:11.569561 32708 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:11.585745 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:11.586421 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:11.590126 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:11.600553 28233 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:13:11.602404 28229 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:13:11.602672 28235 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:11.604995 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:11.605808 28228 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:11.562835   753 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.558881   723 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:11.568923   349 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:11.620707 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:11.578311   762 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:11.594620   751 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:11.627362 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65994.5us
I20260709 13:13:11.596941   727 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:11.632349 28050 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:11.556697   706 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.629693 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:11.602095 32708 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:11.628327 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:11.571208   731 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:11.642748 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:11.623867   700 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:11.643708   755 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
W20260709 13:13:11.664294 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 74233.6us
I20260709 13:13:11.665939 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:11.646356 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.560288   524 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:11.663177   700 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.667095   727 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:11.645596 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 466346us
I20260709 13:13:11.615947   695 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.672243 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2141) took 1182 ms. Trace:
I20260709 13:13:11.664770 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:11.592607   735 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.548168   734 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.630389   312 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:11.563598   687 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.699250 28228 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:11.669241   716 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:11.621740   526 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:11.711619 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46966.4us
I20260709 13:13:11.711191 28086 rpcz_store.cc:276] 0709 13:13:10.489315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.492252 (+  2937us) service_pool.cc:224] Handling call
0709 13:13:10.492547 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.551879 (+1059332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.671943 (+120064us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400267fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:11.672171 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2434406656}
I20260709 13:13:11.642270   751 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:11.674185 32708 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:11.672947   313 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:13:11.772022 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 106143us
W20260709 13:13:11.715194 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 82942.5us
I20260709 13:13:11.674505   729 raft_consensus.cc:2764] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:13:11.726423 28154 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
W20260709 13:13:11.728029 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.704241   350 raft_consensus.cc:2764] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:13:11.757217 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 179265us
I20260709 13:13:11.782794 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:11.710040   581 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.675375   746 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:13:11.629031   753 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:11.673909   706 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.764330   694 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:11.721628   684 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:13:11.780146 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:11.797201 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.681595   731 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.735634   727 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:11.797739 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 168946us
I20260709 13:13:11.765609   736 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.772100   349 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:11.784435   313 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:11.809638 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:11.806160 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 71954.5us
I20260709 13:13:11.820803 28755 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:11.803001   749 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:11.818256   736 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:11.827579 28761 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:11.825013 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:11.805030   695 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:11.844333   777 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:11.844918   777 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:11.845350   777 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:11.845212   313 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:11.824301   716 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:11.848925   761 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:13:11.849298   769 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:13:11.849660   716 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:11.853725   771 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:13:11.853585   778 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:11.824030   749 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.855952   749 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:11.857366   752 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:13:11.817353   766 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.792181   581 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:11.817564   684 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:11.823350   312 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:11.873987 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 64003.9us
I20260709 13:13:11.884527   452 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.817170   349 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:11.866801 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:11.861601   700 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:11.789022   723 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:11.902900 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 120217us
W20260709 13:13:11.897015 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 114732us
W20260709 13:13:11.956323 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44652.9us
W20260709 13:13:11.957878 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:11.953830 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 87154.5us
W20260709 13:13:11.970553 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 268232us
I20260709 13:13:11.967998 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:11.975286 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:12.005105 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:12.004355 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:12.007519 28761 raft_consensus.cc:2410] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:13:11.999678 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:11.979799 28226 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:12.008812 28081 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
W20260709 13:13:11.969959 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65837.7us
I20260709 13:13:11.903522   349 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:12.011322 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:11.970211   746 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:12.017051 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:12.018117 28079 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:12.021136 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:12.021855 28153 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:11.870085   735 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.016930 28085 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:11.964498   526 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:11.979077   350 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:13:12.033766 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:12.034476 28159 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:12.028007 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 127943us
I20260709 13:13:12.028861 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:12.030689 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:12.036908 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:12.037048 28229 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:11.970319   695 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.045492 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:12.046314 28160 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:13:12.052635 28127 leader_election.cc:336] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.794s (SENT)
I20260709 13:13:12.053330 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:12.053859 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:12.058099 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.965098   700 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
W20260709 13:13:12.065129 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47850.5us
I20260709 13:13:11.970216   694 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:13:11.953686   452 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.038964   706 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:11.971029   755 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:13:12.004711   729 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:13:12.090566 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:12.109567 28085 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:12.056440   756 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:12.098767 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:12.104954   776 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:13:12.068117   788 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.902323   766 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.127063 28160 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:12.088974 28153 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:12.099886 28079 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:12.063746   524 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.063355   697 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:12.116353 28229 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:12.051653   695 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:12.159528 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46346.8us
I20260709 13:13:12.059548   705 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:12.161448 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 119092us
W20260709 13:13:12.164072 28127 leader_election.cc:336] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.871s (SENT)
I20260709 13:13:12.164603 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:12.165591 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:12.166512 28757 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:12.161758   724 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.109298   753 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.174521 28761 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:12.182945 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.217190 28757 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:13:12.218290 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:12.220095 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.070258   684 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.139528   335 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.082s
W20260709 13:13:12.233824 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69780.1us
I20260709 13:13:12.172317   731 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:12.174949   735 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.238780 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:12.232681 28157 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 13:13:12.250950 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 225403us
I20260709 13:13:12.121549   756 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:12.230228   524 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:13:12.259610 28127 leader_election.cc:336] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.921s (SENT)
I20260709 13:13:12.253520   746 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:13:12.273456 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:12.278087 28050 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:12.241111   697 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:12.294893 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 119693us
I20260709 13:13:12.295049 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:12.296068 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:12.138476   788 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.301558 28764 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:12.273893   312 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.305702 28156 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:13:12.274179   581 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:12.275707   524 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.309028 28756 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:12.277894   692 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:12.282480   729 raft_consensus.cc:2764] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:13:12.305156 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:12.289430   712 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:12.239739   736 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:12.222627   724 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.270583   735 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:12.256433   716 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
W20260709 13:13:12.336285 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 85556.6us
I20260709 13:13:12.315847   524 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:12.331290   697 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:12.343237 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:12.344928   782 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:12.346340   800 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:13:12.349112   794 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.351616   695 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:12.353225   783 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:13:12.355306 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:13:12.360591 28127 leader_election.cc:336] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.921s (SENT)
W20260709 13:13:12.348655   791 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:13:12.407100   749 consensus_meta.cc:287] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 13:13:12.312888   312 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:12.343251   793 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:13:12.325183   692 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.401312   710 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.342820   786 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:12.408217 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53047.4us
I20260709 13:13:12.364210   804 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:12.353868   798 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:12.365640   800 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:12.340116   801 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.087s
W20260709 13:13:12.428422 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 144033us
I20260709 13:13:12.367193   695 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:13:12.340494   712 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:12.439742 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 161716us
I20260709 13:13:12.356125   335 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 13:13:12.466993 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.415639   452 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:12.477664 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 137519us
W20260709 13:13:12.478643 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44272.6us
I20260709 13:13:12.490072 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:12.523006 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:12.528534 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:12.530913   724 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:12.532068   729 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:12.532617   729 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.533026   729 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:12.537324 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:12.538239   581 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:12.537098 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:12.538686   581 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:12.539129   581 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:12.539259 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 58558.8us
I20260709 13:13:12.540165 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:12.539139   351 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:12.541551   351 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:12.542057   351 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:12.543267 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:12.544288 28229 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:12.545459 28230 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:12.544270 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:12.543350   351 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.565186   724 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:12.567531 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:12.569137 28125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.005s	sys 0.054s
W20260709 13:13:12.569645 28125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.005s	sys 0.054s
W20260709 13:13:12.570286 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 516527us
W20260709 13:13:12.574784 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57545us
W20260709 13:13:12.576828 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53961.8us
I20260709 13:13:12.578147 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:12.474695   721 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:12.474267   800 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:12.366519   794 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:12.592267 28127 leader_election.cc:336] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.515s (SENT)
I20260709 13:13:12.489490   804 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:12.420884   749 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.596004 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:12.605772 28159 consensus_meta.cc:287] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.002s
I20260709 13:13:12.606417 28159 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
W20260709 13:13:12.585615   751 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.801s	user 0.012s	sys 0.000s
I20260709 13:13:12.492923   766 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:12.608994 28228 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.002s	sys 0.021s
W20260709 13:13:12.660697 28228 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.002s	sys 0.021s
I20260709 13:13:12.595530   798 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.663416   798 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.666507   798 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.596333   452 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.432622   710 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:12.581602   746 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:12.615509   724 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:12.676756   618 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:12.692929 28083 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.036s	sys 0.046s
W20260709 13:13:12.710026 28083 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.037s	sys 0.047s
W20260709 13:13:12.712034 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:12.704698 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 112532us
W20260709 13:13:12.714264 28127 leader_election.cc:336] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.091s (SENT)
I20260709 13:13:12.603986   721 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:12.672256   786 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:12.615586   797 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:12.705363 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:12.727939 28760 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:12.672276   792 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:12.729956 28229 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:12.698606   746 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:12.631726   751 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:12.721553   790 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.313s	user 0.002s	sys 0.000s
W20260709 13:13:12.744630 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 152987us
I20260709 13:13:12.749316 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:12.680264   452 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.708483   742 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:12.750521 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:12.726440   788 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.720918   335 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:12.720490   813 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:12.742461   746 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:12.760856 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46630.7us
W20260709 13:13:12.790187 28127 leader_election.cc:336] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.134s (SENT)
W20260709 13:13:12.791095 28127 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.223s (SENT)
I20260709 13:13:12.791448 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:12.793401   786 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:12.793730   749 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:12.794046   802 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 13:13:12.794171   786 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:12.794775   786 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.795928   721 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.795945   700 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:12.797123   700 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:12.798338 28232 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:13:12.798026   700 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:12.798499   786 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.798714 28232 pending_rounds.cc:85] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 11
I20260709 13:13:12.799014 28232 pending_rounds.cc:107] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:13:12.799319 28232 raft_consensus.cc:2889] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:12.755151   805 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:12.801338   805 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:12.794782   312 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.807442 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1105) took 1216 ms. Trace:
I20260709 13:13:12.809195 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:12.731083   724 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:12.810079 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:12.811055   825 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.003s
I20260709 13:13:12.811015 28153 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:13:12.811919 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:12.812311 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:12.812618 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:12.814110   714 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.772509   729 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.816437   729 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:12.744508   816 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.765122   753 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:12.767640   524 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:12.768697   806 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
W20260709 13:13:12.770656   777 consensus_meta.cc:287] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.925s	user 0.012s	sys 0.000s
I20260709 13:13:12.914629   777 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:12.776855   712 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.750875   581 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:12.771430 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.925344   581 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:12.771771   794 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:12.772497   818 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:13:12.772810   808 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:13:12.759156   804 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.774024   710 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:12.736289   800 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:12.754529   760 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:13:12.776202   452 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.722579   618 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:12.759824 28226 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:12.934188 28158 raft_consensus.cc:2368] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 11. Current term is 12.
I20260709 13:13:12.776799   735 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:12.737632   792 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:12.779062 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:12.782407   751 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:12.783224 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:13.003283 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 220121us
W20260709 13:13:12.785027 32708 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.998s	user 0.012s	sys 0.000s
I20260709 13:13:12.787603   692 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:12.789354   349 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.756s	user 0.012s	sys 0.000s
I20260709 13:13:12.808032 28763 rpcz_store.cc:276] 0709 13:13:11.590610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.590764 (+   154us) service_pool.cc:224] Handling call
0709 13:13:11.591083 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:11.591411 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:11.591439 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:11.591985 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 13:13:11.877862 (+285877us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:11.877909 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.877987 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:12.807157 (+929170us) raft_consensus.cc:1685] finished
0709 13:13:12.807187 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:12.807363 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3285,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":16437,"thread_start_us":285417,"threads_started":1,"wal-append.queue_time_us":439487}
I20260709 13:13:12.821985 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:13:12.757521 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 179515us
I20260709 13:13:13.017100 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:13:12.824600 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:13.021790 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.022811 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:12.865628 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:12.868510 28127 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.679s (SENT)
I20260709 13:13:12.870008 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:12.912899   816 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:12.937539 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:13.023686 28228 raft_consensus.cc:2410] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:13:13.025008 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 87586.1us
I20260709 13:13:13.025285 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.025774 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:13.025918 28227 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:13.028292 28153 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:12.873996   753 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:13.030035 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:13.040684 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.043565   797 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:13.044045   797 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.044399   797 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:13.045377   762 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:13.045815   762 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:13.041767 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:12.933636   524 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.030423 28229 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:13.047485 28228 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:13.014047   794 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:13.002276   452 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.032860 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:13.058043 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.028434   312 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:12.811086   724 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:13.060793 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 192309us
W20260709 13:13:13.062433 28127 leader_election.cc:336] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.731s (SENT)
I20260709 13:13:13.040880   687 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:13.048959   788 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:13.073242 28086 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:13.074560 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.077651 28229 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:13.048872   786 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.046862   792 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:13.089208 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.094146 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2603) took 1118 ms. Trace:
I20260709 13:13:13.028607   695 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:13:13.056322   794 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.094779 28233 rpcz_store.cc:276] 0709 13:13:11.976238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.976392 (+   154us) service_pool.cc:224] Handling call
0709 13:13:12.016446 (+ 40054us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:12.033687 (+ 17241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.033716 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:12.034167 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:13:12.272146 (+237979us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:12.272208 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:12.272315 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.093863 (+821548us) raft_consensus.cc:1685] finished
0709 13:13:13.093889 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.094061 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16851,"raft.queue_time_us":28583,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":12216,"thread_start_us":237564,"threads_started":1,"wal-append.queue_time_us":383565}
I20260709 13:13:13.029984   349 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.107933 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.109201 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:13.109674   692 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:13.110013 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.113420 28077 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:13.122335 28080 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:13.114280 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:13.115042   824 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.395s	user 0.000s	sys 0.008s
I20260709 13:13:13.116120   749 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:13:13.105234 28127 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.867s (SENT)
I20260709 13:13:13.128212 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:13.117285 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.024602 32708 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.065678   312 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:13.072320   687 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:13.127326 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.130311 28078 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:13.131810 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:13.137133 28077 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:13.133013 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.124826 28084 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:13.029090   827 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:13:13.138053 28227 raft_consensus.cc:2368] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:13:13.048094   746 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.097934   751 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.120071   794 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:13.142489 28234 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:13.065287   452 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.143548 28229 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:13.119021   788 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:13.149701 28226 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:13.129016   777 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:13.151034 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 394110us
I20260709 13:13:13.214165 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.215447 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.216018 28228 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:13.137073   312 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:13.218192   705 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.219427   350 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:13.219872   350 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:13.139954   687 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:13.221458   828 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:13:13.220806   350 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.222720   350 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:13.227905   734 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:13.046849   762 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.179385 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.179754 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.116461   797 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.117887   786 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.185925   788 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.153067   816 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:13.191068 28077 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:13.118499   753 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.201310 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.249696 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.208405   349 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:13.211048 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.211544 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2463) took 2607 ms. Trace:
I20260709 13:13:13.211073   803 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:13.212366 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.213445 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.213560 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.233192 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.273150 28232 raft_consensus.cc:2368] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13.
I20260709 13:13:13.252399 28154 rpcz_store.cc:276] 0709 13:13:10.603932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.700840 (+ 96908us) service_pool.cc:224] Handling call
0709 13:13:10.701326 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.779033 (+1077707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.211267 (+1432234us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000a37d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:13.211454 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2356851072}
W20260709 13:13:13.269340 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 130926us
I20260709 13:13:13.263636 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.276643 28077 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:13.270184 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.265244 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.278412 28085 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:13.265903 28231 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:13:13.262722 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.150008   751 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.275812 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:13.282739 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 249787us
I20260709 13:13:13.293411 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.310716 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.303385 28077 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:13.295508 28227 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:13.313619 28233 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:13.306634 28228 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
W20260709 13:13:13.295783 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 202859us
I20260709 13:13:13.296232 28084 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
W20260709 13:13:13.308902 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 191408us
W20260709 13:13:13.316267 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.317498 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:13.318361 28153 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:13.249953   705 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:13:13.329308 28085 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:13.332248 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2150) took 1331 ms. Trace:
I20260709 13:13:13.334921 28228 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:13.335654 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.338233   749 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.295113   753 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:13.298189   524 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:13.349983 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.330011   782 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:13.347340 28153 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:13.349673 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.344492 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:13.333766 28083 rpcz_store.cc:276] 0709 13:13:12.001317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.001473 (+   156us) service_pool.cc:224] Handling call
0709 13:13:12.023746 (+ 22273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.023945 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:12.720710 (+696765us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.331924 (+611214us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.331976 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:13.332015 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.332084 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.332193 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30958,"prepare.queue_time_us":404019,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":4699,"raft.queue_time_us":52780,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":4040,"thread_start_us":1247354,"threads_started":3,"wal-append.queue_time_us":505624}
I20260709 13:13:13.352690 28157 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:13.359431 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.360319 28226 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:13.333627   781 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:13.363540 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.364321 28159 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:13.366111 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.346457   705 consensus_queue.cc:1243] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:13:13.349843   804 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.375630 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.354321   524 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.371796 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:13.340981   749 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.382742 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:13.383054 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.383622 28159 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:13.384338 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:13.385250 28077 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:13.383911 28226 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:13.365082   580 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.398240 28084 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:13:13.398897 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.399956 28077 raft_consensus.cc:2368] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 11. Current term is 12.
I20260709 13:13:13.381450   524 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:13.394829 32708 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.409047   746 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.414418 28159 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:13.381115   804 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.371872   781 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:13.441226 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.423766 28159 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:13.453532 28159 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:13.449514 28762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 850) took 1167 ms. Trace:
I20260709 13:13:13.457450 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.424758 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.458408 28226 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:13.458456 28077 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
W20260709 13:13:13.422153   833 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:13:13.462572 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:13.465167 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:13.474316 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.465814 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.474720 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.461232 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.475899 28226 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:13:13.475999 28077 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:13.466272   797 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:13.466566 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:13.443985   836 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:13:13.472818   831 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.243s	user 0.001s	sys 0.001s
I20260709 13:13:13.473374 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:13.474959 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:13.432336   839 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:13:13.439544   751 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.445070   618 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.455344 28762 rpcz_store.cc:276] 0709 13:13:12.282284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.283487 (+  1203us) service_pool.cc:224] Handling call
0709 13:13:12.305101 (+ 21614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.586563 (+281462us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:13:12.604865 (+ 18302us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.062236 (+457371us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.086177 (+ 23941us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.088206 (+  2029us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.088278 (+    72us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:13:13.088348 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.447307 (+358959us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b1800464740. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a522 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:13:13.448627 (+  1320us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b1800464740. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59d254b58f 00007f59cb58a571 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd
0709 13:13:13.449240 (+   613us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54007e0254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:13.449296 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.449443 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":362331,"prepare.queue_time_us":3011,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":2905,"raft.queue_time_us":29381,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":27894,"spinlock_wait_cycles":791558656,"thread_start_us":431321,"threads_started":1,"wal-append.queue_time_us":576047}
I20260709 13:13:13.477332   700 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.478750   848 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:13.494593   700 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.495030   848 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.492611 28159 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.479476 28231 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:13.445981   804 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:13.422000   835 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:13:13.486615 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.491318 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:13.503571 28159 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:13.473392   734 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.478191 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.497308 28229 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:13.497373 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:13.442327   749 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.499703 28232 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:13.505359 28728 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:13.512576 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.472429   687 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.450119   312 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.518960   848 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:13.520509   787 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.487s	user 0.018s	sys 0.018s
I20260709 13:13:13.521113 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.523232 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.524408 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.499989   722 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:13.524945 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.524993 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:13.526885 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:13.532527 28233 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:13.534394 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:13.534649 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.529874   829 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:13.534097 28050 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:13.534580 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.532863 28226 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:13.537204 28230 raft_consensus.cc:2368] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13.
W20260709 13:13:13.536015 28127 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.197s (SENT)
I20260709 13:13:13.542282 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:13.542327 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:13.538236 28050 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:13.535928 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.544823 28231 raft_consensus.cc:2376] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13. Re-sending same reply.
I20260709 13:13:13.517077   735 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.548388 28082 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:13.550124 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:13.553444 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.554291 28159 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:13:13.554808   747 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:13.555183 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:13.555310   747 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.557384   747 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:13.552235 28762 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:13.564440 28154 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:13.530165   804 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:13.568234 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:13.533419   700 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:13.536319   721 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:13.515177   734 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.535676   829 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.550205   350 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:13.569595 28159 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:13.516132   777 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:13:13.578603 28755 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:13.570662 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.579336 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:13.579949 28231 raft_consensus.cc:2410] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:13.546277   618 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:13.538499   749 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:13.526439   811 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:13:13.535701   722 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 11 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:13.571818   848 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:13.584296   722 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:13.585752   850 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:13:13.587358 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 907) took 1567 ms. Trace:
I20260709 13:13:13.587759   777 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:13.587359   350 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:13.589144   687 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.589296   350 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.590329 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 904) took 1571 ms. Trace:
I20260709 13:13:13.590701 28155 rpcz_store.cc:276] 0709 13:13:12.019345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.019537 (+   192us) service_pool.cc:224] Handling call
0709 13:13:12.041509 (+ 21972us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:12.247938 (+206429us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:12.473573 (+225635us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.069976 (+596403us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:13.097539 (+ 27563us) log.cc:844] Serialized 26 byte log entry
0709 13:13:13.097738 (+   199us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:13.097786 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.097878 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.590125 (+492247us) raft_consensus.cc:1685] finished
0709 13:13:13.590146 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.590274 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6593,"prepare.queue_time_us":6964,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":64725,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":43814,"thread_start_us":851342,"threads_started":3,"wal-append.queue_time_us":533360}
I20260709 13:13:13.591816 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.588186   777 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:13.593552   800 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:13.593997   777 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.595432   844 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:13.595894   844 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.596248   844 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:13.596819 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:13.595433   777 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:13.598088   749 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:13.599018 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:13.599109   749 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.599787 28084 raft_consensus.cc:2376] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 14. Re-sending same reply.
I20260709 13:13:13.581632   618 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.601125   618 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:13.603325   312 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.597477 28759 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:13.608151   848 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:13.609267   848 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:13.609654   848 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:13.610572   848 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.611433 28756 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:13.611816   848 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:13.566465   735 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.572791   721 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.619329 28160 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 9.
I20260709 13:13:13.621399   721 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:13.623371 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.613009 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:13.618366   805 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.601004 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.626677 28230 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:13.628104   781 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.580200   829 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:13.630795   734 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:13.617715   452 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:13.633519   840 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.258s	user 0.000s	sys 0.004s
I20260709 13:13:13.637189 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:13.612582 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:13.637853 28755 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:13:13.638257 28757 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:13.639171 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:13.640513 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:13.641356 28125 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:13.641901 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.642657 28231 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:13.642953 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.613133 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.644098 28226 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:13:13.613678 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.633661 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2609) took 1355 ms. Trace:
I20260709 13:13:13.645525 28234 rpcz_store.cc:276] 0709 13:13:12.278441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.278622 (+   181us) service_pool.cc:224] Handling call
0709 13:13:12.285355 (+  6733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.145084 (+859729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.633419 (+488335us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b54003566d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:13.633601 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970718464}
I20260709 13:13:13.647215 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.624526 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.587796 28156 rpcz_store.cc:276] 0709 13:13:12.020399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.020555 (+   156us) service_pool.cc:224] Handling call
0709 13:13:12.033359 (+ 12804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.306691 (+273332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.587059 (+1280368us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b540036d154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:13.587258 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607014144}
I20260709 13:13:13.652026 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:13.653043 28230 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:13.648090 28155 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:13.652973 28757 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:13.641465   762 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:13.654724   848 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:13:13.655707 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.660058 28084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:13.661096 28084 raft_consensus.cc:606] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:13.663741 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.663962 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:13.667065 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.670248 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:13.671257 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:13.671340 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:13.672693 28156 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 16: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:13.677876 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:13.678745 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.582641   804 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:13.680521 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:13.682837 28757 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:13.684612 28078 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:13.692044 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.682109   312 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.699434   312 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.679965   848 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:13.688557 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:13.704269 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.705632 28763 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:13.702700 28152 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:13.677786   687 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:13.579703   794 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.727972 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.729710 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:13.711283   819 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.030s	user 0.008s	sys 0.004s
I20260709 13:13:13.682636   800 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:13.677567   452 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:13.701776   848 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.712500   312 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.680840   738 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:13:13.731458   687 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.729554   804 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:13.774236   756 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.777094 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:13.778409 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:13.779255 28759 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:13.778568 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.780478 28152 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:13.781354 28083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:13.782076 28083 raft_consensus.cc:606] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:13.782634 28083 raft_consensus.cc:614] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:13.782080   829 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.744617   452 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.784353   452 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:13.781424 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.785391 28152 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:13.786285 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.787770 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:13.788605 28152 raft_consensus.cc:2410] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:13:13.783952 28757 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:13.784577   829 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:13.790920 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:13.791913 28760 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 9: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:13.794371 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.795244 28233 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:13.799696 28755 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:13.803642 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.761188   800 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:13.767067   687 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:13.741492   524 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.803079   786 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:13.823243 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.824123 28233 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:13.809396   847 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.831099 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2608) took 1554 ms. Trace:
I20260709 13:13:13.852520 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.836563 28235 rpcz_store.cc:276] 0709 13:13:12.277001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.277167 (+   166us) service_pool.cc:224] Handling call
0709 13:13:12.277530 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:12.285683 (+  8153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.285715 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:12.286525 (+   810us) log.cc:844] Serialized 52 byte log entry
0709 13:13:12.421792 (+135267us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:12.421857 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:12.421973 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.662178 (+1240205us) spinlock_profiling.cc:243] Waited 8.25 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:13.825640 (+163462us) raft_consensus.cc:1685] finished
0709 13:13:13.825691 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.831015 (+  5324us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8829,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":21462,"spinlock_wait_cycles":18973568,"thread_start_us":94830,"threads_started":1,"wal-append.queue_time_us":174504}
I20260709 13:13:13.857453 28083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8381961442846ab97965dc11b100d75"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
W20260709 13:13:13.858196 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.858353 28083 raft_consensus.cc:606] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:13.859051 28083 raft_consensus.cc:614] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:13.862741 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:13.863652 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.875267 28083 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:13.885637 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.886216 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.892436 28756 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:13.893996 28085 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:13.896396 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.839150   847 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.901659 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.905141 28079 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:13.910252 28083 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:13.914537 28083 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:13.915028 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.915870 28085 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
W20260709 13:13:13.919345 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:13.922286 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 143215us
I20260709 13:13:13.924337 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:13.927086 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 335108us
I20260709 13:13:13.937532 28083 raft_consensus.cc:2368] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16.
W20260709 13:13:13.919010   861 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:13:13.909036   735 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:13.952315 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.865336   804 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.959226 28085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:13.959961 28085 raft_consensus.cc:606] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:13.960318 28085 raft_consensus.cc:614] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:13.962088 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:13.962808 28233 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:13.877003   800 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:13.979423 28233 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:13.980161 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.981382 28234 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:13.947084   794 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:13.983300 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:13.985898 28083 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:13.988183 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:13.948598   692 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:13:13.995384   618 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:13.996296 28155 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:13.996364 28231 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:13:13.995054 28230 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 6. (term mismatch)
I20260709 13:13:14.007987 28230 pending_rounds.cc:85] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 5
W20260709 13:13:14.008229   857 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:13:13.997812 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:13.998621 28233 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:13.972620   781 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:13:13.999838 28227 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:14.009128   452 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:14.000924 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:14.010179   860 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:13:14.010474 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:14.002707 28078 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:14.002612 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:14.003515 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:14.011744 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:14.011991 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:14.012143 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:14.012919 28157 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:14.013736 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:14.010468 28229 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:14.015640 28083 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:14.013736 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:14.014232   452 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:14.021956   847 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:14.022625 28079 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:13.946035   312 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:13:14.025949   692 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:14.026377   692 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:14.026782 28198 connection.cc:256] RPC call timeout handler was delayed by -1.1601542809999046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
W20260709 13:13:14.027261 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:14.027966 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:13.998338   858 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:13:14.036805 28229 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:14.012404 28081 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:14.009335 28756 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:14.027369   692 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:14.029542 28760 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:14.027384 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:14.028702 28198 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 888e64754b8d47aebb2c7c3aba7b0ee2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:14.033062 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2148) took 2034 ms. Trace:
I20260709 13:13:14.031064 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:14.008320 28230 pending_rounds.cc:107] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:13:14.037820 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:14.031175 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:14.041833 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:14.042691 28155 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:14.042912 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:14.041846 28230 raft_consensus.cc:2889] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:14.041406 28086 rpcz_store.cc:276] 0709 13:13:11.998784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.999906 (+  1122us) service_pool.cc:224] Handling call
0709 13:13:12.000240 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.089106 (+1088866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.032769 (+943663us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540027bfd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:14.032984 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2467088128}
W20260709 13:13:14.035339   853 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:13:14.031670   811 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:14.056913 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:14.057754   781 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:14.058434 32708 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:13:14.046802   861 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:13:14.069881 28756 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:14.035246   747 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:14.077167 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.082582 28077 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:14.050930   524 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:14.032032   872 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:13:14.053299   778 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:14.091630 28081 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:14.074246   794 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:14.071000   781 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:14.115669 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.072204   692 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:14.045742   455 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:13:14.077145   811 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:14.119838   778 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:14.087363   312 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:13:14.142253 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:14.143205 28228 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:14.123528   781 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:14.131780   721 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.001s
I20260709 13:13:14.129724   794 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:14.150098   865 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:13:14.149809   794 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:14.150477   862 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:13:14.092137   747 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:14.168653   866 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 13:13:14.171957   874 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:13:14.190377   885 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:13:14.218516 28199 connection.cc:256] RPC call timeout handler was delayed by -2.6085918759999913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
W20260709 13:13:14.234015 28199 connection.cc:256] RPC call timeout handler was delayed by -2.6022918759999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260709 13:13:14.234552 28199 connection.cc:256] RPC call timeout handler was delayed by -2.5482918759998938s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
W20260709 13:13:14.234965 28199 connection.cc:256] RPC call timeout handler was delayed by -2.5383918759998778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
W20260709 13:13:14.219148 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 176463us
I20260709 13:13:14.238579   778 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:14.237588 28198 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:14.146667   618 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:14.246273 28199 connection.cc:256] RPC call timeout handler was delayed by -2.3304918759999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 13:13:14.229338   844 consensus_meta.cc:287] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.000s
W20260709 13:13:14.234073   873 log.cc:927] Time spent T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.008s
I20260709 13:13:14.250231 28153 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:14.255928 28153 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:14.261236 28228 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:14.267134 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:14.305754 28228 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:13:14.248143 28199 connection.cc:256] RPC call timeout handler was delayed by -2.2989918759999455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 13:13:14.311620 28199 connection.cc:256] RPC call timeout handler was delayed by -2.2890918759999295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19030000000000002s for a response
I20260709 13:13:14.201156   721 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:14.252972   778 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:14.314855 28199 connection.cc:256] RPC call timeout handler was delayed by -2.2251918760000535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
W20260709 13:13:14.319664 28199 connection.cc:256] RPC call timeout handler was delayed by -2.0721918760000335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
I20260709 13:13:14.229655   687 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:14.304445   877 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:13:14.321339 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0884918760000346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32370000000000004s for a response
I20260709 13:13:14.308849   335 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:13:14.332552 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0047918759998993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.333s for a response
I20260709 13:13:14.336647 28759 raft_consensus.cc:2368] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14.
W20260709 13:13:14.340962 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 103464us
I20260709 13:13:14.354357 28157 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:14.272444   844 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:14.246575   811 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:14.325529   887 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.318924 32708 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:13:14.374351   898 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:14.375542   884 raft_consensus.cc:2764] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:13:14.377153 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:14.378242 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:14.364362   335 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:14.447374 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:14.448134 28226 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:14.449126 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:14.449882 28226 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:14.450892 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:14.457101 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 78971.9us
I20260709 13:13:14.451614 28226 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:14.382447   898 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:14.448572   878 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:13:14.457990 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:14.460749 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:14.458674   900 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.002s
I20260709 13:13:14.457919   827 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.379300   887 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:14.565075 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:14.467834   834 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.444278   350 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:14.552145 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:14.557385   782 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:14.535761   819 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:13:14.599089 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:14.608901 28234 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:14.572762   782 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:14.615298 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.615551   705 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.616065   800 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.616287 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1114) took 1086 ms. Trace:
I20260709 13:13:14.619354   896 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:14.611219 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:14.631340 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 703180us
W20260709 13:13:14.630712 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.637115 28226 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:14.637336 28728 leader_election.cc:400] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 11. Current term is 12.
I20260709 13:13:14.637660 28728 leader_election.cc:403] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:14.631524 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2610) took 1259 ms. Trace:
I20260709 13:13:14.638108 28155 rpcz_store.cc:276] 0709 13:13:13.371915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.053361 (+681446us) service_pool.cc:224] Handling call
0709 13:13:14.078386 (+ 25025us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:14.092685 (+ 14299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:14.092722 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:14.122527 (+ 29805us) log.cc:844] Serialized 26 byte log entry
0709 13:13:14.309306 (+186779us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:14.324242 (+ 14936us) log.cc:844] Serialized 14 byte log entry
0709 13:13:14.324503 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.324606 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:14.631295 (+306689us) raft_consensus.cc:1685] finished
0709 13:13:14.631314 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:14.631467 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7427,"prepare.queue_time_us":14742,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":521,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":32995,"spinlock_wait_cycles":6493440,"thread_start_us":169527,"threads_started":1,"wal-append.queue_time_us":242672}
I20260709 13:13:14.636003 28763 rpcz_store.cc:276] 0709 13:13:13.530031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.531027 (+   996us) service_pool.cc:224] Handling call
0709 13:13:13.547385 (+ 16358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.706154 (+158769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.615946 (+909792us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54004371d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:14.616202 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363254656}
I20260709 13:13:14.635959 28234 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:13:14.611999   747 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:14.643275   867 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.682s	user 0.002s	sys 0.015s
W20260709 13:13:14.667229   883 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:13:14.667032 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:14.668596 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:14.643163   524 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.640424   896 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:14.569041   827 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:14.635607   684 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:13:14.612111   350 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:14.655103   813 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:14.675415 28155 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:13:14.644256   800 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:14.609822   834 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:14.639717   705 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:14.632766   335 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:14.661756   898 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:14.847013 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:14.858690 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:14.852049   859 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.944s	user 0.001s	sys 0.003s
I20260709 13:13:14.763043 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2619) took 1130 ms. Trace:
I20260709 13:13:14.690980   687 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:14.842039 28157 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:14.842789 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2157) took 1734 ms. Trace:
I20260709 13:13:14.865216 28082 rpcz_store.cc:276] 0709 13:13:13.108385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.108942 (+   557us) service_pool.cc:224] Handling call
0709 13:13:14.842516 (+1733574us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b5400209754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:14.842682 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982653184}
W20260709 13:13:14.770889 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 124542us
I20260709 13:13:14.866652 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2621) took 1208 ms. Trace:
I20260709 13:13:14.867208 28155 rpcz_store.cc:276] 0709 13:13:13.658797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.664208 (+1005411us) service_pool.cc:224] Handling call
0709 13:13:14.674731 (+ 10523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.866365 (+191634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.866596 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:14.767098 28124 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.196:46855 timed out after 1.527s (SENT) [suppressed 31 similar messages]
W20260709 13:13:14.792548   794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.006s	sys 0.021s
W20260709 13:13:14.843690   894 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 13:13:14.821735   889 log.cc:927] Time spent T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.445s	user 0.000s	sys 0.001s
W20260709 13:13:14.844154   902 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.278s	user 0.000s	sys 0.000s
W20260709 13:13:14.814285   839 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:13:14.889057 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.686199   721 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:14.690140   524 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:14.916123   912 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:13:14.763159   872 raft_consensus.cc:1064] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:13:14.858726   800 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:14.691929   710 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:14.924991   797 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:14.883947   786 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:14.931450   692 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.801s	user 0.010s	sys 0.000s
W20260709 13:13:14.925256   841 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.545s	user 0.000s	sys 0.009s
I20260709 13:13:14.935092   845 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.935534 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2612) took 1548 ms. Trace:
I20260709 13:13:14.974586 28156 rpcz_store.cc:276] 0709 13:13:13.386890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.053495 (+666605us) service_pool.cc:224] Handling call
0709 13:13:14.054010 (+   515us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:14.054523 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:14.054566 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:14.069299 (+ 14733us) log.cc:844] Serialized 52 byte log entry
0709 13:13:14.327344 (+258045us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:14.327385 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.327451 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:14.935305 (+607854us) raft_consensus.cc:1685] finished
0709 13:13:14.935324 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:14.935475 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":826,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":111,"thread_start_us":205620,"threads_started":1,"wal-append.queue_time_us":383284}
I20260709 13:13:14.936753   827 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:14.937383   887 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:14.861685 28152 rpcz_store.cc:276] 0709 13:13:13.632940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.404896 (+771956us) service_pool.cc:224] Handling call
0709 13:13:14.409832 (+  4936us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:14.410291 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:14.410358 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:14.411233 (+   875us) log.cc:844] Serialized 26 byte log entry
0709 13:13:14.641349 (+230116us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:14.641386 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.641510 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:14.762788 (+121278us) raft_consensus.cc:1685] finished
0709 13:13:14.762808 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:14.762967 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":184335,"threads_started":1,"wal-append.queue_time_us":304423}
I20260709 13:13:14.811280   844 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:14.844588   893 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:13:14.945223 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2600) took 1610 ms. Trace:
I20260709 13:13:14.689004   607 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:14.915685   794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.011s	sys 0.038s
I20260709 13:13:14.747414   847 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.685889   863 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 11. Current term is 12.
I20260709 13:13:14.952510 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:15.001463 28155 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:15.003027 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:15.003916 28155 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:14.917173   898 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:14.923815   350 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:14.961814 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2617) took 1329 ms. Trace:
I20260709 13:13:14.959982 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:15.015723 28157 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:14.978653 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:14.967272   710 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:14.830286   896 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:14.994423 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:14.995508 28161 rpcz_store.cc:276] 0709 13:13:13.335693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.349488 (+ 13795us) service_pool.cc:224] Handling call
0709 13:13:13.361847 (+ 12359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.370309 (+  8462us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:13.370363 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:13.383749 (+ 13386us) log.cc:844] Serialized 26 byte log entry
0709 13:13:13.709439 (+325690us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:13.709495 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.709589 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:14.851607 (+1142018us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b24001b8c18. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4109c4 00007f59cb40eacb 00007f59cb5c9dc9 00007f59cb5c9a55 00007f59cb5c909d 00007f59ce1fb10e 00007f59ce21288c 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221
0709 13:13:14.941596 (+ 89989us) raft_consensus.cc:1685] finished
0709 13:13:14.941630 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:14.945125 (+  3495us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7967,"raft.queue_time_us":8117,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":10732,"spinlock_wait_cycles":19080064,"thread_start_us":255528,"threads_started":1,"wal-append.queue_time_us":440359}
I20260709 13:13:15.022193 28152 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:13:15.008970 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:15.001533   607 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.015347 28153 rpcz_store.cc:276] 0709 13:13:13.632318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.273331 (+641013us) service_pool.cc:224] Handling call
0709 13:13:14.315992 (+ 42661us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:14.316416 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:14.316464 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:14.322292 (+  5828us) log.cc:844] Serialized 26 byte log entry
0709 13:13:14.612815 (+290523us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:14.613227 (+   412us) log.cc:844] Serialized 14 byte log entry
0709 13:13:14.613472 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.613569 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:14.961533 (+347964us) raft_consensus.cc:1685] finished
0709 13:13:14.961554 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:14.961711 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3877,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":2805,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":3633,"thread_start_us":251841,"threads_started":1,"wal-append.queue_time_us":332157}
W20260709 13:13:15.021184 28154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2597) took 1689 ms (client timeout 1823 ms). Trace:
W20260709 13:13:15.025513 28154 rpcz_store.cc:269] 0709 13:13:13.331338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.331521 (+   183us) service_pool.cc:224] Handling call
0709 13:13:15.020953 (+1689432us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b5400255154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:15.021111 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413438848}
I20260709 13:13:15.027145 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:15.031250 28154 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:15.032895 28154 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:15.034682 28154 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:13:15.027961   607 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:15.021505   710 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:15.042397 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:15.043267 28153 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:14.981313   797 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:14.995384   845 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:15.040411 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 371907us
I20260709 13:13:14.980988   786 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.051854 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:15.052865 28157 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:15.005633   847 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.051317 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:15.059355 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.043820   911 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:13:15.063694 28082 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:15.067945 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:15.042245   896 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:15.071391 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:15.072379 28081 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:15.071388 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:15.069398 28082 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 16 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 13:13:15.069183 28158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 913) took 2250 ms (client timeout 1816 ms). Trace:
I20260709 13:13:15.073437 28086 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
W20260709 13:13:15.075106 28728 connection.cc:256] RPC call timeout handler was delayed by -1.2088849740000569s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 13:13:15.073619 28158 rpcz_store.cc:269] 0709 13:13:12.811383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.811670 (+   287us) service_pool.cc:224] Handling call
0709 13:13:15.053415 (+2241745us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54001978d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:15.061782 (+  8367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279224576}
I20260709 13:13:15.068884   721 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:13:15.074714   781 consensus_meta.cc:287] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.876s	user 0.010s	sys 0.003s
W20260709 13:13:15.083170 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 316084us
W20260709 13:13:15.084537 28728 connection.cc:256] RPC call timeout handler was delayed by -1.0594849740000427s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
W20260709 13:13:15.086659 28757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 1118) took 1395 ms (client timeout 1592 ms). Trace:
I20260709 13:13:15.087338 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:15.087092 28757 rpcz_store.cc:269] 0709 13:13:13.687979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.688178 (+   199us) service_pool.cc:224] Handling call
0709 13:13:15.083173 (+1394995us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b54004d1554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:15.083318 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184222592}
I20260709 13:13:15.094091 28153 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:15.096697 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:15.098729 28762 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:13:15.100288 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:15.098496 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:15.097615   705 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:15.105032 28756 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:13:15.097501   800 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:15.104254   687 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:15.102730   794 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.090044   781 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.117393   834 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:15.126516   705 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:15.125501   721 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:15.102609   788 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:13:15.131934   800 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:15.130283   524 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:15.135581   908 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
W20260709 13:13:15.156925 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.166796   797 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:15.167902 28228 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:15.168035   845 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:15.170365 28228 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:13:15.167588   875 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.027s	user 0.000s	sys 0.003s
I20260709 13:13:15.150736   705 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.157302   800 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:15.181027 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 90539.1us
I20260709 13:13:15.181730 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2609) took 1819 ms. Trace:
W20260709 13:13:15.182479 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.160540   782 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.002s
W20260709 13:13:15.175575   778 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.849s	user 0.014s	sys 0.000s
I20260709 13:13:15.185559 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2636) took 1118 ms. Trace:
I20260709 13:13:15.188405 28227 rpcz_store.cc:276] 0709 13:13:14.067472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.067653 (+   181us) service_pool.cc:224] Handling call
0709 13:13:14.068172 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:14.237270 (+169098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:14.237322 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:14.237983 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 13:13:14.613804 (+375821us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:14.613853 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.613946 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:15.185349 (+571403us) raft_consensus.cc:1685] finished
0709 13:13:15.185371 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.185498 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":283621,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":105,"thread_start_us":498036,"threads_started":2,"wal-append.queue_time_us":617287}
I20260709 13:13:15.185393 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:15.146328   794 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.184239 28159 rpcz_store.cc:276] 0709 13:13:13.362638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.831244 (+468606us) service_pool.cc:224] Handling call
0709 13:13:13.850267 (+ 19023us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:13.879355 (+ 29088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.879388 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:14.605159 (+725771us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b2c000b54b8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:13:14.640974 (+ 35815us) log.cc:844] Serialized 52 byte log entry
0709 13:13:15.057013 (+416039us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:15.057062 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.057159 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:15.172640 (+115481us) raft_consensus.cc:1685] finished
0709 13:13:15.172659 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.181614 (+  8955us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6237,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":5349,"spinlock_wait_cycles":4789504,"thread_start_us":381658,"threads_started":1,"wal-append.queue_time_us":446333}
I20260709 13:13:15.178038   705 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:15.191560 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44646.6us
I20260709 13:13:15.193845 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:15.196909 28152 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:15.203147 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:15.188899   782 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.206615   782 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:15.197705   546 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:15.197316   721 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.203172   786 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:15.198686   863 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
W20260709 13:13:15.217072   919 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 13:13:15.221999   899 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.755s	user 0.000s	sys 0.002s
W20260709 13:13:15.224998   901 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.262s	user 0.002s	sys 0.000s
I20260709 13:13:15.234515 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:15.235392   607 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.202715   847 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:15.241797   794 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:15.242019   879 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.987s	user 0.002s	sys 0.000s
W20260709 13:13:15.249701 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.254791   921 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:13:15.260229 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.261497 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.261956 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.263010 28730 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:15.263478 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:15.264359 28226 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:15.271159 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:15.272293 28757 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:15.267546   788 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.279481 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:15.279632   788 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.282033   788 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:15.283587   800 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.285642   926 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:15.286113   926 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:15.286509   926 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:15.287094   792 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:15.290572 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:15.291803 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:15.292918   700 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:15.293587   700 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:15.293942   700 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:15.294304   700 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:15.298134   735 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:15.298595   735 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:15.298952   735 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:15.305348 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:15.305881   607 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:15.306823 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:15.310073 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:15.318215   847 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:13:15.313038   788 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:15.319067   797 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:15.319679   847 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:15.323349 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.328217 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:15.326970 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:15.329540   607 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:15.327826 28226 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:15.334164 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:15.320313   788 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:15.320221   847 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.340040 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:15.341068 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:15.344671 28077 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:15.335011   863 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:15.349972 28049 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:15.330011   710 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.347209 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:15.336802   928 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:15.353368 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:15.354449 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 151368us
I20260709 13:13:15.395964 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:15.344964   786 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:15.345602   618 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.356046 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:15.330545   705 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.398893 28049 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:13:15.399631   920 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.260s	user 0.001s	sys 0.002s
I20260709 13:13:15.399734   845 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:15.353340   811 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.400446   811 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.402798   811 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:15.403640   926 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.405520 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:15.406723   335 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:15.407225   335 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:15.408068   335 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.408838   778 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:15.409358   778 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.409770   778 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:15.411584   781 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:15.411803   705 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:15.339912   829 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:13:15.415333 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:15.349045   927 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:15.354764   863 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.351482   846 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:15.350298   546 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:15.398200   524 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:15.354931   782 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:15.347604   792 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:13:15.415184 28082 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:15.415773 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:15.467561 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.413878 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57838.4us
I20260709 13:13:15.421578 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:15.422717   607 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:15.385180   928 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:15.356156   721 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:15.426388   710 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:15.426335 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:15.482127 28083 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:15.426846   735 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.428918   847 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:15.431882 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.438619   829 raft_consensus.cc:670] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } [suppressed 2 similar messages]
I20260709 13:13:15.414353   618 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.413934   788 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:15.447260 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:15.502511 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:15.470563 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:15.424698   927 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.475353 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:15.475390 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:15.425679   863 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:15.479372 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:15.487161 28226 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:15.629105 28226 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:15.619263 28085 raft_consensus.cc:2376] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14. Re-sending same reply.
I20260709 13:13:15.466811   782 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:15.497956 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.630276 28226 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.483808   778 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.455911 28077 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:15.480793   721 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:15.465178   846 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.478060   607 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.661859 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:15.529058   928 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.594386 28081 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:15.595016 28759 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:15.500044   546 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.597992   735 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:15.598557 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2637) took 1530 ms. Trace:
I20260709 13:13:15.601471   618 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:15.601902   455 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:15.601496 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 205602us
I20260709 13:13:15.603866   885 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:13:15.603960 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 1.14594e+06us
I20260709 13:13:15.685918 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2640) took 1434 ms. Trace:
W20260709 13:13:15.607115 28084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2168) took 1944 ms (client timeout 1955 ms). Trace:
I20260709 13:13:15.609886 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 968) took 1266 ms. Trace:
I20260709 13:13:15.610263 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 974) took 1265 ms. Trace:
W20260709 13:13:15.686614 28084 rpcz_store.cc:269] 0709 13:13:13.662985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.663139 (+   154us) service_pool.cc:224] Handling call
0709 13:13:15.606854 (+1943715us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002ad854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:15.607039 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2475197184}
I20260709 13:13:15.611121 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 973) took 1266 ms. Trace:
I20260709 13:13:15.664167   721 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:15.611896 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 337904us
I20260709 13:13:15.482877   932 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:13:15.618945 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.627359 28083 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
W20260709 13:13:15.627614 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 157145us
I20260709 13:13:15.698263 28127 leader_election.cc:400] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13.
I20260709 13:13:15.698571 28127 leader_election.cc:403] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:15.634482 28226 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:15.660151 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:15.663592 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:15.704569 28077 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:15.704869 28198 leader_election.cc:400] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14.
I20260709 13:13:15.705205 28198 leader_election.cc:403] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:15.673045   926 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:15.669907 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:15.673286 28233 rpcz_store.cc:276] 0709 13:13:14.067798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.068462 (+   664us) service_pool.cc:224] Handling call
0709 13:13:14.068784 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.069022 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:14.111021 (+ 41999us) log.cc:844] Serialized 14 byte log entry
0709 13:13:14.443015 (+331994us) log.cc:844] Serialized 14 byte log entry
0709 13:13:15.597923 (+1154908us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b1800369520. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:13:15.598070 (+   147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.598134 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:15.598195 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.598295 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.598477 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47658,"prepare.queue_time_us":101855,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":5196,"raft.queue_time_us":1405039,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":319,"spinlock_wait_cycles":19624192,"thread_start_us":291882,"threads_started":1}
I20260709 13:13:15.711376 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:15.628312   927 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:15.632810   797 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:15.644685   782 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:15.632637   882 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.173s	user 0.004s	sys 0.010s
I20260709 13:13:15.605516 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2635) took 1538 ms. Trace:
I20260709 13:13:15.610838   700 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.684239 28081 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:15.691105 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:15.687506 28235 rpcz_store.cc:276] 0709 13:13:14.343229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.634510 (+291281us) service_pool.cc:224] Handling call
0709 13:13:14.634936 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.635137 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:13:14.635448 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 13:13:15.216508 (+581060us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:15.609500 (+392992us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.609560 (+    60us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:15.609611 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.609687 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.609822 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":545494,"prepare.queue_time_us":347,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":546403,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":78,"spinlock_wait_cycles":2989952,"thread_start_us":418785,"threads_started":1,"wal-append.queue_time_us":459557}
I20260709 13:13:15.688068 28232 rpcz_store.cc:276] 0709 13:13:14.345223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.930433 (+585210us) service_pool.cc:224] Handling call
0709 13:13:14.930864 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.931161 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:14.931525 (+   364us) log.cc:844] Serialized 14 byte log entry
0709 13:13:15.609943 (+678418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.609995 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:15.610039 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.610105 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.610208 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":550626,"prepare.queue_time_us":737,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":117,"raft.queue_time_us":542274,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"spinlock_wait_cycles":8478592,"thread_start_us":136403,"threads_started":1,"wal-append.queue_time_us":194290}
I20260709 13:13:15.721649 28229 rpcz_store.cc:276] 0709 13:13:14.067144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.067295 (+   151us) service_pool.cc:224] Handling call
0709 13:13:14.067631 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.067873 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:14.313235 (+245362us) log.cc:844] Serialized 14 byte log entry
0709 13:13:14.674909 (+361674us) log.cc:844] Serialized 14 byte log entry
0709 13:13:15.605095 (+930186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.605159 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:15.605209 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.605289 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.605442 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2106773,"prepare.queue_time_us":63587,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":83,"raft.queue_time_us":991065,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":27871,"thread_start_us":539331,"threads_started":2}
I20260709 13:13:15.696348 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:15.688952 28234 rpcz_store.cc:276] 0709 13:13:14.344905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.800913 (+456008us) service_pool.cc:224] Handling call
0709 13:13:14.809770 (+  8857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.810033 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:13:14.823988 (+ 13955us) log.cc:844] Serialized 14 byte log entry
0709 13:13:15.254129 (+430141us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:15.610759 (+356630us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.610826 (+    67us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:15.610874 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.610945 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.611061 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":538648,"prepare.queue_time_us":13730,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":539530,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":3730816,"thread_start_us":246753,"threads_started":1,"wal-append.queue_time_us":310938}
I20260709 13:13:15.705622 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:15.731168 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:15.731552 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:15.731812 28234 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:15.668654 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:15.691057 28230 rpcz_store.cc:276] 0709 13:13:14.252107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.252331 (+   224us) service_pool.cc:224] Handling call
0709 13:13:14.309766 (+ 57435us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:14.460928 (+151162us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:14.612418 (+151490us) log.cc:844] Serialized 14 byte log entry
0709 13:13:15.607456 (+995038us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:15.608120 (+   664us) log.cc:844] Serialized 52 byte log entry
0709 13:13:15.608322 (+   202us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:15.685340 (+ 77018us) log.cc:844] Serialized 14 byte log entry
0709 13:13:15.685560 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.685639 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:15.685680 (+    41us) raft_consensus.cc:1685] finished
0709 13:13:15.685695 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.685844 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":701179,"prepare.queue_time_us":71808,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":1986124,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":194,"spinlock_wait_cycles":1748352,"thread_start_us":476921,"threads_started":3,"wal-append.queue_time_us":329316}
I20260709 13:13:15.709293 28226 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:13:15.711918 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:15.733103 28226 pending_rounds.cc:85] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 8
I20260709 13:13:15.733453 28226 pending_rounds.cc:107] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:13:15.733705 28226 raft_consensus.cc:2889] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:15.714484 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:15.685585   455 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.717454 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:15.742269 28233 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:15.743238 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:15.685516   524 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:15.629356   898 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:13:15.735697 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:15.774778 28230 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:15.699548   792 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:13:15.737063 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:15.737385 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:15.739671   845 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:15.714602   932 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 11 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:15.740515   829 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:15.743145 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:15.742306   896 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:15.744589 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:15.793064 28233 raft_consensus.cc:2410] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:13:15.750801 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:15.794390 28229 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
W20260709 13:13:15.766348 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 63834.4us
W20260709 13:13:15.767053 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 76013.6us
I20260709 13:13:15.719133   885 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14.
W20260709 13:13:15.775830 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 62704.5us
I20260709 13:13:15.736882   778 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:15.809329 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 111200us
W20260709 13:13:15.781463 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 44168.6us
W20260709 13:13:15.773039   917 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.007s	sys 0.019s
I20260709 13:13:15.695590   928 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:15.698508   936 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 13:13:15.765206 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:15.816184 28728 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:15.797384 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:15.754663   455 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:15.801190 28233 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:15.819725 28233 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:15.805403 28759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:15.775460   705 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13.
I20260709 13:13:15.804555 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:15.777541 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:15.810853 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:15.775794   797 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:15.814267 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:15.811700 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 106023us
I20260709 13:13:15.817654 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:15.820684 28759 raft_consensus.cc:606] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:15.824352 28759 raft_consensus.cc:614] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:15.800062   896 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:15.825328   939 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:13:15.825798   786 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:15.826169   700 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:15.826352   786 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:15.826711   932 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.827334   786 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.775272   773 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:13:15.825348   845 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:15.829177   845 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.829532   845 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:15.782302   847 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:13:15.780305   524 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.830257   847 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:15.830663   847 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:15.831669   524 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:15.811297   917 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.008s	sys 0.020s
W20260709 13:13:15.833431   942 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:13:15.811645   781 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:15.821283   351 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:13:15.838191   797 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.841109 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 972) took 1496 ms. Trace:
I20260709 13:13:15.841548 28231 rpcz_store.cc:276] 0709 13:13:14.344550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.746232 (+401682us) service_pool.cc:224] Handling call
0709 13:13:14.799377 (+ 53145us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:15.605296 (+805919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:15.605337 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:15.605923 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:13:15.690798 (+ 84875us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:15.690846 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.690912 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:15.840883 (+149971us) raft_consensus.cc:1685] finished
0709 13:13:15.840904 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.841047 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":805479,"raft.queue_time_us":805644,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":48,"thread_start_us":84487,"threads_started":1,"wal-append.queue_time_us":121400}
I20260709 13:13:15.843940 28231 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:13:15.809319   829 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:15.845400   829 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:15.848258   846 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:15.849365   941 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:13:15.825371   896 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:15.816479 28226 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 10.
I20260709 13:13:15.831480   847 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.862222 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:15.824934   607 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:15.858958 28230 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:15.850425 28233 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:15.860121 28234 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:15.885716 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:15.889820 28234 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:15.889914 28233 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:15.888499 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:15.888221 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:15.822830   885 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:13:15.897166 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 79550.2us
W20260709 13:13:15.897662 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 86877.1us
I20260709 13:13:15.899134 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:15.899384 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 83279.6us
I20260709 13:13:15.923919 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:15.927723 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 103265us
I20260709 13:13:15.897131   524 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:15.923362   792 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:13:15.888073   913 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:15.941592 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 53217.5us
I20260709 13:13:15.942225 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:15.955138 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:15.943074 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:15.948714 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:15.954476 28125 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:15.944357 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 130173us
I20260709 13:13:15.960111 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:15.960911 28199 leader_election.cc:400] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14.
I20260709 13:13:15.943876 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:15.965026 28763 raft_consensus.cc:2376] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12. Re-sending same reply.
I20260709 13:13:15.947846 28234 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:15.967243 28759 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:15.967998 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:15.968632 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:15.970240 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:15.976274 28760 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:13:15.972898 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:15.977975 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:15.927506   607 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:15.942971   936 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:15.934500   847 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:15.896837   951 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:13:15.949728   524 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:15.989156 28233 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:13:15.983759 28199 leader_election.cc:400] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:15.928408   949 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:15.965411   788 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:15.969065   805 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:16.011828 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:16.012177 28233 pending_rounds.cc:85] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 7
I20260709 13:13:16.012784 28199 leader_election.cc:403] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:13:15.973382   773 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:16.031286 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47626us
I20260709 13:13:15.958508   705 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:13:16.042872 28233 pending_rounds.cc:107] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:13:16.011482   700 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:15.976024   950 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:16.011152   926 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:15.979789   735 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:16.044144 28233 raft_consensus.cc:2889] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:16.013134   524 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:16.085704 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:16.086570 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.035324   898 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:16.017024   607 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:15.981752   909 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:16.044768   949 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:16.162971 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 208382us
I20260709 13:13:16.168059 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:16.042760   455 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:16.152335   805 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.152140   788 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:16.152237   700 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:16.183440   788 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:16.187901   805 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.050558   773 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:16.191154   805 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:16.192170   773 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:16.192579   700 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.193550   607 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.195663   946 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.196431   946 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.049602   782 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:16.204622 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 102456us
I20260709 13:13:16.209319 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:16.210270 28760 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:16.208384 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:16.215754 28232 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:16.154280   735 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.216614 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:16.179597   951 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:16.220620   963 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:16.220371   898 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:16.261642 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:16.214303   947 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.250s	user 0.000s	sys 0.008s
I20260709 13:13:16.208490   773 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:13:16.218842   805 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:16.391346 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:16.393193 28084 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:13:16.411228   455 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:16.412263   455 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:16.414136   946 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:16.259451   909 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:16.415115   909 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.415575   909 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:16.426227   782 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:16.431603   845 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.001s
I20260709 13:13:16.222909   951 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.437266   951 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:16.439335   898 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:16.440109   898 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:16.444486   927 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.703s	user 0.008s	sys 0.001s
I20260709 13:13:16.445127   927 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.453070   805 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:16.453572   805 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:16.453706 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2179) took 1038 ms. Trace:
I20260709 13:13:16.454111 28081 rpcz_store.cc:276] 0709 13:13:15.414953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.415084 (+   131us) service_pool.cc:224] Handling call
0709 13:13:16.453509 (+1038425us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54003a98d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:16.453641 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405980928}
I20260709 13:13:16.468353 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:16.473230   312 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.473968   312 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:16.474612   956 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.475230   956 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.447854   927 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:16.476531   312 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:16.394500   950 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:16.372617   735 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:16.432251   845 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.463721   455 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:16.504881 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.470952   782 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.483597   898 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:16.511421 28227 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:13:16.528998 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 356028us
I20260709 13:13:16.531153 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:16.507611   455 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:16.481580   946 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:13:16.527087   863 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.865s	user 0.012s	sys 0.000s
W20260709 13:13:16.544232 28126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.020s	sys 0.033s
W20260709 13:13:16.547947 28126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.020s	sys 0.033s
W20260709 13:13:16.544673 28235 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.007s	sys 0.041s
W20260709 13:13:16.550009 28235 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.007s	sys 0.042s
W20260709 13:13:16.545002 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.015s	sys 0.038s
W20260709 13:13:16.551105 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.015s	sys 0.038s
I20260709 13:13:16.545006 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:16.544608 28226 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.015s	sys 0.037s
W20260709 13:13:16.556483 28226 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.016s	sys 0.037s
I20260709 13:13:16.561396 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:16.551548 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 588600us
I20260709 13:13:16.586342 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:16.588172 28227 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:16.571398 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:16.594318 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:16.594745 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.527354   782 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:16.527993   936 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:13:16.597196 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:16.583756   927 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:13:16.613663 28231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.021s	sys 0.048s
I20260709 13:13:16.607095   950 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.632097 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:16.598111   312 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:16.554082   863 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:16.641459 28231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.022s	sys 0.052s
I20260709 13:13:16.595866   829 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:16.638693   721 consensus_meta.cc:287] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.932s	user 0.013s	sys 0.000s
W20260709 13:13:16.656234 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:16.657922 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:16.658542 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.636816   695 raft_consensus.cc:2764] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:13:16.632051   956 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:16.620018   786 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:16.674751   618 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:16.644251   927 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:16.643427   312 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.678599 28757 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:16.677170   885 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:16.638912   524 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.683772 28227 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:13:16.682885 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2660) took 1015 ms. Trace:
I20260709 13:13:16.644011   950 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:16.683974   896 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.795s	user 0.004s	sys 0.009s
I20260709 13:13:16.689227   312 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:16.700006   946 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:13:16.708971   695 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:16.670933   721 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.665279   829 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:16.720639 28127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.010s	sys 0.013s
I20260709 13:13:16.688745   618 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:16.689057   885 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:16.684960   351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.005s	sys 0.021s
W20260709 13:13:16.733954 28124 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.000s	sys 0.024s
W20260709 13:13:16.740097 28124 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.000s	sys 0.024s
W20260709 13:13:16.736690 28127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.010s	sys 0.013s
I20260709 13:13:16.687038   786 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:16.693046 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2659) took 1033 ms. Trace:
I20260709 13:13:16.719115   896 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:16.739754   351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.005s	sys 0.022s
I20260709 13:13:16.764961 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:16.760878 28228 rpcz_store.cc:276] 0709 13:13:15.659647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.787996 (+128349us) service_pool.cc:224] Handling call
0709 13:13:15.788297 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.788531 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:16.220019 (+431488us) log.cc:844] Serialized 14 byte log entry
0709 13:13:16.685989 (+465970us) log.cc:844] Serialized 14 byte log entry
0709 13:13:16.692551 (+  6562us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:16.692623 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:16.692688 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.692800 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:16.692971 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24977,"prepare.queue_time_us":138165,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":3768,"raft.queue_time_us":58439,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":6509,"thread_start_us":869719,"threads_started":2}
W20260709 13:13:16.765579 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 755536us
I20260709 13:13:16.766942 28234 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:13:16.751506 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 796361us
I20260709 13:13:16.784217 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:16.789331 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:16.793133 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:16.812529 28155 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:16.798527 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 985) took 1087 ms. Trace:
I20260709 13:13:16.815366 28231 rpcz_store.cc:276] 0709 13:13:15.711006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.844587 (+133581us) service_pool.cc:224] Handling call
0709 13:13:15.857244 (+ 12657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:15.865909 (+  8665us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:13:15.970195 (+104286us) log.cc:844] Serialized 14 byte log entry
0709 13:13:16.647051 (+676856us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:16.670410 (+ 23359us) log.cc:844] Serialized 26 byte log entry
0709 13:13:16.670576 (+   166us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:16.670625 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.670723 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:16.798284 (+127561us) raft_consensus.cc:1685] finished
0709 13:13:16.798307 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:16.798462 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67056,"prepare.queue_time_us":180718,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":96150,"raft.queue_time_us":30664,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":47303,"thread_start_us":632104,"threads_started":1,"wal-append.queue_time_us":814737}
I20260709 13:13:16.802371 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:16.793602   967 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
W20260709 13:13:16.842124 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 583982us
I20260709 13:13:16.845352 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:16.846225 28152 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:16.847913 28156 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:13:16.817056   847 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.806s	user 0.003s	sys 0.009s
W20260709 13:13:16.805968   970 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:13:16.805362   965 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:13:16.828605   971 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:16.817467 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 201977us
I20260709 13:13:16.853639 28155 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:16.847486 28199 leader_election.cc:400] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16.
I20260709 13:13:16.793306   975 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:16.773557   607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.000s	sys 0.035s
I20260709 13:13:16.862121 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:16.809640   863 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:16.742982   927 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:16.803548   788 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:16.844305   782 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:16.866838 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:16.878893 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:16.879539 28231 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:13:16.880468 28079 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:16.853593   526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.003s	sys 0.016s
W20260709 13:13:16.866730   924 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.717s	user 0.008s	sys 0.000s
I20260709 13:13:16.887852 28159 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:13:16.908916 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48184us
W20260709 13:13:16.865491   607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.000s	sys 0.040s
I20260709 13:13:16.921144   950 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:16.882269   782 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.856801   618 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.922139   782 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:16.910084   946 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.866237   966 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:13:16.925099   946 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.925683 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:16.927403   946 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:16.913332   973 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:16.838964   885 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:16.928066 28161 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:16.929117   985 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:13:16.911849   526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.004s	sys 0.016s
I20260709 13:13:16.930393   526 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:16.930943   526 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.932209 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:16.866963   829 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:16.916992 28159 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:16.917157   524 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:16.919497 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:16.928200   973 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:16.937176   984 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:13:16.938243 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:16.938704   973 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:16.924460   782 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:16.939458 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:16.940155 28079 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:16.946705 28158 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:16.949147 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:16.951377 28159 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:16.921016 28231 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:16.953560 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:16.955031 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:16.952656 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:16.957533   946 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:16.968879 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:16.975818 28158 raft_consensus.cc:2376] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13. Re-sending same reply.
I20260709 13:13:16.942823   805 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:16.984241 28230 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 13:13:16.984767 28230 pending_rounds.cc:85] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 13
I20260709 13:13:16.985049 28230 pending_rounds.cc:107] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 13:13:16.985323 28230 raft_consensus.cc:2889] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:16.969616   932 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:16.989202 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:16.865521   971 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:16.991122 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 203230us
I20260709 13:13:16.962872   985 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:13:16.981931   946 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:16.997484 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:17.003473 28158 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:16.976778   991 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:13:16.996387 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 192373us
I20260709 13:13:17.022861 28156 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:17.023236 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:16.992022   932 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.028918   786 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:17.030857   986 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:13:17.036597   896 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:17.042021   805 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:17.001299   607 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:13:17.044521   721 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:17.058769 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:17.059126   971 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:17.059494 28231 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:17.060478 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:17.061488   988 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:13:17.061739 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:17.062574   526 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:17.046718   932 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:17.084550 28156 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
W20260709 13:13:17.063596   983 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:13:17.088704 28159 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:17.092276 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.057044   805 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:17.097726 28231 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
W20260709 13:13:17.111814 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.125694 28755 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:17.072180   455 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:17.129580 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:17.131426 28158 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:17.135823 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:17.139393 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:17.140719 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:17.141811 28760 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:17.138403 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:17.158847   455 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:17.123745   800 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:17.166206 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:17.177016 28755 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:17.180970 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:17.181437 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 184029us
I20260709 13:13:17.181973 28154 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:17.182359 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:17.183228 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.184067 28160 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:17.184118 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:17.188638 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 50336.9us
I20260709 13:13:17.188882 28762 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:17.188894 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:17.189390 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.190368 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:17.190827 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:17.193351 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:17.186728 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:17.195830 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.196116 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.197183 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:17.200266 28127 connection.cc:256] RPC call timeout handler was delayed by -2.1019565560000046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260709 13:13:17.097412   946 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:17.205070 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:17.186465   999 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:17.201004   792 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:17.199390   800 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:17.198204   991 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:17.213807 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:17.214633 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:17.201421   524 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:13:17.212639   951 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.773s	user 0.014s	sys 0.000s
I20260709 13:13:17.216197 28153 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:13:17.224138   992 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:13:17.195353   618 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:13:17.195313   312 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:17.229210 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.230976 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:17.232457 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:17.234244 28762 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:13:17.234391 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:17.235702   991 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:17.236981 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.237879 28153 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
W20260709 13:13:17.238770   909 consensus_meta.cc:287] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.823s	user 0.007s	sys 0.006s
I20260709 13:13:17.239032 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.239580 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.240228 28159 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:17.239198   909 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:17.244212 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50966.2us
I20260709 13:13:17.195091   797 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:13:17.246315 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:17.246316 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:17.245327 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:17.248992 28153 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:17.248868 28157 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:17.210858   786 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 13:13:17.256281   995 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:13:16.693596 28229 rpcz_store.cc:276] 0709 13:13:15.667547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.795518 (+127971us) service_pool.cc:224] Handling call
0709 13:13:15.795883 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.796132 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:16.258811 (+462679us) log.cc:844] Serialized 14 byte log entry
0709 13:13:16.682536 (+423725us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:16.682611 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:16.682650 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.682714 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:16.682828 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20762,"prepare.queue_time_us":184432,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4604,"raft.queue_time_us":1409,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":6396,"thread_start_us":806396,"threads_started":2}
I20260709 13:13:17.224867   792 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:17.227254   999 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:17.263944 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:17.267071 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.270340 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:17.270991 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54441.8us
I20260709 13:13:17.244323   994 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:17.276536 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:17.293035 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.295343 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:17.296209 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.296857 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:17.269563   797 consensus_queue.cc:1243] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:13:17.225837   951 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:17.302153 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:17.303586   999 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:17.303881   994 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:17.305100 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:17.251294   927 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:13:17.305716   926 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:17.319387   999 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:17.321704   999 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:17.302460   872 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:17.324023   845 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:17.278316   786 consensus_queue.cc:1243] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:13:17.326067 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:17.374409 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72835.7us
I20260709 13:13:17.374855 28154 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:13:17.380052 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54093.6us
I20260709 13:13:17.380156 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:17.337227 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:17.381026 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:17.381210 28231 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:17.335702 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:17.383275 28227 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:17.265341   928 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:13:17.379302   607 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:13:17.387794  1012 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:13:17.376544 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:17.392160 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:17.397966 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 66137.4us
I20260709 13:13:17.399505 28155 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:17.381529 28229 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:17.378686  1016 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:13:17.325193   994 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:17.399083 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:17.391023   710 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:17.403028   845 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:13:17.423684   710 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:17.424249   710 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:17.299247   526 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:13:17.308561   781 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:13:17.424744   710 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:17.398329   607 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:17.423731  1008 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:13:17.398559  1002 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:17.392372  1009 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:17.308261   976 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.136s
W20260709 13:13:17.450333 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 116035us
W20260709 13:13:17.448005 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50275.7us
W20260709 13:13:17.471029 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 175813us
W20260709 13:13:17.471460 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 72475.6us
I20260709 13:13:17.451876   607 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:17.465724   946 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:17.483809   909 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:17.485560   932 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:17.400483   792 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:17.451902  1017 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:17.491050   962 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.454s	user 0.008s	sys 0.005s
I20260709 13:13:17.502540  1018 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:17.481943 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:17.472419  1016 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:17.449331   994 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:17.480614  1010 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:13:17.510138 28763 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
W20260709 13:13:17.506906 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 114850us
I20260709 13:13:17.513126 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:17.514587 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:17.521876 28159 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:13:17.526088   312 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:17.480095  1002 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:13:17.498415  1017 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:17.529699 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:17.540504 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.540611 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:17.543557 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:17.544335 28226 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:17.512611   909 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:17.512343   951 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:17.551597 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:17.556907 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:17.536185  1016 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:17.557814  1019 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:17.537871  1017 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:17.564337 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:17.527222   829 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:13:17.548905   781 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:17.546990  1021 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:17.568192 28762 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:17.555433   773 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.564159   991 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:17.567409  1019 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.576858   781 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.618634 28226 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:17.624418 28764 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:17.620610 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:17.629891 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 159843us
W20260709 13:13:17.630234 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 59903.3us
I20260709 13:13:17.619092   781 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:17.589704  1019 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:17.619616   909 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:17.624003   845 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.619743   991 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:17.590844   773 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:17.720749 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:17.664719  1023 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:13:17.723666 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 52592.1us
I20260709 13:13:17.651271  1005 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:13:17.664947  1022 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:13:17.673708   786 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.743103 28235 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:17.748821 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:17.753824 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:17.664017   845 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:17.669783  1032 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:13:17.781764 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.668305   350 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:13:17.787091 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.756682   932 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:17.777123  1042 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:17.650581  1016 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:17.781653  1032 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:17.785722   963 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:17.776621   526 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:13:17.738472  1034 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:13:17.802347   991 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:17.810071  1002 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:17.849313  1048 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:13:17.851416 28226 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:17.841269   932 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:13:17.852820 28231 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:17.852936   700 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:17.981694   926 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:17.982798   926 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:17.982307   700 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:17.983203   926 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:17.856117   959 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.690s	user 0.004s	sys 0.010s
I20260709 13:13:17.857326 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:17.859592 28761 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:17.863283  1030 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:13:17.865334  1045 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.870029 28758 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:17.824712   963 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:17.875061 28158 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 16: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:17.882254 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:17.807525   710 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:17.907366   773 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:17.914263 28232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 2652) took 2314 ms (client timeout 1893 ms). Trace:
I20260709 13:13:17.808820  1042 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:17.851501  1047 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:18.013177 28232 rpcz_store.cc:269] 0709 13:13:15.599823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.736814 (+136991us) service_pool.cc:224] Handling call
0709 13:13:17.894326 (+2157512us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54001c70d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:17.914126 (+ 19800us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":996828416}
I20260709 13:13:17.915652 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2623) took 1102 ms. Trace:
I20260709 13:13:17.915863  1032 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:17.921911 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:17.930615 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1034) took 2001 ms. Trace:
I20260709 13:13:17.931658 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:17.746727   786 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:17.945833 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 316283us
I20260709 13:13:17.975765 28762 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:17.983433  1043 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:18.042614 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:18.019791 28756 rpcz_store.cc:276] 0709 13:13:15.929561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.929745 (+   184us) service_pool.cc:224] Handling call
0709 13:13:15.963010 (+ 33265us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:16.053236 (+ 90226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:16.053280 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:16.152616 (+ 99336us) log.cc:844] Serialized 52 byte log entry
0709 13:13:16.527617 (+375001us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:16.848456 (+320839us) log.cc:844] Serialized 14 byte log entry
0709 13:13:16.848719 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.893566 (+1044847us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b54004d21d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:17.930094 (+ 36528us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.930164 (+    70us) raft_consensus.cc:1685] finished
0709 13:13:17.930282 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.930530 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3927,"prepare.queue_time_us":142598,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":1815,"raft.queue_time_us":4172,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":17815,"spinlock_wait_cycles":182284160,"thread_start_us":649993,"threads_started":2,"wal-append.queue_time_us":531377,"wal-append.run_cpu_time_us":7337,"wal-append.run_wall_time_us":1250945}
I20260709 13:13:18.052517 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:18.052902 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.053990 28232 raft_consensus.cc:2410] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:13:18.054647 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:18.055529 28758 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:18.055747 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:18.056464 28227 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:18.014616 28161 rpcz_store.cc:276] 0709 13:13:16.812966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.842879 (+ 29913us) service_pool.cc:224] Handling call
0709 13:13:16.855501 (+ 12622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.928625 (+ 73124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.896287 (+967662us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b5400313154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:17.915573 (+ 19286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165747456}
I20260709 13:13:18.056495 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.057581 28226 raft_consensus.cc:2410] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 14 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:13:18.058275 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:18.058516 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:17.975375   909 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.037676 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:18.041049 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:18.064990 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 444466us
I20260709 13:13:18.069480 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:18.069480 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:18.069664  1059 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:18.070780  1059 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:18.071089 28760 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:18.071642  1059 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:18.072856 28229 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:18.074290 28229 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:18.059346   926 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:18.078907   926 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:18.081044 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:18.081637   845 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:18.085068 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:18.058842  1032 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:18.088709  1055 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:13:18.085058   710 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:18.089824   963 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:18.090968 28764 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:18.091053   963 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:18.018879  1047 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.095093  1047 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:18.095361  1043 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:18.095796  1043 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:18.015671  1045 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:18.111392 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2673) took 1318 ms. Trace:
I20260709 13:13:18.121759 28234 rpcz_store.cc:276] 0709 13:13:16.793297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.794373 (+  1076us) service_pool.cc:224] Handling call
0709 13:13:16.794737 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:17.972601 (+1177864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.972641 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:17.972695 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:17.972762 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.111122 (+138360us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400306e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:18.111179 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:18.111326 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2534355840}
I20260709 13:13:18.108326 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:18.131244 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.141247 28231 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:18.144855 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.147511 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:18.148772 28227 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
W20260709 13:13:18.129369   985 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.072s	user 0.015s	sys 0.013s
I20260709 13:13:18.060160   350 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:18.157423 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.149091 28159 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:18.150221 28049 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:18.089557  1058 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.131417   700 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:18.159094 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:18.155819   985 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.015s	sys 0.013s
I20260709 13:13:18.183531 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:18.140093   773 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:18.148970   797 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.076926   909 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:18.194473 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.182299  1060 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 13:13:18.197201 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.207702 28229 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:18.207767 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:18.210253  1067 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:18.212105   971 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:18.212548   971 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:18.213522   971 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:18.217132   786 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:18.220961   350 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:18.189327  1011 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:13:18.225188  1011 consensus_queue.cc:1243] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:18.226037  1042 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:18.189144   976 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:18.107718  1043 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:18.238168 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.239068 28235 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:18.214632   971 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:18.178599   526 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:13:18.242674 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:18.161922 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:18.252694 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.228497  1045 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:18.275213 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.193675   773 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:18.196954   797 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:18.287396 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.246970   827 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:18.178583  1058 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:18.308223 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:18.278249   999 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.956s	user 0.018s	sys 0.009s
W20260709 13:13:18.323407 28757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 1130) took 2996 ms (client timeout 1518 ms). Trace:
I20260709 13:13:18.325305 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:18.330662 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 880) took 1206 ms. Trace:
I20260709 13:13:18.342278 28763 rpcz_store.cc:276] 0709 13:13:17.124219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.124880 (+   661us) service_pool.cc:224] Handling call
0709 13:13:17.125246 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.510666 (+385420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.330312 (+819646us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b5400562cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:18.330588 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":884569472}
W20260709 13:13:18.301786   607 consensus_meta.cc:287] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.803s	user 0.015s	sys 0.002s
W20260709 13:13:18.339402 28757 rpcz_store.cc:269] 0709 13:13:15.326490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.326613 (+   123us) service_pool.cc:224] Handling call
0709 13:13:18.323072 (+2996459us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54004371d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:18.323262 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3084178304}
I20260709 13:13:18.366753 28160 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:18.368063 28081 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:18.299190  1043 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:18.309070   994 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.787s	user 0.012s	sys 0.008s
I20260709 13:13:18.268652   794 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 13:13:18.380944 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.370832 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:18.321409  1068 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:13:18.373238  1058 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:18.375837 28226 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:18.378654 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:18.395488 28078 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:18.280117  1002 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:18.381897 28081 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:18.371606 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:18.301736   786 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:18.326093  1017 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.749s	user 0.021s	sys 0.003s
I20260709 13:13:18.382201 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:18.383625 28079 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:18.398094 28086 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:18.402101 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:18.335608   797 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:18.403448 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:18.404562 28082 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:13:18.409142 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:18.421157 28085 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:13:18.425586 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.427611 28764 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:13:18.432570 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.399122   526 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:18.437742 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:18.419638 28154 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:18.401249   781 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.747s	user 0.007s	sys 0.013s
I20260709 13:13:18.387271   994 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:18.443599 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:18.445731   705 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:13:18.447417  1065 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:13:18.448262   773 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:18.369621   999 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:18.407433  1017 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:18.405381   786 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:18.381000   607 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:18.437177   526 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:18.456314 28158 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 934) took 1245 ms (client timeout 1584 ms). Trace:
I20260709 13:13:18.456195  1047 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:18.499717 28158 rpcz_store.cc:269] 0709 13:13:17.210535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.212774 (+  2239us) service_pool.cc:224] Handling call
0709 13:13:18.456029 (+1243255us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b540010a754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:18.456230 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511006464}
W20260709 13:13:18.459125 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.462122 28757 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:18.461920 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:18.462410 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:18.462786 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:18.482734   797 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:18.466285 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.466898 28763 raft_consensus.cc:1217] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Deduplicated request from leader. Original: 12.12->[13.13-13.13]   Dedup: 13.13->[]
I20260709 13:13:18.428323 28078 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:18.485074 28761 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:13:18.496443 28232 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:18.507592 28763 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:13:18.502341 28085 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:18.464284 28759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 879) took 1358 ms. Trace:
I20260709 13:13:18.502769 28233 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
W20260709 13:13:18.493551  1063 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:13:18.507556 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:18.511030 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:18.511603 28235 raft_consensus.cc:2376] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11. Re-sending same reply.
I20260709 13:13:18.512485 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.514914 28235 raft_consensus.cc:2410] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:13:18.520347 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:18.521493 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.521723 28083 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:18.482507   781 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:18.525234 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.532534 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.508679 28759 rpcz_store.cc:276] 0709 13:13:17.105864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.105990 (+   126us) service_pool.cc:224] Handling call
0709 13:13:17.106468 (+   478us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:17.134890 (+ 28422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.134941 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:17.857925 (+722984us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b18004d7340. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a
0709 13:13:17.859366 (+  1441us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b2c00047778. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:13:18.037837 (+178471us) log.cc:844] Serialized 52 byte log entry
0709 13:13:18.307501 (+269664us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:18.389573 (+ 82072us) log.cc:844] Serialized 14 byte log entry
0709 13:13:18.397107 (+  7534us) log.cc:844] Serialized 14 byte log entry
0709 13:13:18.397597 (+   490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.397706 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:18.464012 (+ 66306us) raft_consensus.cc:1685] finished
0709 13:13:18.464040 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:18.464215 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1020,"prepare.queue_time_us":45408,"prepare.run_wall_time_us":41,"raft.queue_time_us":2834,"raft.run_cpu_time_us":458,"raft.run_wall_time_us":24160,"spinlock_wait_cycles":34064768,"thread_start_us":214640,"threads_started":1,"wal-append.queue_time_us":297598}
I20260709 13:13:18.497741   773 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:13:18.536540 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:18.546618  1069 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:13:18.552671 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:18.553210 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:18.548907 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 171563us
I20260709 13:13:18.554553 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:18.561012 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:18.562808 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:18.564271 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:18.527729   963 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:13:18.620615   863 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:18.621229   863 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:18.621760  1019 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.968s	user 0.009s	sys 0.014s
I20260709 13:13:18.622195  1019 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:18.623436   863 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:18.556974  1071 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:13:18.624253  1019 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:18.625659  1073 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:13:18.626063  1017 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:18.631783   524 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:18.634419 28229 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:13:18.639855   926 consensus_meta.cc:287] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.000s
I20260709 13:13:18.641235   994 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:18.645726   786 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:18.648167  1047 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:18.532020   526 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:18.679353 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.558136  1077 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.906822 28760 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:13:18.907452 28227 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:13:18.931388 28760 pending_rounds.cc:85] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 11
I20260709 13:13:18.931708 28760 pending_rounds.cc:107] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:13:18.932032 28760 raft_consensus.cc:2889] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:18.938987 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 374781us
I20260709 13:13:18.939766 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:18.942121 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.981891   705 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:19.021471 28125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.044s	sys 0.115s
W20260709 13:13:19.033269 28125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.044s	sys 0.115s
W20260709 13:13:19.033689 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 561349us
I20260709 13:13:19.052655 28229 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:13:18.982442   786 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:18.987465   526 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:19.064069  1067 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.853s	user 0.011s	sys 0.002s
I20260709 13:13:19.062754   994 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:19.062520  1017 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:19.117053 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.119518 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:19.122674 28082 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:19.082815   773 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:19.125754 28078 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:18.941905  1077 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:19.121502 28161 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.037s	sys 0.055s
I20260709 13:13:19.093873   705 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:19.145094 28161 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.037s	sys 0.056s
I20260709 13:13:19.107399  1067 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:19.166147 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:19.153496 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:19.167567 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2651) took 1358 ms. Trace:
I20260709 13:13:19.159945 28082 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:19.152024   705 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.125253   786 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.134936  1086 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:19.148356 28229 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:13:19.140012  1028 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.516s	user 0.011s	sys 0.009s
W20260709 13:13:19.194621  1072 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 13:13:19.195323 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2682) took 1860 ms. Trace:
W20260709 13:13:19.196173  1079 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
I20260709 13:13:19.197649  1084 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:19.198395   909 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:19.199007   909 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:19.199504   909 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.201331  1029 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:19.201838  1029 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:19.198418  1084 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.204635  1084 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:19.169378 28161 rpcz_store.cc:276] 0709 13:13:17.809333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.083360 (+274027us) service_pool.cc:224] Handling call
0709 13:13:18.083780 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.084050 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:18.084422 (+   372us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.167077 (+1082655us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:19.167158 (+    81us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:19.167219 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.167311 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.167490 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13107,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":113173,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":1082148,"threads_started":2,"wal-append.queue_time_us":472181}
I20260709 13:13:19.206751 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1146) took 1139 ms. Trace:
I20260709 13:13:19.207147 28760 rpcz_store.cc:276] 0709 13:13:18.067219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.071816 (+  4597us) service_pool.cc:224] Handling call
0709 13:13:18.072119 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.932529 (+860410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.206536 (+274007us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54005519d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:19.206697 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1892007296}
I20260709 13:13:19.133684  1042 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:19.208191  1042 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.194439   800 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:19.209197   800 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:19.210803  1042 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:19.210145   800 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.211638  1046 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:19.211735   800 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:19.195924 28228 rpcz_store.cc:276] 0709 13:13:17.334407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.337484 (+  3077us) service_pool.cc:224] Handling call
0709 13:13:17.337870 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.389296 (+ 51426us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:17.389333 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:17.389953 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 13:13:17.472593 (+ 82640us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:17.472634 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.472695 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:18.911962 (+1439267us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:19.194927 (+282965us) raft_consensus.cc:1685] finished
0709 13:13:19.194957 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.195241 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50934,"raft.queue_time_us":123260,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":14426,"spinlock_wait_cycles":9114880,"thread_start_us":82182,"threads_started":1,"wal-append.queue_time_us":190056}
I20260709 13:13:19.209213 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2699) took 1018 ms. Trace:
I20260709 13:13:19.213568 28228 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:13:19.214171  1077 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:19.214850  1077 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:19.216233  1077 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.208313  1029 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.144752   607 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:19.213276 28227 rpcz_store.cc:276] 0709 13:13:18.190996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.191696 (+   700us) service_pool.cc:224] Handling call
0709 13:13:18.894648 (+702952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.931488 (+ 36840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.209032 (+277544us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b54001f59d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:19.209157 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1615953152}
I20260709 13:13:19.146876   999 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:19.139581   898 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:19.217336 32708 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:19.217901 32708 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:19.217728   898 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.218650 28227 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:19.218961 32708 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.219831   898 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:19.219996 28227 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:13:19.220347 28227 pending_rounds.cc:85] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 8
I20260709 13:13:19.220618 28227 pending_rounds.cc:107] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:13:19.221288 28227 raft_consensus.cc:2889] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:19.225371   971 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.952s	user 0.007s	sys 0.007s
I20260709 13:13:19.225965   971 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:19.227099   800 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:19.230384 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2653) took 1419 ms. Trace:
I20260709 13:13:19.233836 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 925) took 2030 ms. Trace:
I20260709 13:13:19.234364 28157 rpcz_store.cc:276] 0709 13:13:17.203490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.203642 (+   152us) service_pool.cc:224] Handling call
0709 13:13:17.214389 (+ 10747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:17.250477 (+ 36088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.233518 (+1983041us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b5400051c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:19.233758 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78604032}
I20260709 13:13:19.236513 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2661) took 1254 ms. Trace:
I20260709 13:13:19.240041 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:19.240208 28157 rpcz_store.cc:276] 0709 13:13:17.982437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.235464 (+1253027us) service_pool.cc:224] Handling call
0709 13:13:19.235893 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:19.236099 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:19.236135 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:19.236191 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:19.236239 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.236310 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.236445 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:19.224341   885 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:19.152148  1032 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 1.065s	user 0.010s	sys 0.005s
I20260709 13:13:19.241326 28078 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:19.243400 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 303674us
I20260709 13:13:19.147017   781 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:19.230938 28153 rpcz_store.cc:276] 0709 13:13:17.810605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.114490 (+303885us) service_pool.cc:224] Handling call
0709 13:13:18.114860 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.115140 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:18.115488 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.229492 (+1114004us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b180045a180. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:13:19.229768 (+   276us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.229995 (+   227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:19.230057 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:19.230115 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.230192 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.230316 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45475,"prepare.queue_time_us":2134,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":66,"raft.queue_time_us":4282,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":6663,"spinlock_wait_cycles":47637888,"thread_start_us":128745,"threads_started":1,"wal-append.queue_time_us":289313}
I20260709 13:13:19.251734 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:19.255137 28153 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:19.125052   994 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.257457 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2493) took 1123 ms. Trace:
I20260709 13:13:19.260856 28153 rpcz_store.cc:276] 0709 13:13:18.134078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.252308 (+1118230us) service_pool.cc:224] Handling call
0709 13:13:19.252648 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:19.257182 (+  4534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.257379 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:19.135185  1017 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.258780 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:19.260043 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:19.239588 32708 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:19.216255 28228 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:19.264820 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:19.265687 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:19.255226  1076 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.686s	user 0.000s	sys 0.001s
I20260709 13:13:19.263913 28157 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:19.191004  1086 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:19.212122  1046 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:19.271595 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:19.275411 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2492) took 1142 ms. Trace:
I20260709 13:13:19.278721 28157 rpcz_store.cc:276] 0709 13:13:18.133288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.242975 (+1109687us) service_pool.cc:224] Handling call
0709 13:13:19.263691 (+ 20716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:19.275073 (+ 11382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.275339 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:19.273972 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:19.285604 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:19.289143 28082 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:19.242848   786 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.275272 28228 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:19.295038 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.268493   971 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:19.296638 28228 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.216694  1077 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.305187 28228 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:19.301920 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:19.306486 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 362758us
I20260709 13:13:19.290836  1042 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:13:19.299240  1043 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.917s	user 0.012s	sys 0.000s
W20260709 13:13:19.335757 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.297782  1087 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:19.343750 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.308065   971 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:19.320751   863 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:19.357734 28156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2628) took 2500 ms (client timeout 1677 ms). Trace:
W20260709 13:13:19.359539 28156 rpcz_store.cc:269] 0709 13:13:16.858396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.861857 (+  3461us) service_pool.cc:224] Handling call
0709 13:13:19.357482 (+2495625us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400254c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:19.357645 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367948160}
I20260709 13:13:19.360611 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:19.362142 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:19.363209 28153 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 16: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:19.345543   524 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:19.345620  1091 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.369539 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:19.373163 28228 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:19.375361 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2232) took 1050 ms. Trace:
I20260709 13:13:19.375891 28228 rpcz_store.cc:276] 0709 13:13:18.324526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.215860 (+891334us) service_pool.cc:224] Handling call
0709 13:13:19.216138 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:19.374846 (+158708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.375292 (+   446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:19.376928 28153 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:19.352705   971 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:19.379150 28156 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:13:19.379375 28157 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:13:19.348032  1042 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:19.352403  1085 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:19.347813   946 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
W20260709 13:13:19.384464 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.399631 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:19.402326 28156 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 13:13:19.404675  1017 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:19.357405  1046 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.404505  1090 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:19.403533  1077 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:19.366683  1082 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:19.347586  1087 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:19.407793 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.407914 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:19.404500 28156 pending_rounds.cc:85] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 12
I20260709 13:13:19.414492 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:19.395953   524 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.420809 28156 pending_rounds.cc:107] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:13:19.420007 28759 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:19.359053   863 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.423353 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:19.400592  1093 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.408067   705 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:19.426167 28761 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:19.430014 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:19.431252 28156 raft_consensus.cc:2889] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:19.433315 28760 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:19.427136 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:19.346040  1086 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.434487 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:19.432159   607 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:13:19.448029   932 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:19.448436  1093 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.409875   909 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:19.448927   946 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:13:19.451697 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:19.395538  1088 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 13:13:19.448318   705 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:19.467345  1093 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:19.468637   896 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:13:19.422791  1087 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:19.471359 28156 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:19.452960  1046 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:19.380671  1091 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:19.483042 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 438378us
I20260709 13:13:19.474524  1086 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:19.484203 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:19.473739   705 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.475998  1088 consensus_queue.cc:1243] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 17.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:13:19.439999   994 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:19.492208 28125 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:19.493741 28125 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:19.488672  1089 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:13:19.489982   705 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:19.482765   896 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:19.494747  1088 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:19.498831  1087 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:19.496639   786 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:19.501796 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:19.495414  1092 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:13:19.474144  1042 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.431599   524 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.504154   896 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.514707   896 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:19.515621 28127 leader_election.cc:336] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.736s (SENT)
I20260709 13:13:19.505007  1087 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:19.526870 28127 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.839s (SENT)
I20260709 13:13:19.529081 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:19.530439 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2702) took 1305 ms. Trace:
W20260709 13:13:19.531771 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.506628  1088 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:13:19.543879 28127 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.925s (SENT)
I20260709 13:13:19.520363   524 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:19.552054 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2658) took 1570 ms. Trace:
I20260709 13:13:19.572599 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 937) took 2360 ms. Trace:
I20260709 13:13:19.554100 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1003) took 2468 ms. Trace:
I20260709 13:13:19.573233 28152 rpcz_store.cc:276] 0709 13:13:17.211836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.449030 (+237194us) service_pool.cc:224] Handling call
0709 13:13:17.449372 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.459297 (+  9925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.459344 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:17.464463 (+  5119us) log.cc:844] Serialized 26 byte log entry
0709 13:13:18.867120 (+1402657us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b18003460c0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:13:19.082420 (+215300us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:19.082507 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.082637 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.567779 (+485142us) raft_consensus.cc:1685] finished
0709 13:13:19.567809 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.572504 (+  4695us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9453,"raft.queue_time_us":11742,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":2900,"spinlock_wait_cycles":3717888,"thread_start_us":175541,"threads_started":1,"wal-append.queue_time_us":1888356}
W20260709 13:13:19.557685 28127 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.969s (SENT)
I20260709 13:13:19.541589 28235 rpcz_store.cc:276] 0709 13:13:18.225353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.791221 (+565868us) service_pool.cc:224] Handling call
0709 13:13:18.910516 (+119295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.910820 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:19.305784 (+394964us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.528468 (+222684us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:19.528526 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:19.528570 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.530212 (+  1642us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.530363 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":574940,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":1384,"raft.queue_time_us":123,"thread_start_us":511943,"threads_started":2}
W20260709 13:13:19.562916 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.530736  1058 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:19.519881   863 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:19.574164 28230 rpcz_store.cc:276] 0709 13:13:17.072701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.093440 (+ 20739us) service_pool.cc:224] Handling call
0709 13:13:17.098717 (+  5277us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:17.099161 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.099205 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:17.099979 (+   774us) log.cc:844] Serialized 52 byte log entry
0709 13:13:17.198439 (+ 98460us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:13:19.216431 (+2017992us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x7b1800437620. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf
0709 13:13:19.216601 (+   170us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.522407 (+305806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.522507 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.522548 (+    41us) raft_consensus.cc:1685] finished
0709 13:13:19.540783 (+ 18235us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.541014 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13998,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":1220,"raft.queue_time_us":61196,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":3168,"spinlock_wait_cycles":10546048,"thread_start_us":544717,"threads_started":3,"wal-append.queue_time_us":272106,"wal-append.run_cpu_time_us":2069,"wal-append.run_wall_time_us":1085504}
I20260709 13:13:19.527750   805 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:19.570680 28156 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:19.576427 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:19.583695   885 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:19.584241   913 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.587260 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:19.588551 28759 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:19.589771 28763 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:19.593255 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:19.593339  1088 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.567059  1083 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:19.564072   946 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.595551 28758 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:19.598770 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:19.630087 28756 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:19.599149 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2205) took 1223 ms. Trace:
I20260709 13:13:19.630980 28083 rpcz_store.cc:276] 0709 13:13:18.376087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.376222 (+   135us) service_pool.cc:224] Handling call
0709 13:13:18.376568 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.522083 (+145515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.598926 (+1076843us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54000e1d54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:19.599091 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333379328}
I20260709 13:13:19.571259 28154 rpcz_store.cc:276] 0709 13:13:17.981559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.484615 (+503056us) service_pool.cc:224] Handling call
0709 13:13:18.484897 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:18.890136 (+405239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:18.890169 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:18.982041 (+ 91872us) log.cc:844] Serialized 26 byte log entry
0709 13:13:19.089312 (+107271us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:19.551344 (+462032us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.551711 (+   367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.551787 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.551828 (+    41us) raft_consensus.cc:1685] finished
0709 13:13:19.551838 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.551994 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181878,"prepare.queue_time_us":215697,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":8749,"raft.queue_time_us":45850,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":14607,"thread_start_us":710546,"threads_started":3,"wal-append.queue_time_us":313830}
I20260709 13:13:19.633306 28154 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:19.619735  1099 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:19.531086  1087 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:19.622931 28086 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:19.626452 28152 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:19.633728 28154 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:19.642355 28152 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:19.578150 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:19.645618 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 87982us
I20260709 13:13:19.623713   800 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:19.627914   885 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.639832   913 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:19.640976   863 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.657949   863 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.655856   800 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.661167   863 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:19.651336  1032 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:19.663256  1032 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.665513 28755 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:19.665525  1032 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:19.666787 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 888) took 2077 ms. Trace:
I20260709 13:13:19.667191 28755 rpcz_store.cc:276] 0709 13:13:17.589792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.589949 (+   157us) service_pool.cc:224] Handling call
0709 13:13:17.590289 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:19.665950 (+2075661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.666498 (+   548us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b5400686fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:19.666715 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4772031360}
I20260709 13:13:19.668983  1029 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:19.671687  1032 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.672245  1032 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.674823  1032 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:19.678311 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2695) took 1577 ms. Trace:
I20260709 13:13:19.678833 28232 rpcz_store.cc:276] 0709 13:13:18.100713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.101550 (+   837us) service_pool.cc:224] Handling call
0709 13:13:18.101975 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.509145 (+407170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.678002 (+1168857us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b5400288054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:19.678238 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906632448}
I20260709 13:13:19.679545 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:19.681843 28229 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:13:19.685163 28229 pending_rounds.cc:85] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 7
I20260709 13:13:19.683836 28231 raft_consensus.cc:1217] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Deduplicated request from leader. Original: 14.13->[16.14-16.14]   Dedup: 16.14->[]
I20260709 13:13:19.686055 28231 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
W20260709 13:13:19.688901 28155 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2630) took 2771 ms (client timeout 1536 ms). Trace:
W20260709 13:13:19.689482 28155 rpcz_store.cc:269] 0709 13:13:16.918433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.919241 (+   808us) service_pool.cc:224] Handling call
0709 13:13:19.688598 (+2769357us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b540010ac54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:19.688802 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1050150272}
I20260709 13:13:19.691082 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:19.691898 28155 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:19.695593 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:19.685493 28229 pending_rounds.cc:107] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 11.8
I20260709 13:13:19.712705 28229 raft_consensus.cc:2917] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Aborting config change with OpId 11.8: Aborted: Op aborted by new leader
I20260709 13:13:19.713555 28229 raft_consensus.cc:2889] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:19.680941  1093 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:19.624759  1058 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.708184   829 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:19.718166 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:41728 (request call id 0) took 9348 ms. Trace:
I20260709 13:13:19.702976 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:19.726797 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.733003 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:19.714910   885 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:19.654047   946 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.729628 28229 rpcz_store.cc:276] 0709 13:13:10.365443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.365615 (+   172us) service_pool.cc:224] Handling call
0709 13:13:10.402349 (+ 36734us) log.cc:844] Serialized 506 byte log entry
0709 13:13:10.901851 (+499502us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b180031d4a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f7b26 00007f59ce5f149a 00007f59ce5f0714
0709 13:13:11.487348 (+585497us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b54000b43a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f0142 00007f59ce5d0eaa 00007f59ce5d7d53 00007f59ce5d7cd9 00007f59ce5d7c69 00007f59ce5d7c31 00007f59ce5d6dfd 000000000038f354 000000000038f199 00007f59cb58a475
0709 13:13:12.142207 (+654859us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b54000b43a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f7b26 00007f59ce5f149a 00007f59ce5f0714 00007f59ce5d0eaa 00007f59ce5d7d53 00007f59ce5d7cd9 00007f59ce5d7c69 00007f59ce5d7c31 00007f59ce5d6dfd 000000000038f354
0709 13:13:19.714184 (+7571977us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38331,"raft.queue_time_us":136990,"raft.run_cpu_time_us":19235,"raft.run_wall_time_us":2741820,"spinlock_wait_cycles":97221760,"thread_start_us":162200,"threads_started":2,"wal-append.queue_time_us":208713,"wal-append.run_cpu_time_us":2090,"wal-append.run_wall_time_us":1016361}
I20260709 13:13:19.715142   863 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:13:19.739748  1042 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:19.740698   792 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:19.720016  1095 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:19.777114  1042 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:19.794056  1042 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:19.799760  1101 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.056s	user 0.001s	sys 0.001s
W20260709 13:13:19.862460 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:19.863677 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.743796   705 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.865258 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2659) took 1883 ms. Trace:
I20260709 13:13:19.869086   705 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:19.867908 28158 rpcz_store.cc:276] 0709 13:13:17.981888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.501142 (+519254us) service_pool.cc:224] Handling call
0709 13:13:18.503453 (+  2311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:18.913398 (+409945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:18.913431 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:18.913908 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:13:19.520260 (+606352us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:19.562056 (+ 41796us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.562345 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.562463 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.864492 (+302029us) raft_consensus.cc:1685] finished
0709 13:13:19.864562 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.864978 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1715,"prepare.queue_time_us":46271,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":187325,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":10755,"thread_start_us":888379,"threads_started":2,"wal-append.queue_time_us":723037}
I20260709 13:13:19.873273 28158 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:13:19.877487 28764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 1135) took 2020 ms (client timeout 1575 ms). Trace:
W20260709 13:13:19.878183 28764 rpcz_store.cc:269] 0709 13:13:17.856911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.857046 (+   135us) service_pool.cc:224] Handling call
0709 13:13:19.876779 (+2019733us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54004d21d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:19.877135 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245013888}
W20260709 13:13:19.879868 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.875573 28154 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.889971 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:19.892254 28082 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: [15.13-15.13]
W20260709 13:13:19.892630 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.895663   773 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:19.896183   773 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.896641   773 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:19.898916 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 888e64754b8d47aebb2c7c3aba7b0ee2: Aborted: Op aborted by new leader
W20260709 13:13:19.892395 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 212826us
I20260709 13:13:19.909060 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:19.909454 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:19.910830 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.911675 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:19.914211 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 991) took 3745 ms. Trace:
I20260709 13:13:19.914790 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:19.915221 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:19.916306 28086 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:19.916615 28082 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:19.917531 28232 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:19.919821   524 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:19.914686 28233 rpcz_store.cc:276] 0709 13:13:16.168274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.169018 (+   744us) service_pool.cc:224] Handling call
0709 13:13:18.089005 (+1919987us) spinlock_profiling.cc:243] Waited 7.84 ms on lock 0x7b58001c0598. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:13:18.089256 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.509377 (+420121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.913483 (+1404106us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b5400305554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:19.913935 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968568448}
W20260709 13:13:19.920965 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:19.922070 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.922389 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:19.922700 28152 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:19.888629 28154 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:19.924777 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:19.912045 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:19.927392 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.890259 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:19.958570 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:19.960944 28085 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:19.971822 28083 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:19.970757 28230 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:19.972144 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.955619   524 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:19.988970 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.988529 28234 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:19.828513  1087 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:19.964673  1046 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.992506   524 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.008211 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:19.798544   946 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:19.999687   705 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:13:20.038304 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.036182 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:20.043562   455 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:20.043854 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.044567   455 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.046639 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:20.047888   455 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:20.049103 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:20.049793   797 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:20.050393   797 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:20.053785   805 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:20.054590   312 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.056305 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2655) took 2193 ms. Trace:
I20260709 13:13:20.056493   949 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.057549   949 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.063913 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:20.065366 28755 raft_consensus.cc:2410] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:13:20.018921  1046 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.074678   705 consensus_queue.cc:1243] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:13:20.075763 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:20.055328   312 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.072156 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:20.077705 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:20.065474   792 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:20.072232   800 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:20.084478 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:20.043027   971 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.001s
I20260709 13:13:20.100716 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.079861   844 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:13:20.089505 28082 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:20.090850   846 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:20.092228 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:20.092691 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:20.094781 28759 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:20.052165   797 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.064586   913 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:20.095530 28158 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:20.077636  1046 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:20.109627 28234 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:20.056883 28159 rpcz_store.cc:276] 0709 13:13:17.811866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.435918 (+624052us) service_pool.cc:224] Handling call
0709 13:13:18.488329 (+ 52411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:18.491337 (+  3008us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:19.509768 (+1018431us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.581158 (+ 71390us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b480024c258. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:13:19.904918 (+323760us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:19.905764 (+   846us) log.cc:844] Serialized 26 byte log entry
0709 13:13:19.906013 (+   249us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:19.906471 (+   458us) log.cc:844] Serialized 14 byte log entry
0709 13:13:19.906681 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.906816 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.004928 (+ 98112us) raft_consensus.cc:1685] finished
0709 13:13:20.004972 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.005179 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":948779,"prepare.queue_time_us":857618,"prepare.run_cpu_time_us":170,"prepare.run_wall_time_us":9195,"raft.queue_time_us":224554,"raft.run_cpu_time_us":18006,"raft.run_wall_time_us":1046997,"spinlock_wait_cycles":4503296,"thread_start_us":373977,"threads_started":2,"wal-append.queue_time_us":243501}
I20260709 13:13:20.116819 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2696) took 2015 ms. Trace:
I20260709 13:13:20.117420 28226 rpcz_store.cc:276] 0709 13:13:18.101005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.102768 (+  1763us) service_pool.cc:224] Handling call
0709 13:13:18.103121 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.394848 (+291727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.116087 (+1721239us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54003b3654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:20.116535 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626114432}
I20260709 13:13:20.122381 28763 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:20.126310   971 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:20.127012  1085 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:20.127323 28235 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:20.127581  1085 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:20.128830   797 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:20.129705  1091 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:20.132447 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2224) took 1809 ms. Trace:
I20260709 13:13:20.135203 28229 rpcz_store.cc:276] 0709 13:13:18.322394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.198740 (+876346us) service_pool.cc:224] Handling call
0709 13:13:19.199180 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:19.872390 (+673210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.131964 (+259574us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b54000e8654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:20.132242 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108791680}
I20260709 13:13:20.142500 28232 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:13:20.143913 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:20.147830 28755 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:20.145291 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:20.121163   455 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:20.148643 28154 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:13:20.163332 28152 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:20.165154 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 113227us
I20260709 13:13:20.166241 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:20.167310   805 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:20.169670  1077 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:13:20.174118 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:20.128875   312 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:20.176259 28154 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:20.185704  1085 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.187274 28229 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:20.201588 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:20.177414 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:20.202718 28232 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:20.202865 28235 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
W20260709 13:13:20.189142 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 331963us
I20260709 13:13:20.191017   971 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:20.193900 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:20.208701 28156 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:20.215425  1058 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:20.216835 28125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.003s	sys 0.006s
W20260709 13:13:20.217653 28125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.003s	sys 0.006s
W20260709 13:13:20.223528 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53486.2us
I20260709 13:13:20.205376 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:20.225296 28759 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 10 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:13:20.226446   966 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:20.206861   312 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:20.229609 28234 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
W20260709 13:13:20.231447 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 77879.8us
I20260709 13:13:20.199227  1032 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:20.232621 28764 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:20.233075  1032 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:20.233690 28156 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:20.234946 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:20.234607  1032 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.235805 28156 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:20.236083  1032 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:20.238440   966 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:20.239730   966 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:20.207310 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:20.208423 28158 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: [15.11-16.12]
I20260709 13:13:20.247260 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:20.211328   949 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:20.262849 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55927.4us
I20260709 13:13:20.264186 28156 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 15. Ops: [13.12-13.12]
I20260709 13:13:20.264070 32708 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:20.259697   312 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:13:20.268153 28156 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:20.279641 28159 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:20.289126 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 56455.9us
I20260709 13:13:20.266501 32708 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:20.298374 28158 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:20.295039 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:20.300066  1043 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:20.301926  1087 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:20.317840  1116 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:20.319001  1116 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.076354 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:20.300644  1043 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:20.332438 28050 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:20.333218   312 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.334126   312 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.336288   312 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:20.323076  1116 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:20.337426 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 74056us
I20260709 13:13:20.340072   966 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:20.342876  1061 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:20.343081   455 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:20.344242 28230 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:20.344722  1047 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:20.345183  1047 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:20.345923   773 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.346239  1047 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.347703  1047 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.349879  1043 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.351295  1043 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.352284 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:20.353113   872 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:20.353235 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2660) took 2370 ms. Trace:
I20260709 13:13:20.353921   872 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.338747 28159 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:20.304688   524 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.356690   773 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:20.357445 32708 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.353883 28161 rpcz_store.cc:276] 0709 13:13:17.982163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.206410 (+1224247us) service_pool.cc:224] Handling call
0709 13:13:19.206768 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.207085 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:19.207112 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:19.207573 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:13:19.396001 (+188428us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:19.417831 (+ 21830us) log.cc:844] Serialized 14 byte log entry
0709 13:13:20.143131 (+725300us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b180049c1e0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:13:20.299737 (+156606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.321398 (+ 21661us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.327252 (+  5854us) raft_consensus.cc:1685] finished
0709 13:13:20.339632 (+ 12380us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.352999 (+ 13367us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21195,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":50493,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":140,"spinlock_wait_cycles":5528960,"thread_start_us":330488,"threads_started":2,"wal-append.queue_time_us":343631}
I20260709 13:13:20.359318 32708 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:20.360996   829 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:13:20.369182 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:20.370056 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:20.375526  1120 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:13:20.377210  1046 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:13:20.379230 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.379238  1029 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:20.389159 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:20.389782  1127 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:13:20.391865 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 129525us
I20260709 13:13:20.387670   801 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:13:20.398700 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66453.5us
I20260709 13:13:20.399722 28156 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:20.382733  1117 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
W20260709 13:13:20.403142  1125 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.002s
W20260709 13:13:20.406471  1119 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:13:20.407410 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.404325 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.412194 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.420104   872 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:20.420162 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2221) took 2116 ms. Trace:
W20260709 13:13:20.421042  1121 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.167s	user 0.000s	sys 0.002s
I20260709 13:13:20.421393 28231 rpcz_store.cc:276] 0709 13:13:18.303582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.941009 (+637427us) service_pool.cc:224] Handling call
0709 13:13:19.541621 (+600612us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.694162 (+152541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.217430 (+523268us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400287154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:20.419699 (+202269us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b5400287150. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:20.419955 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350018560}
W20260709 13:13:20.427711 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:20.410724 28127 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.723s (SENT) [suppressed 26 similar messages]
I20260709 13:13:20.425972  1085 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:20.428453   863 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:20.431394 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.432703 28085 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:20.417260   801 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260709 13:13:20.437306  1122 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.129s	user 0.002s	sys 0.000s
W20260709 13:13:20.439821 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 67725.3us
I20260709 13:13:20.439759   872 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:20.441269 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:20.441733   312 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:20.442395   312 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:20.443924 28083 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:13:20.443960 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:20.444871 28155 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:20.445158  1046 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:20.445919   863 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:20.447593  1046 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:20.443711   312 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.452028   312 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:20.453420 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:20.454195  1061 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:20.454697   792 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:20.455461   792 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.455935   792 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:20.456629  1128 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.098s	user 0.001s	sys 0.001s
W20260709 13:13:20.458093  1123 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.205s	user 0.002s	sys 0.002s
I20260709 13:13:20.467717   524 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:20.468397 28083 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:20.469921 28078 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: [10.11-12.12]
W20260709 13:13:20.471786 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.472235 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.473448 28082 raft_consensus.cc:2410] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:13:20.472994 27980 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 15, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 15 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:20.475093 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.475808 27980 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.477236 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.478929 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.490459 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:20.481169 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.491372 28160 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:20.491663 28083 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:20.488008 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:20.488111 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.492863 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:20.493903  1061 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:13:20.494112 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66279.3us
I20260709 13:13:20.495410 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.490572 28082 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:20.486652 28157 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:20.496917   966 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:13:20.480211 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.493134 28080 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:20.501214 28085 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:20.496634 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:20.504972  1061 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:20.506103 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:20.506856 28233 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:20.492342 27980 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 12, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 12 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:20.489825 27982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58554 (request call id 283) took 38268 ms (client timeout 15000 ms). Trace:
I20260709 13:13:20.454502 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:20.521064 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 66680.9us
W20260709 13:13:20.522353 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 87710.2us
W20260709 13:13:20.525172 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 129109us
I20260709 13:13:20.505595  1061 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:20.527616 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:20.530200 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:20.531037 28085 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:20.534771 28233 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:20.529131 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:20.540656 28155 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:20.546712 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 957) took 1157 ms. Trace:
I20260709 13:13:20.531665  1046 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:20.547297 28152 rpcz_store.cc:276] 0709 13:13:19.389171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.625967 (+236796us) service_pool.cc:224] Handling call
0709 13:13:19.626312 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:19.921509 (+295197us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b4c00159680. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cc943a20 00007f59cc954a29 00007f59cc91a644 00007f59cc941f2d 00007f59cc941b93 00007f59ce67218b 00007f59ce672cb5 00007f59ce66df30 00007f59ce66f7de 00007f59ce679b49 00007f59ce67a416
0709 13:13:20.163016 (+241507us) spinlock_profiling.cc:243] Waited 99.1 ms on lock 0x7b5400200ba0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5eca1d 00007f59ce5d31a4 00007f59ce655447 00007f59ce66dfe7 00007f59ce66f7de 00007f59ce679b49 00007f59ce67a416 00007f59ce676d8f 00007f59ce67683e 00007f59d363b419
0709 13:13:20.527654 (+364638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:20.527687 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:20.541554 (+ 13867us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:20.541606 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.546436 (+  4830us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.546646 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9803,"raft.queue_time_us":9874,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":12314,"spinlock_wait_cycles":346646016}
I20260709 13:13:20.550711 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:20.536238   524 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:20.562012 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.514873   792 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.570170 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:20.571446 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:20.587996 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 47600.5us
W20260709 13:13:20.601147 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.592300 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:20.597004  1134 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.072s	user 0.003s	sys 0.000s
I20260709 13:13:20.597275 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:20.603438 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.605211 28152 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:20.601907 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:20.606930 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:20.554842  1046 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.602064 27980 catalog_manager.cc:5697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 15. New cstate: current_term: 15 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.613986 28085 raft_consensus.cc:2410] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:13:20.606519   966 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:20.625566 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:20.623767 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:20.624488 28078 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:20.614080  1046 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.611024   846 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:20.633330 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:20.637833 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:20.638724 28082 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:20.639761 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47891.1us
W20260709 13:13:20.641201 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.637050 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:20.610517   524 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.648847 27980 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.656303 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:20.657387 28084 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.642472  1043 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:20.640969   801 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:20.670791 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 124934us
I20260709 13:13:20.661173   863 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:20.617048  1061 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.659364   773 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:20.683193 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:20.688323 27980 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 10, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 10 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:13:20.508949 27982 rpcz_store.cc:269] 0709 13:12:42.221665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.222031 (+   366us) service_pool.cc:224] Handling call
0709 13:13:20.489291 (+38267260us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:10.715250 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:10.776784 (+ 61534us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:10.776847 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:13:10.776869 (+    22us) tablet.cc:662] Decoding operations
0709 13:13:10.780812 (+  3943us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:10.780920 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:10.780954 (+    34us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:13:10.782176 (+  1222us) tablet.cc:701] Row locks acquired
0709 13:13:10.782235 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:13:10.782419 (+   184us) write_op.cc:270] Start()
0709 13:13:10.782578 (+   159us) write_op.cc:276] Timestamp: P: 1783602790782349 usec, L: 0
0709 13:13:10.782620 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:13:10.869541 (+ 86921us) log.cc:844] Serialized 5956 byte log entry
0709 13:13:11.729228 (+859687us) op_driver.cc:464] REPLICATION: finished
0709 13:13:12.171901 (+442673us) write_op.cc:301] APPLY: starting
0709 13:13:12.221218 (+ 49317us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:13.127108 (+905890us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b1800328720. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce3b60ec 00007f59ce402c0a 00007f59ce4024f7 00007f59ce402490 00007f59ce3ad754 00007f59d20cdf13 00007f59d20ce532 00007f59d2115b93 00007f59d1fc2de3
0709 13:13:13.449107 (+321999us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:13.449158 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:13.701396 (+252238us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:13.701908 (+   512us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 13:13:13.701973 (+    65us) write_op.cc:312] APPLY: finished
0709 13:13:13.756978 (+ 55005us) log.cc:844] Serialized 139 byte log entry
0709 13:13:13.962231 (+205253us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:13.962405 (+   174us) write_op.cc:454] Released schema lock
0709 13:13:13.962482 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:13.965717 (+  3235us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:13.965751 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":277752,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":333046,"apply.run_cpu_time_us":52172,"apply.run_wall_time_us":1828268,"cfile_cache_hit":62,"cfile_cache_hit_bytes":67037,"num_ops":15,"prepare.queue_time_us":201017,"prepare.run_cpu_time_us":21791,"prepare.run_wall_time_us":427041,"raft.queue_time_us":277759,"raft.run_cpu_time_us":46178,"raft.run_wall_time_us":280177,"replication_time_us":897886,"spinlock_wait_cycles":6573312,"thread_start_us":783070,"threads_started":5,"wal-append.queue_time_us":495948,"wal-append.run_cpu_time_us":76681,"wal-append.run_wall_time_us":2689497}]]}
W20260709 13:13:20.694486 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 88137.3us
I20260709 13:13:20.677430   863 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.697391 28759 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:13:20.675343  1043 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:20.699901 28155 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:20.699710 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:20.701725 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:20.685663  1029 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.712610 27980 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 10 to 15, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 15 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:20.699499   863 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.721279   829 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:20.741578 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.743330 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:20.747638 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48082.1us
I20260709 13:13:20.731433   792 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:20.724706  1029 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.729228   872 consensus_queue.cc:1059] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:13:20.747732  1058 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:20.770751 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 56184us
W20260709 13:13:20.773485  1130 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:13:20.777482 27980 catalog_manager.cc:5697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 4 to 11, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 11 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:13:20.788476 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.791306 27980 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.767527   829 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
W20260709 13:13:20.798249 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 99914.9us
I20260709 13:13:20.780567   524 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:20.769995  1043 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.786899   966 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:20.805059   966 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:20.805429  1131 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:13:20.791922  1058 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.787258   846 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:20.810457  1058 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:20.810806   846 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.810998  1058 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:20.811561  1058 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.813752  1029 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:20.815026  1029 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:13:20.818743  1135 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:13:20.823896   312 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:20.826308   312 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:20.827546   312 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.825362   455 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:13:20.833396 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49878us
I20260709 13:13:20.834431 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:20.836715   524 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:20.838357 28082 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:20.838546  1116 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.845995  1058 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:13:20.846395  1116 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:20.844348 28050 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:20.842383   846 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:20.847523 28084 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:20.831569   801 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:20.849583 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.849149 27980 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:20.849143 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 66312.7us
I20260709 13:13:20.855358 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:20.860368 27980 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 9 to 14, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 14 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.873709 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:20.874657 28763 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:20.874837   846 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:20.859656  1116 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:20.876710  1116 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.877143  1116 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:20.876224   846 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:20.881867 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:20.883028 28763 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:20.865927   792 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:13:20.892555 28755 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:20.892928   524 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:20.895408 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:20.895777 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:20.896145 28755 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:20.896226 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:20.897039 28231 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.897090   524 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:20.895432  1099 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:13:20.899825  1099 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:20.897908   524 consensus_queue.cc:1243] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:20.903537 28759 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:20.908013 28755 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:20.908250 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:20.909409 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:20.910671   524 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:20.914767   524 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.915359   524 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:20.878079 27980 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 10 to 13, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 13 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:20.924304 28763 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:20.961803 28759 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:20.963518 28079 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:20.966965 28152 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 17. Ops: []
W20260709 13:13:20.967509 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 48025.7us
I20260709 13:13:20.970826 27980 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:20.974589 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54272.4us
I20260709 13:13:20.976557 27980 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.984663 28157 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:13:20.985358 28157 pending_rounds.cc:85] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 7
I20260709 13:13:20.917374   863 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:20.996688 28152 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:13:20.993782 28157 pending_rounds.cc:107] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:13:21.002403  1042 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.002434 28157 raft_consensus.cc:2889] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:21.003187  1042 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.007295 28231 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:21.007563  1042 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:21.014883 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 226804us
I20260709 13:13:21.014808 27980 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 10 to 12, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 12 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:21.020172 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:20.990880  1099 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.020938 28759 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:21.000247   773 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:20.980517  1046 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.022511 27980 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:21.027971 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.032042 27980 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 9, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 9 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:13:21.037593 28764 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 13:13:21.045373 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:21.060693 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 274783us
I20260709 13:13:21.041507   773 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:21.077522 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 233252us
I20260709 13:13:21.078784 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:21.079440 28160 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:21.035768   846 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:21.063838 28198 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:21.088514 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:21.089828 28154 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:13:21.029615   966 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:21.096071 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 109299us
I20260709 13:13:21.097810 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:21.100467   913 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.101118   913 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.103428   913 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:21.113015 28198 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:21.071255  1058 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:21.044754  1099 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.103930   913 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.107585   792 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.136382 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:21.136695 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:21.139918 28152 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
W20260709 13:13:21.156306 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.161247 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:21.162101 28154 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:21.164884 28156 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:21.166767 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:21.182046 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:21.135685  1152 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:21.176422   863 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:21.232949 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:21.234349 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.237289 28152 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:21.237355 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:21.181984   773 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:21.240012 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 142244us
I20260709 13:13:21.244230 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:21.243629 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:21.245425 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65368.7us
I20260709 13:13:21.248497 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:21.164763  1058 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.250528 28759 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:21.192188  1152 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:21.258407  1152 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:21.182663   792 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.262507 28154 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:13:21.256582 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:21.263062 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.189007   913 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:21.263247 28154 pending_rounds.cc:85] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 11
I20260709 13:13:21.270556 28154 pending_rounds.cc:107] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:13:21.270924 28154 raft_consensus.cc:2889] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:21.276278  1144 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:13:21.283393  1162 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:13:21.290001  1058 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:21.290666  1058 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:21.291038  1058 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:21.291416  1058 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:21.292514  1047 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:21.292982  1047 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:21.260927  1046 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:21.293632   773 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:21.294195   773 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:21.293900  1047 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.283664   312 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:21.295214   773 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.251855   846 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:21.297219   966 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:21.300432 28160 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
W20260709 13:13:21.315130  1142 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:13:21.317262  1047 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:21.322731 28084 raft_consensus.cc:2410] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:13:21.325134 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.327390 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.328203 28079 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:21.328639 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:21.328949   872 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.329094 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:21.329416 28081 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:21.329694   872 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.385344   792 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:21.387224  1099 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:21.388366  1145 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:13:21.393146 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:21.393608 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:21.393989  1116 consensus_meta.cc:287] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.000s
I20260709 13:13:21.394479  1116 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.397176 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.397367 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.398849   913 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:21.399662   872 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:21.399964 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.398782 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.404821 28084 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:13:21.405295 28084 pending_rounds.cc:85] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 11
I20260709 13:13:21.405552 28084 pending_rounds.cc:107] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:13:21.405813 28084 raft_consensus.cc:2889] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:21.407509 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 151103us
I20260709 13:13:21.398736 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:21.430713 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 324483us
I20260709 13:13:21.431456 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:21.378809  1150 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.442710 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.442869  1150 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:21.444285 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.445006 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.448266 28077 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:21.445731 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:21.449429 28084 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:21.450623 28085 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:21.453166 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.454237 28077 raft_consensus.cc:2410] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:13:21.440260 28083 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:13:21.473903 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:21.473054 28198 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:21.418745  1116 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:21.481071 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:21.436405 28079 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:21.441439  1160 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:13:21.514605 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 82794.6us
I20260709 13:13:21.518527 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:21.519930 28084 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:21.535307 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:21.404642  1152 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.472558  1161 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:21.542253  1178 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:13:21.539356   455 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:21.546248   455 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:21.498195  1150 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:21.513163  1099 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:21.536669   913 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:21.549296 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:21.512749  1043 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:21.555559   455 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:21.555821 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:21.561996 28086 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:21.559139 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:21.558171 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:21.564255 28081 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
W20260709 13:13:21.562826 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 77579.4us
I20260709 13:13:21.547986 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:21.574295 28760 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:21.498857   792 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:13:21.590764 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:21.593112  1173 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.593842  1173 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.596861  1099 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:21.561748  1150 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:21.597556  1150 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:21.598031  1099 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.598876  1150 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.579931   913 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:21.602183   913 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:21.553761 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1445924259999174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 13:13:21.563359  1043 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:21.610363 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1355924259999028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
I20260709 13:13:21.611260 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:21.632404 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:21.634719 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:21.637419 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:21.638119 28230 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:21.630375 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1229924260001098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
I20260709 13:13:21.596534  1173 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:21.631496  1043 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:21.612099  1150 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:21.590829  1182 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:21.681317 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:21.637614  1165 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:21.675130  1042 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:13:21.737254 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 102619us
W20260709 13:13:21.737920 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 92025.5us
I20260709 13:13:21.708314  1153 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:21.675464   872 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:21.740643   524 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.768s	user 0.016s	sys 0.000s
I20260709 13:13:21.608614  1058 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.734815   455 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:21.794535 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 259866us
I20260709 13:13:21.781917 28226 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:21.782699 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:21.736644  1173 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:21.704069  1159 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:21.743418 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.801677   312 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:21.803309   312 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:21.780946 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.805604 28081 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:21.710757  1161 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:21.778067  1085 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:21.817420 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 306924us
I20260709 13:13:21.794227  1182 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:21.770767  1187 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:21.782132  1061 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:13:21.887524 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 104933us
I20260709 13:13:21.888691   312 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.882341 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:21.893955 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 88790.1us
I20260709 13:13:21.901401 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2672) took 1005 ms. Trace:
I20260709 13:13:21.902159 28155 rpcz_store.cc:276] 0709 13:13:20.895879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.919592 (+ 23713us) service_pool.cc:224] Handling call
0709 13:13:20.967240 (+ 47648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:20.990679 (+ 23439us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:21.135680 (+145001us) log.cc:844] Serialized 14 byte log entry
0709 13:13:21.643074 (+507394us) log.cc:844] Serialized 14 byte log entry
0709 13:13:21.900648 (+257574us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:21.900789 (+   141us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:21.900913 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.901070 (+   157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.901320 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18990,"prepare.queue_time_us":265026,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":101,"raft.queue_time_us":247975,"spinlock_wait_cycles":82399744,"thread_start_us":798886,"threads_started":4,"wal-append.queue_time_us":497785}
I20260709 13:13:21.804494  1156 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:13:21.869679  1182 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:21.780459  1163 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.410s
I20260709 13:13:21.909864   312 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:21.907189 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 266910us
I20260709 13:13:21.913375 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:21.915936 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:21.811240  1161 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:21.802824   524 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:21.918673 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49704.8us
I20260709 13:13:21.909509 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:21.935292 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:21.899115  1175 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:21.811228  1173 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:21.948041 28125 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:21.852802   872 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.805277  1153 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:21.891269   455 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:21.962440 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 49151us
I20260709 13:13:21.964367 28153 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 17. Ops: []
I20260709 13:13:21.966413 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:21.969914 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:21.973114 28759 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:13:21.927533  1061 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:21.904623  1085 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.946573  1152 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:21.971436 28760 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:21.948689  1175 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.952497  1165 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:21.979614 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:21.962984 28228 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
W20260709 13:13:22.043021 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64843.8us
I20260709 13:13:21.901551  1159 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.981740 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:21.989078   524 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:21.893098  1187 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:21.991585 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:21.991940  1182 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:13:21.992717  1188 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:13:21.993309  1175 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:22.061728 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:21.994086 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.062336 28764 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:21.994724  1204 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:21.999778 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:22.009574  1206 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:13:22.065102  1182 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.020828  1177 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.116s
W20260709 13:13:22.029460  1191 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.319s	user 0.000s	sys 0.000s
I20260709 13:13:22.029821  1149 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:22.030699  1186 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
W20260709 13:13:22.034603  1047 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.717s	user 0.012s	sys 0.000s
I20260709 13:13:22.036031  1058 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.037772 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.039247 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:22.041772  1194 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:13:22.056079 28231 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:22.056371 28230 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:22.058451 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.107695 28084 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:22.059422  1181 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:22.060076 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.060305 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:22.060758 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.018088 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.074107 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 972) took 1024 ms. Trace:
W20260709 13:13:22.113690 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 53451.3us
I20260709 13:13:22.074587 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.077128  1204 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:22.082770   455 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:22.089421  1205 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:13:22.069573  1165 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:22.093581 28229 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:22.126425 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:22.141710 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:21.974609  1160 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:22.069237  1182 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:22.113742 28228 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:22.114018 28159 rpcz_store.cc:276] 0709 13:13:21.049599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.049750 (+   151us) service_pool.cc:224] Handling call
0709 13:13:21.050178 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.470986 (+420808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:21.471037 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:21.518824 (+ 47787us) log.cc:844] Serialized 26 byte log entry
0709 13:13:21.685061 (+166237us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:21.685104 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.685187 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.073739 (+388552us) raft_consensus.cc:1685] finished
0709 13:13:22.073898 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.074047 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4083,"raft.queue_time_us":202354,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"thread_start_us":445020,"threads_started":2,"wal-append.queue_time_us":268340}
I20260709 13:13:22.119695 28226 raft_consensus.cc:2410] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:13:22.114533 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:22.159493 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45051.6us
I20260709 13:13:22.114148 28227 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:22.089661 28764 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:22.124305 28233 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
W20260709 13:13:22.118934   773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.001s	sys 0.012s
I20260709 13:13:22.132961  1061 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:22.134337  1215 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:22.178030 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:22.120416  1152 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:22.179394  1152 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.124370  1149 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.144695  1204 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:22.145970  1165 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.183701  1149 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.149551 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2681) took 1049 ms. Trace:
I20260709 13:13:22.184597 28152 rpcz_store.cc:276] 0709 13:13:21.105763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.256406 (+150643us) service_pool.cc:224] Handling call
0709 13:13:21.263244 (+  6838us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.483027 (+219783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:21.483076 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:21.534901 (+ 51825us) log.cc:844] Serialized 26 byte log entry
0709 13:13:21.752907 (+218006us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:21.790657 (+ 37750us) log.cc:844] Serialized 14 byte log entry
0709 13:13:21.791098 (+   441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.791205 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.149197 (+357992us) raft_consensus.cc:1685] finished
0709 13:13:22.149230 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.149449 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44262,"prepare.queue_time_us":41771,"prepare.run_cpu_time_us":3130,"prepare.run_wall_time_us":24888,"raft.queue_time_us":308143,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":24777,"thread_start_us":378937,"threads_started":2,"wal-append.queue_time_us":342754}
I20260709 13:13:22.165158 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 968) took 1117 ms. Trace:
I20260709 13:13:22.127113  1117 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:22.174528 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.189301 28233 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:22.175083 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.191864 28230 raft_consensus.cc:2410] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:13:22.193074 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:22.193426 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.176596 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.195636 28235 raft_consensus.cc:2410] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:13:22.196990 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:22.198297 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:22.167620   773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.001s	sys 0.013s
I20260709 13:13:22.208161 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:22.199172 28226 raft_consensus.cc:2410] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:13:22.188777 28153 rpcz_store.cc:276] 0709 13:13:21.047955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.048098 (+   143us) service_pool.cc:224] Handling call
0709 13:13:21.081084 (+ 32986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.971631 (+890547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.164783 (+193152us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b5400051c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:22.165061 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030752512}
I20260709 13:13:22.208578 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.208596 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.211189 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.211978 28228 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:22.213145 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:22.216464 28227 raft_consensus.cc:2410] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 14 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
W20260709 13:13:22.216493 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.217124 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:22.188247  1042 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:22.206223  1117 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:22.178375  1215 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:22.190320  1032 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:13:22.233173 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:22.209662  1149 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:22.221632 28231 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:22.222718 28085 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:22.259691 28086 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:13:22.247413 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.306856 28235 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:22.222455  1203 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:22.246668  1043 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.309934  1117 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.311751  1043 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:22.313494 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:22.314499  1171 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:22.314788   773 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:22.315104  1171 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:22.315500   773 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:22.316821  1179 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:22.317279  1179 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.317651  1179 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:22.317965   773 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:22.321414 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 66971.9us
I20260709 13:13:22.322057 28227 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:22.323004 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:22.239802 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:22.324427 28760 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:22.222011 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:22.281095 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:22.236943 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.325493 28233 raft_consensus.cc:2410] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:13:22.238648 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.322193 28232 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
W20260709 13:13:22.325996 28199 connection.cc:256] RPC call timeout handler was delayed by -1.8490907190000598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20650000000000002s for a response
I20260709 13:13:22.283325 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:22.228971  1195 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:22.330063 28226 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:22.336299 28152 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:22.338536 28154 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:22.338809 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:22.340410 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:22.224861  1201 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:22.346042  1201 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.346395 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.347630 28082 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:22.349030 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:22.233595  1160 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.352385 28234 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:22.359786 28234 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:22.360425  1173 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:22.360718 28234 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:22.362202 28234 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:22.355851 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:22.352939 28232 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:13:22.362962 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.363565 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:22.364238 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.359136 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:22.365310 28078 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:22.365968 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:22.361833  1161 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.367298 28228 pending_rounds.cc:85] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 6
I20260709 13:13:22.367653 28228 pending_rounds.cc:107] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:13:22.342339  1175 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.231896  1187 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.305742  1042 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:22.369946  1085 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.370324  1042 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:22.371369  1218 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.059s	user 0.002s	sys 0.000s
I20260709 13:13:22.316233  1171 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.373306 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:22.356693 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:22.367918 28228 raft_consensus.cc:2889] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:22.222249  1213 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:22.369969 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:22.343793 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.237219  1148 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:22.379751 28085 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:13:22.371274  1175 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.375272 28153 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:22.391494 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:22.392527 28158 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:22.343315  1149 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:22.377135  1043 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.377414  1201 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.412062 28086 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
W20260709 13:13:22.408335 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 65520.2us
I20260709 13:13:22.418179 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.424322 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.427891 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.406404   913 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:22.385461   863 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:13:22.460598 28156 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:13:22.425661  1213 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.449009  1152 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.483139 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:22.449995  1175 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:22.488631 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:22.489753 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:22.492616 28156 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:22.492591 28231 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:22.494781 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 151982us
W20260709 13:13:22.496825 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.426611  1043 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.494308  1180 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:22.484369  1173 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:22.457175  1215 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:22.471273   913 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:22.516114 28234 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:13:22.561381 28234 pending_rounds.cc:85] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 7
I20260709 13:13:22.562913 28234 pending_rounds.cc:107] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:13:22.563373 28234 raft_consensus.cc:2889] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:22.563895 28231 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:22.566514 28227 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:13:22.567682 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45963.9us
W20260709 13:13:22.562346 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 113127us
I20260709 13:13:22.571942 28083 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:22.571002 28730 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:22.522913  1165 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:22.501171  1175 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:22.605470 28227 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:22.539767  1180 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:22.535506  1141 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:22.609166 28078 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:22.612169 28760 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:22.614329 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:22.576306  1215 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:22.619340  1159 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.612915  1187 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:22.620491  1187 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.621592  1156 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:22.622182  1156 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.622668  1187 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.623973  1213 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.624642 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:22.627224 28231 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:22.628168 28231 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:22.624941  1224 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:13:22.537302  1182 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.611562  1117 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:22.540117  1173 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.624523  1156 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:22.644100 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:22.532265  1149 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:22.649578 28231 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:22.675190 28156 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:22.680891 28231 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:22.683130  1180 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.673736  1117 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:22.685587  1117 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:22.693485  1227 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 13:13:22.697407 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2100907190001635s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2775s for a response
W20260709 13:13:22.697847 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2100907190001635s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2775s for a response
I20260709 13:13:22.688266 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:22.628700  1165 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.701586   846 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:22.704985 28231 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:13:22.705461 28231 pending_rounds.cc:85] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 9
I20260709 13:13:22.705725 28231 pending_rounds.cc:107] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:13:22.706001 28231 raft_consensus.cc:2889] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:22.714342 28127 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.904s (SENT)
I20260709 13:13:22.715337 28127 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:22.716498  1187 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:22.718252 28127 leader_election.cc:336] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.056s (SENT)
I20260709 13:13:22.719039 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:22.653482  1061 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 13:13:22.727578  1232 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:13:22.699601  1152 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.733620   312 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:22.653249  1179 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.642378  1141 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:22.699146  1177 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:22.712258  1047 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.089s
W20260709 13:13:22.799325 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 34353a7e5a8346119cf9f0836165a1b7: Illegal state: Replica 11f10c12c03e47dcac88b050d4c1431b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
W20260709 13:13:22.802341 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.799757 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:22.803406 28759 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
W20260709 13:13:22.808940 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.800943   312 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:13:22.810977 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 92792.7us
I20260709 13:13:22.795694  1099 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:22.839993 28127 leader_election.cc:336] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.146s (SENT)
I20260709 13:13:22.804337  1043 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:22.842890   913 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:22.845270  1226 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.230s	user 0.002s	sys 0.000s
I20260709 13:13:22.846170 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:22.857931   913 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:13:22.860889 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet 8d3461aa17604e69b08928a9bc0f8d18: Illegal state: Replica 11f10c12c03e47dcac88b050d4c1431b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:22.813192   455 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:22.797822  1152 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:22.915138  1244 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:13:22.923084 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:22.901479 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59160.1us
I20260709 13:13:22.788587 28080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:22.964850 28080 raft_consensus.cc:606] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:22.908701  1175 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:22.974457  1187 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:22.996922  1187 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:22.983201 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 271311us
I20260709 13:13:23.039986 28081 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
W20260709 13:13:23.051251 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.898342  1160 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:23.042809  1219 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.731s	user 0.005s	sys 0.003s
I20260709 13:13:23.043224  1251 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:23.052055 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:22.974043   455 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:23.052827 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 241174us
W20260709 13:13:23.047160  1249 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.133s	user 0.001s	sys 0.009s
I20260709 13:13:23.065464 28081 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:23.065784  1152 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.066485  1152 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:23.068199  1141 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:23.057408   455 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:23.069025  1165 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:23.072089  1153 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:23.079443 28127 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.177s (SENT)
I20260709 13:13:23.080008 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:23.055148  1251 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:23.083544  1251 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:23.086520  1241 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
W20260709 13:13:23.093097 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 47647.2us
I20260709 13:13:23.099727 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.100286 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.100440 28230 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:23.100476 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.091192  1085 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:23.102344 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.092590  1251 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:23.105500 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.107686 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.098436  1179 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:23.111562 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59596.3us
W20260709 13:13:23.125447 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 68554us
W20260709 13:13:23.121981  1235 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:13:23.124138 28077 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:13:23.129935 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.132218 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:23.137262 28077 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:13:23.140866 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:23.142180 28764 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:23.125202   455 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.140592   913 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:23.152103  1173 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.153149  1258 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:23.154053  1187 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.160234 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.165305 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:23.167120 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:23.156287  1195 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.810s	user 0.008s	sys 0.003s
I20260709 13:13:23.168881 28127 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:23.168283 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:23.150029   913 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:23.175151  1234 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:23.176121 28764 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:13:23.177017 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:23.175719   455 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:23.180428 28230 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:23.180619  1152 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:23.180895 28230 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:23.181181  1152 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:23.181700  1152 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:23.182181  1182 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.181820 28230 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.182947 28230 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:23.185478 28233 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:23.186614 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.187288 28759 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 10.
I20260709 13:13:23.188055  1085 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:23.188473  1085 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:23.188915  1085 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:23.189814  1029 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:23.190220  1029 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:23.191131  1029 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.192814  1029 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:23.193280 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.193928 28235 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
I20260709 13:13:23.193279  1180 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.191092 28230 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:13:23.197422 28230 pending_rounds.cc:85] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
I20260709 13:13:23.197727 28230 pending_rounds.cc:107] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:13:23.198005 28230 raft_consensus.cc:2889] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:23.195885 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2750) took 1004 ms. Trace:
I20260709 13:13:23.198931 28226 rpcz_store.cc:276] 0709 13:13:22.191192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.395194 (+204002us) service_pool.cc:224] Handling call
0709 13:13:22.424399 (+ 29205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.599393 (+174994us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.599459 (+    66us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.599522 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:22.599572 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.195629 (+596057us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54000fb9d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:23.195679 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.195812 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401754368}
I20260709 13:13:23.200351 28198 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:23.199764 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:23.167676  1259 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:23.203701 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:23.204393 28757 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:23.177273  1195 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:23.178311   913 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:23.208560 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:23.212605 28155 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:23.195768  1213 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:23.195885  1234 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.209949 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:23.210994 28759 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:13:23.205891  1202 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:23.222308 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:23.223894 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:23.224902 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:23.227182  1150 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:23.196638  1173 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:13:23.228611 28155 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:23.229420 28152 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:23.210287   455 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:23.219237  1259 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:23.220206  1213 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:23.205699  1117 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:13:23.220890  1234 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:23.201958  1180 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.222143  1202 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.206360  1237 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
W20260709 13:13:23.245292 28127 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 3.576s (SENT)
I20260709 13:13:23.231648  1258 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:23.236482   455 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:23.251722 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:23.241526   872 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.247730  1213 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:23.251107  1202 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:23.255468   455 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:23.251317  1150 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:23.263345 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:23.252208  1258 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:23.268445  1160 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:23.269134  1201 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:23.259605  1195 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:23.270301 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:23.271279 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 468275us
I20260709 13:13:23.272004 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:23.268703 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:23.272760 28756 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:23.273052 28759 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:23.271873  1182 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:23.274653   312 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:23.274964 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.275691 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.276775   312 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.279189   312 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.282729 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:23.283099 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 231610us
I20260709 13:13:23.283383 28158 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:23.284384  1151 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:23.285166   966 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:23.285832 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:23.286686   966 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:23.287256   966 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:23.287638   966 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:23.288842 28757 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:23.289753 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:23.293401 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2251) took 1033 ms. Trace:
I20260709 13:13:23.296141 28159 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:23.297834 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.298501 28757 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:23.299517  1046 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.301226  1199 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:23.274474  1258 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:23.306879  1168 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:23.307442 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.300203  1148 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:23.308401  1148 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.310668  1182 consensus_queue.cc:1243] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:13:23.312247  1180 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:23.312853 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.312994  1156 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.313119 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:23.313978  1263 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:13:23.313769  1156 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.313879 28085 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:23.299935  1259 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.265578  1150 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:23.318181 28160 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:23.318451 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.326630 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:23.328786 28158 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:23.330118 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.330966 28234 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:23.335256 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.317637  1156 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.337561 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:23.338510 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49944.4us
I20260709 13:13:23.343361 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:23.343611 28761 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:23.344619 28761 raft_consensus.cc:606] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:23.347112 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:23.347968 28085 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:23.351361 28086 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:13:23.356863   863 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.357563   863 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:23.359932 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.360893 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:23.329749  1150 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:23.362607 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.363585 28757 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:23.310668  1148 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.301733  1046 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.372124 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:23.372879 28077 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:23.371604 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:23.380916 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 43502.1us
I20260709 13:13:23.381213 28756 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:23.384707 28085 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:23.385723   872 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:23.358454 28083 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:23.391745 28759 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:23.394134 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:23.343451  1061 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:13:23.339885  1162 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 13:13:23.398049 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64187.2us
I20260709 13:13:23.337191  1212 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:13:23.403719 28728 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.396997 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:23.405378 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:23.409102 28085 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:23.307452  1168 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.357295  1152 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.422137  1258 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.426183  1117 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:23.426924  1117 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:23.427366  1117 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:23.427577 28077 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:23.429013   863 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:23.429649 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.430393 28231 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:23.431989 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:23.397315  1156 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:23.432941   312 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:23.433282  1156 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.433423   312 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.433831  1156 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:23.293871 28078 rpcz_store.cc:276] 0709 13:13:22.259888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.639324 (+379436us) service_pool.cc:224] Handling call
0709 13:13:22.666766 (+ 27442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.666970 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:22.974055 (+307085us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.293047 (+318992us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.293107 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:23.293170 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.293235 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.293344 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":213470,"prepare.queue_time_us":59845,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":203479,"thread_start_us":386313,"threads_started":2,"wal-append.queue_time_us":129206}
I20260709 13:13:23.434036   312 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:23.426262  1168 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.436076 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.437536 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:23.416850   913 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.466760   455 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.467654   913 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:23.468840  1157 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:23.468585  1202 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.469146 28234 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:23.469314  1157 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.469836  1157 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:23.454187  1258 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
W20260709 13:13:23.473830 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47088.7us
I20260709 13:13:23.475245 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:23.475987 28226 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:23.477977 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:23.478379 28084 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:23.485816   312 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.493088 28758 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:23.496819 28085 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:23.420866  1267 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:23.500629   312 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.504190 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:23.504860 28230 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:23.507970 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:23.476990  1047 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:23.488977   913 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:23.484318  1202 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.525125 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:23.525597 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:23.439520  1085 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:23.529196 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 125511us
W20260709 13:13:23.536198 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.537739 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.513363   455 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:23.524641  1177 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:23.540547 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:23.542145 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:23.542829 28159 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:23.517810  1213 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.557020 28159 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:23.561405 28161 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:23.561457 28235 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:23.541450  1162 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 13:13:23.563326 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 88688.8us
I20260709 13:13:23.564395 28159 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.541246  1233 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:23.565650 28159 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:23.566303  1046 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:23.567129   966 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.564790 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.578330 28081 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:23.579550 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.568230  1156 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.584088  1251 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:23.584293 28080 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:13:23.570689  1157 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.587285  1157 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.581099   966 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.585522 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.585397  1156 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:23.580183   913 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:23.588778  1251 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:23.590148  1234 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.590626 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.591598  1251 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:23.592126  1251 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:23.593104  1251 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:23.593184 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 52750.7us
I20260709 13:13:23.592427  1234 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:23.556641 28230 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:23.606729 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:23.606947 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.609267 28230 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:23.609848  1157 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:23.608639 28159 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:23.613095   913 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.613698   913 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.615856 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:23.616025  1061 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:23.617156  1061 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:23.617596   966 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:23.617664  1061 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:23.615862   913 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:23.618902  1061 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.595475  1177 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:23.621454 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2252) took 1361 ms. Trace:
I20260709 13:13:23.621831 28235 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:23.618247  1152 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:23.598146   829 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:23.623121   829 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:23.621413  1177 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.623651 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 983) took 1166 ms. Trace:
I20260709 13:13:23.623976   829 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.627436 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.621922 28084 rpcz_store.cc:276] 0709 13:13:22.260125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.808659 (+548534us) service_pool.cc:224] Handling call
0709 13:13:22.809013 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.478967 (+669954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.621207 (+142240us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54003a98d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:23.621401 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1538946944}
I20260709 13:13:23.630388 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.624022 28160 rpcz_store.cc:276] 0709 13:13:22.456857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.603490 (+146633us) service_pool.cc:224] Handling call
0709 13:13:23.317877 (+714387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.327359 (+  9482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.623463 (+296104us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b5400238550. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:23.623588 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1542903680}
I20260709 13:13:23.634531 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.636078   846 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:23.637432  1152 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:23.637740   846 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.638072  1152 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:23.638240   846 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.628585   966 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:23.638882  1047 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:23.641990  1047 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:23.641821   966 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.643024  1047 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.643522   966 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:23.644582  1047 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:23.647737 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:13:23.648321 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.651381 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:23.635238 28230 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:23.655812 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:23.656723   312 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:23.657137 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:23.652029  1152 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:23.650110   846 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:23.668720 28758 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:13:23.669029 28759 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:13:23.669477  1187 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:13:23.628113  1150 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.672827  1150 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:23.651369   913 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:23.675364  1117 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.689853 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.696767 28757 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:23.698117   312 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:23.698170 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.699482   312 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.700145 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.702438 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.710681  1085 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:23.710903  1213 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:23.712590  1264 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.368s	user 0.001s	sys 0.001s
I20260709 13:13:23.715847  1161 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:23.692103   913 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:23.719574 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.722112  1161 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:13:23.702960  1150 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:13:23.740908 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.745489 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.751026 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.755266 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.759631 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.764256 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.765287 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:23.768003  1247 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.727s	user 0.004s	sys 0.000s
I20260709 13:13:23.768281 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.769177 28230 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:23.769903 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 988) took 1123 ms. Trace:
I20260709 13:13:23.770304 28157 rpcz_store.cc:276] 0709 13:13:22.646399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.648460 (+  2061us) service_pool.cc:224] Handling call
0709 13:13:22.648851 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.649213 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.649238 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.649670 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.798589 (+148919us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:22.807055 (+  8466us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.811739 (+  4684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.811793 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.663235 (+851442us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b18004917a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a522 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:13:23.764163 (+100928us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:23.769648 (+  5485us) raft_consensus.cc:1685] finished
0709 13:13:23.769671 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.769843 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":752,"prepare.queue_time_us":8223,"prepare.run_cpu_time_us":9341,"prepare.run_wall_time_us":27621,"raft.queue_time_us":5387,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":33075,"spinlock_wait_cycles":18866944,"thread_start_us":103019,"threads_started":1,"wal-append.queue_time_us":229540}
I20260709 13:13:23.782307 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:23.808401  1280 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.002s
I20260709 13:13:23.808825 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:23.813812 28080 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
W20260709 13:13:23.809104  1288 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.002s
W20260709 13:13:23.809577  1287 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:13:23.810277  1117 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:23.779480   829 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:13:23.834287  1213 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:23.834964  1213 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.836167 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:23.837612 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.838765 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:23.839838 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.840243  1085 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:23.824719 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:23.841450 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:23.842577  1085 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:13:23.843083  1182 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:13:23.843678  1182 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:23.844091  1160 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:23.844556  1182 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.846071  1117 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:23.846494  1117 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:23.825886 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:23.847394 28161 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:23.847731  1117 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:23.849103  1117 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:23.849800  1140 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:23.851094  1175 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:23.851466  1175 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:23.852356  1175 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.854002  1175 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:23.848981  1029 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:23.857384 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:23.829074 28231 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:23.867561 28161 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:23.868675  1152 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:23.869596  1152 consensus_queue.cc:1243] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:13:23.869656  1171 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:23.870157  1116 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:23.868683  1160 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:23.870962  1160 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:23.871392 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:23.869689 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.872102 28086 raft_consensus.cc:2368] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16.
I20260709 13:13:23.880487 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:23.881657 28077 raft_consensus.cc:2368] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16.
I20260709 13:13:23.881816 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:23.882265 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:23.882990 28227 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 11: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:23.886055 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:23.887771 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:23.888381 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:23.891767 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.844215  1085 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:13:23.889122 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:23.895095 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:23.902240 28233 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:23.898124 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.905790 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.895159 28158 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 11: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:23.898415  1126 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:23.906980 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:23.907599 28158 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:23.899379 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:23.909336 28232 raft_consensus.cc:2410] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:13:23.910079 28233 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:23.911258 28233 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:13:23.900599 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:23.911396 28232 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:23.896180  1187 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:23.902288 28084 raft_consensus.cc:2368] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16.
I20260709 13:13:23.914680  1126 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:23.915258  1126 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:23.908470 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:23.918068 28158 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:23.919212 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:23.920113  1237 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:23.855240  1058 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:23.924650 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:23.928365 28231 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:13:23.933473 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:23.937400 28080 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:23.944281 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:23.953579 28086 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:13:23.893237  1173 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.920650  1237 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:23.936012  1058 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:23.941897  1207 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:13:24.065780 28084 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:24.068953  1270 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.126s
W20260709 13:13:24.069751  1244 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:13:23.890967  1160 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.120406  1160 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:24.089268  1213 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:24.097244  1182 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:24.101260  1058 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.125324 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:24.126186  1058 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:24.129160 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:24.113718 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:24.125408  1182 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:24.116182 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:24.131620 28080 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:24.051628  1258 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:24.138337 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:24.143792 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:24.051293  1212 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:24.150040 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:24.161147 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:24.177768 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:24.180311 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:24.155489 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:24.164422 28078 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:24.183090 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:24.185546 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:24.191800 28078 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 13:13:24.195268 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:24.196250 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:24.148299   455 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:24.141626  1061 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.148952  1252 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:13:24.217700 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 278103us
I20260709 13:13:24.218786 28080 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:24.221098  1254 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:24.222954  1285 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.460s	user 0.003s	sys 0.000s
I20260709 13:13:24.231419 28050 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.226702 28199 leader_election.cc:400] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16.
W20260709 13:13:24.212536  1278 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:13:24.212996   792 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:13:24.233227 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 284886us
I20260709 13:13:24.151281  1258 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:24.134435  1173 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:24.261250 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:24.265336 28085 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:24.266607 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:24.268934 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.270916 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.217033  1150 raft_consensus.cc:2764] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:13:24.261875 28226 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:24.273290 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:24.290629 28153 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:24.257951  1140 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:24.259341   913 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:24.304776 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 89378.4us
I20260709 13:13:24.306171 28199 leader_election.cc:400] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16.
I20260709 13:13:24.307952  1149 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:24.308919  1149 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:24.317216 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46390.3us
I20260709 13:13:24.318423 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.285549  1213 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:24.260811   792 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:24.216596  1061 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.285988  1085 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:24.255683  1254 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.285218  1160 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:24.323448 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:24.297549 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 353345us
I20260709 13:13:24.296294  1140 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:24.327615   966 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:13:24.341360 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:24.341969  1258 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:24.257696  1237 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.319918  1199 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:24.331234  1182 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:24.358229 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.359954 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:24.324939  1175 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:13:24.316642   913 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.350387  1140 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:24.371609 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:24.357012  1258 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:24.374756  1254 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:24.375962  1300 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:13:24.378638   913 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:24.385334 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 61963.8us
I20260709 13:13:24.388372 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:24.389286 28155 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:24.396250  1085 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:24.406080 28199 leader_election.cc:400] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16.
I20260709 13:13:24.406469 28199 leader_election.cc:403] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:13:24.418875 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:24.388036  1171 consensus_meta.cc:287] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.005s
I20260709 13:13:24.418874 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:24.432621 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:24.403959  1254 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:13:24.440912 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54836us
I20260709 13:13:24.446164 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:24.447131 28156 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:24.448236 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 88339us
I20260709 13:13:24.450649 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:24.467502 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.370415   792 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.471058 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:24.478346 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.471218 28079 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:24.444720  1254 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:24.474025 28155 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:24.481389 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:24.485683 28155 raft_consensus.cc:2410] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:13:24.504297 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:24.507915 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:24.469983  1305 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:24.472644  1182 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:24.479823  1156 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.553867  1047 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:24.554462  1047 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:24.553997  1156 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.555495  1047 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:24.502930  1303 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:24.531086  1141 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:24.557682  1141 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.558542  1149 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:24.538084  1085 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:24.560844  1061 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:24.560106  1254 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:24.562664  1254 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:24.539749  1252 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:24.565193  1252 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:24.510870  1304 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:24.537209 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:24.566576 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:24.503358  1173 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:24.576174 28153 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:24.576351 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:24.545568  1308 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:13:24.577339 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:24.578884 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:24.590837 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.566746 28156 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
W20260709 13:13:24.599704 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 340952us
I20260709 13:13:24.512118  1165 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:13:24.588614  1171 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:24.609112  1165 consensus_queue.cc:1243] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:13:24.583595  1149 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:24.609637  1304 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:24.610183  1149 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:24.611575 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.581986  1303 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:24.613739   966 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:24.614544  1303 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:24.601209  1173 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:24.616410 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.611806  1149 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.618171   312 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:24.618893  1173 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:24.626498  1126 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:24.626499   312 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:24.623766 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.623351  1252 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.627689 28226 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:24.632313   312 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:24.638969 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.654528 28082 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:24.641158  1126 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:13:24.677536 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 387715us
I20260709 13:13:24.656951  1187 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:24.651324  1173 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:24.645956  1307 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:24.735298 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 290021us
W20260709 13:13:24.742808 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.786993 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:24.788028 28230 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:24.791015 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.795018 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:24.798772 28228 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:13:24.801632 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:24.807176 28227 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:24.807804 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 206920us
W20260709 13:13:24.808565 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:24.807351 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 143396us
I20260709 13:13:24.811143 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:24.806313 28232 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:24.815503 28234 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:24.818451 28161 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:24.821942 28154 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:24.833076 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:24.836777 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:24.837186  1116 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:24.834525 28728 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:24.843852 28079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2499) took 2496 ms (client timeout 1952 ms). Trace:
I20260709 13:13:24.816331   913 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16.
I20260709 13:13:24.845746 28755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:24.849453 28755 raft_consensus.cc:606] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.850814 28085 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:24.851045 28228 pending_rounds.cc:85] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 9
I20260709 13:13:24.851370 28228 pending_rounds.cc:107] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:13:24.851601 28228 raft_consensus.cc:2889] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:24.840701  1173 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:13:24.855047  1237 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:13:24.844296 28079 rpcz_store.cc:269] 0709 13:13:22.335088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.365692 (+ 30604us) service_pool.cc:224] Handling call
0709 13:13:24.831668 (+2465976us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b54002add54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:24.831793 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4740248832}
I20260709 13:13:24.844892  1304 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:24.787525  1307 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.858946  1304 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:24.849434   863 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:13:24.898047 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 83664.4us
I20260709 13:13:24.867468  1321 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:24.875305  1304 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:24.896009  1140 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.522s	user 0.007s	sys 0.006s
I20260709 13:13:24.884063  1252 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:24.910255 28229 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:24.911859  1151 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:24.916998  1307 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:24.917182  1318 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:13:24.916718  1151 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:24.919128  1151 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:24.919502   312 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:24.920799  1140 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:24.922907  1140 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:24.924410 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:24.925607 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:24.933876 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:24.932126 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:24.939666 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:24.905097   455 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:24.952839 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:24.958279 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 221592us
I20260709 13:13:24.980584 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:24.983088 28155 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:24.984452 28157 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:24.984634 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:24.978101  1126 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:13:24.988041 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:24.989444 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.989298 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:24.987691  1126 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:24.991465  1126 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:24.992393  1126 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:24.993772  1126 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:24.942126  1317 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:13:24.915050  1326 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:13:25.012629 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:24.952463  1150 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:25.015349   312 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:25.015867 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:24.941588  1141 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:25.021751 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 82157.6us
I20260709 13:13:24.949635  1187 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:25.019929 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:25.002764  1258 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:24.999449  1333 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.138384 28154 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:13:25.023473  1314 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
W20260709 13:13:25.020011 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 168487us
I20260709 13:13:25.036711 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:25.037523 28082 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:25.042519 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:25.145365 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:25.146339 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:25.146598 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:25.136906 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 84490us
I20260709 13:13:25.015642  1331 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:25.035776 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:24.915596  1321 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.016950  1199 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.154858 28229 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:25.158440 28082 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:25.035844  1140 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:25.142431  1319 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.334s	user 0.000s	sys 0.002s
I20260709 13:13:25.173944 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:25.175057 28079 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:25.140364  1187 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:25.021260   455 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.179605 28229 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:13:25.181003  1140 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.174563  1199 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.154866  1333 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.198546 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:25.181888  1187 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:25.243767 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1009) took 1355 ms. Trace:
I20260709 13:13:25.187562  1336 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:25.246752 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48432.7us
W20260709 13:13:25.212531  1328 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:13:25.213518  1322 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:13:25.248261 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 68149.5us
I20260709 13:13:25.244266 28152 rpcz_store.cc:276] 0709 13:13:23.887874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.924454 (+ 36580us) service_pool.cc:224] Handling call
0709 13:13:23.924825 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.936390 (+ 11565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.936425 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.945609 (+  9184us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.079464 (+133855us) spinlock_profiling.cc:243] Waited 4.37 ms on lock 0x7b480024c258. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:13:24.522539 (+443075us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:24.522599 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.522693 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.214604 (+691911us) raft_consensus.cc:1685] finished
0709 13:13:25.214627 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.243681 (+ 29054us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11064,"raft.queue_time_us":11213,"raft.run_cpu_time_us":2719,"raft.run_wall_time_us":140878,"spinlock_wait_cycles":10047360,"thread_start_us":471577,"threads_started":1,"wal-append.queue_time_us":697287}
I20260709 13:13:25.251818 28085 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:25.173952  1331 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:25.263967 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53909.3us
I20260709 13:13:25.197069  1307 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:13:25.265841 28050 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:25.266618 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 127539us
I20260709 13:13:25.201440  1141 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:13:25.202221  1347 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:13:25.258466   846 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:13:25.252604  1336 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:25.269564 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.325424 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 926) took 1393 ms. Trace:
I20260709 13:13:25.326256 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:25.291414 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:25.348999 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 999) took 1464 ms. Trace:
I20260709 13:13:25.336493 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:25.357383 28160 rpcz_store.cc:276] 0709 13:13:23.885025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.885150 (+   125us) service_pool.cc:224] Handling call
0709 13:13:23.939087 (+ 53937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.254275 (+315188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.254303 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.265479 (+ 11176us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.632400 (+366921us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:24.632438 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.632514 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.348759 (+716245us) raft_consensus.cc:1685] finished
0709 13:13:25.348781 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.348937 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99151,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3166,"thread_start_us":608909,"threads_started":2,"wal-append.queue_time_us":677889}
I20260709 13:13:25.270403  1345 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:25.372241 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:25.351042 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:25.373066 28084 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
W20260709 13:13:25.369289 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 75875.3us
W20260709 13:13:25.271854  1341 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:13:25.375867 28233 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:25.409301 28755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:25.418747 28755 raft_consensus.cc:606] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:25.328428  1182 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.775s	user 0.008s	sys 0.001s
W20260709 13:13:25.324972  1343 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:13:25.308558  1320 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:25.410744 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 388005us
W20260709 13:13:25.412966 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 62062.7us
I20260709 13:13:25.356269 28757 rpcz_store.cc:276] 0709 13:13:23.921984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.922109 (+   125us) service_pool.cc:224] Handling call
0709 13:13:23.949758 (+ 27649us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:24.117748 (+167990us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.117794 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:24.188355 (+ 70561us) log.cc:844] Serialized 52 byte log entry
0709 13:13:24.584966 (+396611us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:24.585009 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.585092 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.315019 (+729927us) raft_consensus.cc:1685] finished
0709 13:13:25.315050 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.315232 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24624,"raft.queue_time_us":184236,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":32741,"thread_start_us":348329,"threads_started":1,"wal-append.queue_time_us":459881}
I20260709 13:13:25.419853 28078 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:13:25.270948  1339 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:13:25.330000  1331 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.372972  1258 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:25.319298  1140 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.289039  1340 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.270190  1277 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:25.285822  1152 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:25.325248  1348 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:25.430322 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:25.431207 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.262892  1150 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:25.348996  1346 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:25.435012  1210 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.472959 28233 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:25.473567  1210 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.474417 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:25.475332 28233 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:25.475996  1210 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:25.477248 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:25.477880 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:25.478245  1210 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:13:25.436041  1342 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:13:25.484418 28082 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:25.419485  1233 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:25.486773  1233 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:25.447880  1333 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.273041  1029 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.159s
I20260709 13:13:25.449641  1336 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:25.312935  1207 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
I20260709 13:13:25.451303  1321 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.453066   455 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:25.454510  1199 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.421139  1303 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:25.496757  1303 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.499217  1303 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:25.504940 28226 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:25.433741  1320 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.505728  1258 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.507166 28226 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:25.507560  1320 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:25.509294  1150 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.511128  1156 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.489226  1152 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.514786  1152 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:25.457583 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:25.516386 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 58900us
I20260709 13:13:25.518286 28227 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:25.518615  1029 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:25.519376  1029 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:25.519788  1175 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:25.422750  1305 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:25.521040  1029 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:25.521436  1029 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:25.521059  1305 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.427419 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:25.524190 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 96859.7us
I20260709 13:13:25.462639 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:25.463539  1340 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.525038 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:13:25.464322  1304 consensus_meta.cc:287] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.001s
I20260709 13:13:25.466198 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:25.467820 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:25.479151  1210 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:25.468925 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:25.527014 28081 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:25.445945  1349 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.485369 28233 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:13:25.488658 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.470355  1277 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:25.429936  1182 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:25.548050 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:25.549024 28081 raft_consensus.cc:2376] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16. Re-sending same reply.
I20260709 13:13:25.550976 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:25.532888 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.518199 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:25.550757 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:25.522281  1029 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:25.556273 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.558706 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:25.537420  1210 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:25.488229  1346 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.561393 28158 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:25.559787 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:25.540699  1321 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:25.541541  1152 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:25.539315  1350 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.547201  1277 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:25.564507 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:25.611426 28228 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:25.563963 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:25.557859  1333 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:25.615033  1333 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:25.618829  1333 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:25.619009  1345 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:25.564324  1355 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:25.620651  1047 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:25.621169 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:25.541042  1233 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.547464  1349 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.616937  1153 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:25.622179  1171 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:25.623703  1171 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:25.623535  1153 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.624107  1171 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:25.600553  1157 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:25.624657  1349 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.625353  1156 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:25.539580  1304 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.625735  1153 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.614766  1152 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:25.627429  1304 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:25.627902  1152 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.616993  1350 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:25.630662 28049 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.196:46855 timed out after 1.593s (SENT) [suppressed 16 similar messages]
I20260709 13:13:25.625746  1156 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:25.632100  1315 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.898s	user 0.006s	sys 0.010s
I20260709 13:13:25.605877   846 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:25.632477  1156 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.633201   846 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.621832  1340 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:25.591706  1149 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:13:25.593255  1334 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:13:25.640118 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 928) took 1716 ms. Trace:
I20260709 13:13:25.640576  1233 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:25.640676 28760 rpcz_store.cc:276] 0709 13:13:23.923471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.923582 (+   111us) service_pool.cc:224] Handling call
0709 13:13:23.950333 (+ 26751us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.135296 (+184963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.135341 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.135887 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.520684 (+384797us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:24.520770 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.520882 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.332072 (+811190us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b48000609d8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:13:25.631351 (+299279us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:25.639876 (+  8525us) raft_consensus.cc:1685] finished
0709 13:13:25.639899 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.640057 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162572,"raft.queue_time_us":186134,"raft.run_cpu_time_us":21364,"raft.run_wall_time_us":1183355,"spinlock_wait_cycles":60566784,"thread_start_us":324027,"threads_started":1,"wal-append.queue_time_us":477384}
I20260709 13:13:25.639807  1156 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:25.615260  1354 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:25.645269 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:25.645691  1357 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:13:25.591948  1347 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 13:13:25.649655 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 85841.3us
I20260709 13:13:25.651203 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:25.658262 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:25.660240 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:25.662725  1335 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.642s	user 0.004s	sys 0.000s
I20260709 13:13:25.662515  1333 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.665083   792 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:25.669176 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1081) took 1384 ms. Trace:
I20260709 13:13:25.670388  1333 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.669622 28235 rpcz_store.cc:276] 0709 13:13:24.284789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.284981 (+   192us) service_pool.cc:224] Handling call
0709 13:13:24.287422 (+  2441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.637353 (+349931us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.637387 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.638157 (+   770us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.815158 (+177001us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:24.815195 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.815268 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.662048 (+846780us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:25.668844 (+  6796us) raft_consensus.cc:1685] finished
0709 13:13:25.668868 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.669106 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45952,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":8817536,"thread_start_us":482289,"threads_started":2,"wal-append.queue_time_us":266110}
I20260709 13:13:25.670953   846 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.651327 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:25.680366 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:25.681102 28234 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:25.684360  1303 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:25.687965  1303 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.689652 28234 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:25.690430  1303 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:25.691205 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:25.654835 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:25.655171  1305 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:25.656431  1356 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:13:25.697263  1187 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.699609  1350 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:25.700359 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:25.704278 28757 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:25.709408 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:25.717281 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:25.723738 28755 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:25.726241 28156 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:25.728120 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:25.737295 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 86167us
I20260709 13:13:25.737897 28153 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:13:25.738648 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:25.739464 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:25.737365 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:25.741940 28759 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:13:25.744414 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:25.733458  1333 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:25.750839 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:25.732630  1187 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:25.762131 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:25.763022 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:25.759548 28757 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:25.771785 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:25.772787 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:25.745764  1321 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:25.751608  1355 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:25.807377 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:25.811882 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 181227us
I20260709 13:13:25.770771  1349 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:13:25.859403 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.776438  1345 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:25.775104  1333 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:25.862071 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 47312.6us
I20260709 13:13:25.894354 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:25.897635 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:25.898533 28082 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:25.901631 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:25.903523 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 131317us
I20260709 13:13:25.805035  1058 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:25.905853 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:25.862192   846 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:25.911929 28764 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:13:25.910836 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.875505  1177 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:25.906661 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 250013us
I20260709 13:13:25.878068  1149 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.784194  1210 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.790323  1258 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.117s
W20260709 13:13:25.923866 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.861485  1354 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:13:25.861678   966 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:13:25.966480 28755 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:25.930279  1345 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:25.954113  1360 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:13:25.953187  1354 consensus_queue.cc:1243] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:13:25.980583 28080 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:25.981738 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:25.918211  1058 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:25.961833  1199 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.018807 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:25.943178  1149 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.928248  1177 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:25.968016  1303 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:25.946017  1305 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:26.091312  1305 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.019662 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.093111 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.053218 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.093817 28756 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
W20260709 13:13:26.094254  1361 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:13:26.094769 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.096272 28756 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:26.092590 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.097934 28761 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:26.093137 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:26.099951 28762 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:26.101364  1366 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:13:26.101964  1177 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.104876  1305 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.112208 28764 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:26.081372 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:26.082721 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:26.086850  1363 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:13:26.087574 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.090243 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.116009 28757 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
W20260709 13:13:26.092521 28186 heartbeater.cc:646] Failed to heartbeat to 127.26.83.254:42913 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.26.83.254:42913 timed out after 15.000s (SENT)
I20260709 13:13:26.054123  1199 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:26.122761  1376 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:13:26.124579  1199 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.138748  1149 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.117710  1303 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.143280  1058 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.144871  1210 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:25.973276  1333 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.149471  1303 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.127348 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2533) took 1151 ms. Trace:
I20260709 13:13:26.150883 28154 rpcz_store.cc:276] 0709 13:13:24.976277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.976429 (+   152us) service_pool.cc:224] Handling call
0709 13:13:25.000091 (+ 23662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.138875 (+138784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.127057 (+988182us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b54001973d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:26.127285 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90171264}
I20260709 13:13:26.158120 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:26.159351  1277 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.000s
I20260709 13:13:26.159866  1277 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.162375  1277 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:26.166435  1316 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.362s	user 0.002s	sys 0.005s
I20260709 13:13:26.168118 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2292) took 2006 ms. Trace:
I20260709 13:13:26.168568 28231 rpcz_store.cc:276] 0709 13:13:24.161276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.184445 (+ 23169us) service_pool.cc:224] Handling call
0709 13:13:24.184830 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.201144 (+ 16314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.201204 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.201840 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.567284 (+365444us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:24.594819 (+ 27535us) log.cc:844] Serialized 14 byte log entry
0709 13:13:24.595553 (+   734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.595661 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.186197 (+590536us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b180043e460. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a522 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:13:26.165660 (+979463us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:26.167844 (+  2184us) raft_consensus.cc:1685] finished
0709 13:13:26.167876 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.168051 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24563,"prepare.queue_time_us":26445,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":763,"raft.queue_time_us":17684,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":744,"spinlock_wait_cycles":23298944,"thread_start_us":364967,"threads_started":1,"wal-append.queue_time_us":463092}
I20260709 13:13:26.171160 28760 raft_consensus.cc:2410] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 13:13:26.157562  1171 consensus_meta.cc:287] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 13:13:26.180840 28762 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:26.181983 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2534) took 1202 ms. Trace:
I20260709 13:13:26.182602 28158 rpcz_store.cc:276] 0709 13:13:24.979406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.979733 (+   327us) service_pool.cc:224] Handling call
0709 13:13:25.017237 (+ 37504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.594217 (+576980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.180508 (+586291us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b540011e254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:26.181898 (+  1390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250376448}
I20260709 13:13:26.187142  1029 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:26.193347  1352 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.174366  1171 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.193917  1352 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:26.205273 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 185340us
I20260709 13:13:26.206485 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:26.207319  1352 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:26.208103 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:26.210949 28125 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:26.210012  1384 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:26.214272  1182 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:26.216449  1171 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.211257  1187 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
W20260709 13:13:26.217499  1372 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:13:26.216607  1182 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:26.219064  1165 raft_consensus.cc:993] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:13:26.218587  1182 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:26.219156 28758 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:13:26.219744 28758 pending_rounds.cc:85] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 12
I20260709 13:13:26.220034 28758 pending_rounds.cc:107] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:13:26.220285 28758 raft_consensus.cc:2889] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:26.221661 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:26.212762  1384 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:26.215010  1126 raft_consensus.cc:2764] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:13:26.223766  1187 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:26.225100  1384 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:26.226766  1187 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.219751  1165 raft_consensus.cc:1081] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:13:26.232939 28078 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:26.236786 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:26.240253  1151 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:13:26.238183 28049 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5c654c99561449cb823642a1b85879ac. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:26.251578 28049 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:26.253122  1385 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:26.253396  1252 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:26.254269  1385 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:26.254565  1252 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:26.253541 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:26.257666  1331 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:26.258745 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:26.259538  1331 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:26.259716  1347 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:26.248529  1151 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:26.242264  1047 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:26.264617  1151 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:26.265712  1385 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.264957  1331 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:26.266525  1151 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:26.238277  1187 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.262485 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:26.267727  1331 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:26.242992  1126 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:26.270591  1126 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:26.271078  1126 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:26.272253 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 350003us
W20260709 13:13:26.274333 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.274762 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:26.277431 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:26.279773  1321 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:26.276266   455 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:26.280426  1321 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:26.275735  1349 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:13:26.282117  1349 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:26.282490  1349 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:26.283298  1349 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:26.284696  1321 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:26.285904  1346 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:26.268213  1331 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:26.286067  1252 raft_consensus.cc:2955] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:13:26.287281  1331 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.284492  1349 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:26.297959 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.298264 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:26.298647 28084 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:26.315649 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:26.317349 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.318912 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.320385 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.327899 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.331159 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.334591 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:26.335544 28762 raft_consensus.cc:2376] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18. Re-sending same reply.
I20260709 13:13:26.338964 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.339684 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.340422 28158 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:26.337051 28125 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:26.343674 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:26.344398 28083 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:26.347116 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:26.347429 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:26.347791 28763 raft_consensus.cc:2376] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16. Re-sending same reply.
I20260709 13:13:26.349169 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.349926 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:26.238474  1384 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:26.350553 28081 raft_consensus.cc:2376] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14. Re-sending same reply.
I20260709 13:13:26.351684 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:26.352252 28081 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:26.355633 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:26.356256 28757 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:13:26.356285 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:26.364301  1173 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:26.364948  1173 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:26.365399  1173 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:26.344105  1029 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:13:26.362108 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.363082 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.377339 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.318955  1347 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:26.362751 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:26.366145 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:26.327517  1258 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.344842  1237 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.383704 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:26.384656 28227 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:26.375442 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:26.336226  1326 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:26.377300  1388 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:13:26.392601 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.342360  1187 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:26.403640 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.408375 28079 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:26.410116 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:26.391062 28233 raft_consensus.cc:2955] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:13:26.410176 28757 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:26.415542 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.417266 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.418213 28728 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:26.414762  1187 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.421639 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.380599  1347 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:26.429894 28081 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:26.439141 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.445938   312 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:26.446484   312 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.410982  1199 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.449360 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.453255 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:26.455284 28154 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:26.413866  1042 raft_consensus.cc:2764] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:13:26.462881 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.464682 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.434442  1165 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:26.467939 28757 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:26.437376  1347 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:26.473102 28160 pending_rounds.cc:85] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 11
I20260709 13:13:26.473693 28160 pending_rounds.cc:107] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:13:26.467021  1199 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:26.482344 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:26.483690  1384 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.484264  1381 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:26.487742 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.468152  1337 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:26.474172 28160 raft_consensus.cc:2889] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:26.488900 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.489074  1337 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:26.489530 28086 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
W20260709 13:13:26.490381 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.490047  1337 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.451195  1338 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.403059  1237 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.491277 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.492147 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.493271 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:26.444142  1302 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:26.402681  1258 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:26.525866 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44125.4us
I20260709 13:13:26.527160 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:26.457290   966 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:13:26.460004  1175 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.516389 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.531412 28079 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:26.473321  1165 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.518270  1338 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.531692 28154 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 15: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:26.521015 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:26.521371  1386 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:13:26.521605 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.519256 28227 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:26.528033 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:26.557219 28759 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:26.633828 28762 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:13:26.532075  1382 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:26.518309  1199 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.545372  1165 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:26.660434 28085 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:13:26.556300  1354 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:26.660079  1161 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:26.552516  1175 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:26.671963  1277 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:26.738549 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.727303  1354 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:26.755944  1393 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:13:26.757242  1395 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:13:26.762331 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.763991 28762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:26.766544  1254 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:26.767107  1254 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:26.785318  1258 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.786118  1237 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.786912  1237 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.788154  1338 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.768245  1254 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.788482 28762 raft_consensus.cc:606] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:26.784528  1173 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:26.792706  1126 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.521s	user 0.007s	sys 0.000s
I20260709 13:13:26.797042  1173 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.796764 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:26.792855 28762 raft_consensus.cc:614] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:26.787535  1237 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.747465  1187 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.816339 28083 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:26.831097 28086 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:13:26.741410  1161 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.822726 28235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:44070
I20260709 13:13:26.845103 28235 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:26.860186 28235 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:26.797741  1126 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:26.844755 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:26.865778 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:26.862635 28762 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:26.838064  1385 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:26.893482  1385 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:26.894892  1347 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.895753  1385 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.870757  1396 ts_tablet_manager.cc:933] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:13:26.871021  1354 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.874780  1141 raft_consensus.cc:993] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:13:26.900902  1141 raft_consensus.cc:1081] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:13:26.878099  1175 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.879904  1161 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:26.881604  1165 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.884460  1126 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.842994  1149 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:26.819124  1173 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:13:26.891399 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.910113 28157 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:26.908932 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:26.917325 28755 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:26.926450 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:26.925239 28235 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.930245  1165 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:26.931262 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:26.931914 28157 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:13:26.943454  1237 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.910207  1385 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:13:26.944244 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:26.944525  1385 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:26.944999  1385 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:26.945039 28154 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:26.945997  1385 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:26.947224  1385 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:26.947525 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:26.950349 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.911656  1149 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:26.956998 28157 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:26.931737  1252 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:36752 (request call id 18) took 3613 ms. Trace:
I20260709 13:13:26.960114 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:26.956216 28235 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.961557  1149 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:26.959570  1252 rpcz_store.cc:276] 0709 13:13:23.311496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.317992 (+  6496us) service_pool.cc:224] Handling call
0709 13:13:23.514692 (+196700us) log.cc:844] Serialized 506 byte log entry
0709 13:13:23.883420 (+368728us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b5400331ba0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f0142 00007f59ce5d0eaa 00007f59ce5d7d53 00007f59ce5d7cd9 00007f59ce5d7c69 00007f59ce5d7c31 00007f59ce5d6dfd 000000000038f354 000000000038f199 00007f59cb58a475
0709 13:13:24.098139 (+214719us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b18002fe060. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f7b26 00007f59ce5f149a 00007f59ce5f0714
0709 13:13:24.099740 (+  1601us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b5400331ba0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f7b26 00007f59ce5f149a 00007f59ce5f0714 00007f59ce5d0eaa 00007f59ce5d7d53 00007f59ce5d7cd9 00007f59ce5d7c69 00007f59ce5d7c31 00007f59ce5d6dfd 000000000038f354
0709 13:13:24.217337 (+117597us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b5400331ba0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f0142 00007f59ce5d0eaa 00007f59ce5d7d53 00007f59ce5d7cd9 00007f59ce5d7c69 00007f59ce5d7c31 00007f59ce5d6dfd 000000000038f354 000000000038f199 00007f59cb58a475
0709 13:13:26.925002 (+2707665us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36829,"raft.queue_time_us":15812,"raft.run_cpu_time_us":13120,"raft.run_wall_time_us":1375452,"spinlock_wait_cycles":53849728,"thread_start_us":141617,"threads_started":1,"wal-append.queue_time_us":214977,"wal-append.run_cpu_time_us":2741,"wal-append.run_wall_time_us":1063909}
I20260709 13:13:26.965423 28161 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43184
I20260709 13:13:26.966135 28161 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:26.966187   312 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.966589 28161 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:26.952270  1161 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:26.975356 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.975095 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:26.979611 28125 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:26.982066 28082 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:13:26.982036 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:26.958143  1347 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.984069 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:26.986735  1384 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:26.986845 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:26.987622  1384 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:26.990530 28079 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:26.990746 28085 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:26.988649  1199 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:26.993144 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:27.001820 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:27.009178  1405 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:13:27.010059  1199 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:26.955681  1396 tablet_copy_client.cc:323] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.193:45587
I20260709 13:13:26.984994  1187 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:27.016705  1187 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:27.016988 28083 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:27.018152  1187 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.030759 28161 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:27.035796 28082 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
W20260709 13:13:27.043985 28160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.006s	sys 0.007s
W20260709 13:13:27.044476 28160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.006s	sys 0.007s
I20260709 13:13:27.051882 28161 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:26.998329 28080 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:27.054322 28050 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:27.056362  1140 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:27.056831  1140 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.057276  1140 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:27.057768 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.998771 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:26.980472  1042 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:27.061007  1042 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:27.061439  1042 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:26.997165  1384 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.080986 28152 raft_consensus.cc:2955] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:13:27.082624 28084 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
W20260709 13:13:27.085817 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 109210us
I20260709 13:13:27.090821 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:27.092983 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2811) took 1414 ms. Trace:
W20260709 13:13:27.093927 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.094035  1350 raft_consensus.cc:2764] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:13:27.095907 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:27.096598 28760 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:27.097911 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:27.086334 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 100516us
I20260709 13:13:27.103116 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:27.103482 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:27.099843 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:27.103173  1321 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:27.105329 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:27.105636  1321 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.106105 28158 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:27.093436 28228 rpcz_store.cc:276] 0709 13:13:25.678597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.679440 (+   843us) service_pool.cc:224] Handling call
0709 13:13:25.679777 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:25.742913 (+ 63136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.742941 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:25.743433 (+   492us) log.cc:844] Serialized 52 byte log entry
0709 13:13:25.914160 (+170727us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:26.047108 (+132948us) log.cc:844] Serialized 14 byte log entry
0709 13:13:26.190779 (+143671us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b48000600d8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d35a0aa1 00007f59d3597b37 00007f59d35979ca 00007f59d368f26f 00007f59d368ef37 00007f59d36a355c 00007f59d36a3508 00007f59d36a3488 00007f59d36a3443 00007f59d36a26ec 00007f59d25cca96
0709 13:13:26.278688 (+ 87909us) log.cc:844] Serialized 14 byte log entry
0709 13:13:27.092435 (+813747us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b18004e5f20. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:13:27.092616 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.092693 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:27.092772 (+    79us) raft_consensus.cc:1685] finished
0709 13:13:27.092788 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.092926 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":440033,"prepare.queue_time_us":243077,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":1111871,"raft.run_cpu_time_us":1472,"raft.run_wall_time_us":444996,"spinlock_wait_cycles":25512064,"thread_start_us":226638,"threads_started":2,"wal-append.queue_time_us":423773}
I20260709 13:13:27.114230 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:27.115491 28081 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:27.120657 28759 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:27.120774 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:27.122787 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:27.126456 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:27.121460 28161 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:27.139269 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:27.139933 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:27.143759 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:27.145431 28081 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
W20260709 13:13:27.150279 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 51282.5us
W20260709 13:13:27.152448 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.138157  1150 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:27.155980  1150 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.163209 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:27.165114 28079 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:13:27.179669  1354 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:27.184986  1354 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:27.186090 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46232us
I20260709 13:13:27.187268 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:27.174526  1258 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:27.137570  1161 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:27.192219 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:27.199892 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.200804 28230 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:27.173231  1346 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:27.246599 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54447.9us
I20260709 13:13:27.246978 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:27.248199  1325 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:27.248817  1325 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:27.249027  1162 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:27.248239 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.249632  1162 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:27.249712  1325 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:27.251856  1162 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:27.253748 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:27.254447 28233 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:27.254544 28759 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:27.255255 28759 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:27.252430 28730 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:27.203578  1151 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:27.261181 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:27.261799 28758 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:27.262100 28758 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:27.263232 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 124045us
I20260709 13:13:27.263993 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:27.264380 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:27.264994 28080 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:27.265754 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:27.257673 28759 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:27.162370  1042 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:27.268438 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:27.262852 28758 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:27.290464 28758 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:27.269889  1152 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:27.297712 28096 tablet_copy_service.cc:140] P 0faaace2b36c4bcebf2fa6835235e92b: Received BeginTabletCopySession request for tablet 5c654c99561449cb823642a1b85879ac from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:36768)
I20260709 13:13:27.297966  1350 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:27.298725 28096 tablet_copy_service.cc:161] P 0faaace2b36c4bcebf2fa6835235e92b: Beginning new tablet copy session on tablet 5c654c99561449cb823642a1b85879ac from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:36768: session id = 70996712c3c942468a6006602de6477f-5c654c99561449cb823642a1b85879ac
W20260709 13:13:27.301086 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48745.7us
I20260709 13:13:27.246042  1161 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:27.304028 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.316244 28233 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:27.305011 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:27.317469 28229 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:27.306010 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.287173   846 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:27.288317  1258 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:27.323660 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.329128 28228 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:27.324014 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:27.325016 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.330732 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.295636  1149 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:27.335999 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:27.338515 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.340152 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:27.340435 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:27.288221  1346 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:13:27.324447  1233 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:27.341220 28233 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:27.341218 28228 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:27.341116 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 354326us
I20260709 13:13:27.341503 28227 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:27.353802 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51473.7us
I20260709 13:13:27.340107 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:27.358281 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:27.359226 28081 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:27.327616   846 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.360769 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:27.361495 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:27.365207 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.367064 28230 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:13:27.368793 28232 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:27.373026 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.373332 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:27.374104 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:27.374257 28125 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:27.375084 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:27.376673 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:27.374713 28079 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:27.379794  1305 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:27.380391  1305 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:27.381361  1305 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.383111  1303 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:27.383742  1303 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:27.384654  1303 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.385785  1042 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:27.387996 28032 auto_rebalancer.cc:629] Failed to schedule move for tablet c5b0e8aadf244e7eb41e7e732bc27cf9: Illegal state: Replica 0faaace2b36c4bcebf2fa6835235e92b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:27.388914  1182 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:27.394044 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:27.395306 28096 tablet_copy_source_session.cc:215] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:13:27.398039  1187 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:27.398900  1187 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:27.399464  1187 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:27.400992  1187 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:27.403039 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.379065 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:27.406052  1152 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:27.407110  1149 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.407429 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:27.407940  1149 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.408823   792 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:27.409186   455 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:27.410689 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:27.412276 28758 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:27.376755 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:27.355302  1233 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.419912 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:27.421612 28160 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
I20260709 13:13:27.364269  1254 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:27.368192   846 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:27.425158 28156 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:13:27.428705 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:27.429639 28160 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:27.423864  1233 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:27.336647  1258 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.370687  1424 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:27.412526  1423 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:27.457909 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:27.459933 28156 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:27.377734  1418 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.089s
I20260709 13:13:27.467867 28152 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:27.492611 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:27.469436  1420 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:13:27.493088 28078 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:27.514389 28229 raft_consensus.cc:2410] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:13:27.420596  1161 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:27.548170 28160 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:27.491575  1424 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.579013 28078 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:27.561390  1423 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:27.585106 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:27.592854  1422 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:13:27.585353  1258 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:27.598342  1258 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:27.599135  1258 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.590838  1303 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:27.610373  1258 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:27.613054  1258 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.613844  1303 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:27.585086  1423 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:27.615183  1303 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:27.618073  1258 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.627120 28759 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:27.614261  1421 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:13:27.616274  1423 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:27.660547  1385 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:27.661307  1385 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:27.670861 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.673218 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:27.674144 28230 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:27.676920 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:27.682027 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 271441us
I20260709 13:13:27.662999  1321 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:27.690346  1326 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.721580 28759 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:27.736691 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:27.711351  1140 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.003s
W20260709 13:13:27.786497 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:27.798118 27980 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 192) took 16713 ms (client timeout 15000 ms). Trace:
I20260709 13:13:27.801103 28226 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:27.807492 28232 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:27.709326  1321 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:27.816220 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 79624.9us
I20260709 13:13:27.823902 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:27.724259  1326 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.829592  1350 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:27.829811  1171 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:27.824290  1152 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:27.818244 27980 rpcz_store.cc:269] 0709 13:13:11.013598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.015364 (+  1766us) service_pool.cc:224] Handling call
0709 13:13:12.366665 (+1351301us) spinlock_profiling.cc:243] Waited 65.2 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e1090 00007f59ce67ff3a 00007f59d44361cc 00007f59d453437e 00007f59d29f52a4 00007f59d29f5232 00007f59d29f5161 00007f59d29f50dc 00007f59d29f4392 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:27.727143 (+15360478us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:21.843015 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:22.080766 (+237751us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:22.080841 (+    75us) write_op.cc:435] Acquired schema lock
0709 13:13:22.080862 (+    21us) tablet.cc:662] Decoding operations
0709 13:13:22.241502 (+160640us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:22.241604 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:22.241636 (+    32us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:13:22.243160 (+  1524us) tablet.cc:701] Row locks acquired
0709 13:13:22.243211 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:13:22.243435 (+   224us) write_op.cc:270] Start()
0709 13:13:22.243617 (+   182us) write_op.cc:276] Timestamp: P: 1783602802243353 usec, L: 0
0709 13:13:22.243652 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:13:22.244424 (+   772us) log.cc:844] Serialized 6728 byte log entry
0709 13:13:23.087142 (+842718us) op_driver.cc:464] REPLICATION: finished
0709 13:13:23.233469 (+146327us) write_op.cc:301] APPLY: starting
0709 13:13:23.248118 (+ 14649us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:23.594758 (+346640us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:23.628549 (+ 33791us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:23.791735 (+163186us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:23.835225 (+ 43490us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:13:23.835312 (+    87us) write_op.cc:312] APPLY: finished
0709 13:13:23.835786 (+   474us) log.cc:844] Serialized 156 byte log entry
0709 13:13:23.835989 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:23.836167 (+   178us) write_op.cc:454] Released schema lock
0709 13:13:23.836252 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:23.836677 (+   425us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:23.836705 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11933,"reported_tablets":33,"spinlock_wait_cycles":149867392,"child_traces":[["op",{"apply.queue_time_us":124478,"apply.run_cpu_time_us":29427,"apply.run_wall_time_us":615002,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75425,"num_ops":17,"prepare.queue_time_us":363519,"prepare.run_cpu_time_us":62376,"prepare.run_wall_time_us":719770,"raft.queue_time_us":202916,"raft.run_cpu_time_us":18650,"raft.run_wall_time_us":87038,"replication_time_us":843288,"thread_start_us":629164,"threads_started":4,"wal-append.queue_time_us":276938,"wal-append.run_cpu_time_us":16836,"wal-append.run_wall_time_us":2320794}]]}
I20260709 13:13:27.833941  1321 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.855608  1438 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.860155  1149 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:27.712196  1385 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:27.834808  1424 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:27.630185 27979 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:27.929497 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:27.934641 28082 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:27.934648 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:27.840461  1029 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:27.832086  1140 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:27.958187 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:27.977392 28086 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:27.937160  1171 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.918401  1252 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:28.003983 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 154348us
I20260709 13:13:27.967340  1437 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:27.980931  1424 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.987416  1171 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:28.022406 28080 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:28.022363   312 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:13:28.029110 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:28.030632 28231 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
W20260709 13:13:28.037070  1433 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:13:27.957589  1438 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.023200  1337 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.020510  1258 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:28.023635  1320 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.028424  1385 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:27.998951  1252 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.027704  1424 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.056334 28081 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:28.057102 28160 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:13:28.057348 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:28.057915 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:28.058007 28152 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:28.046303  1187 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.046942  1126 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:28.066532 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:28.050685  1305 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:28.067888 28085 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:28.028146  1437 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.068463 28086 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:28.058815  1417 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:28.058663  1173 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.049580  1320 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.072039  1305 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.062552  1303 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:28.047689  1337 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:28.108412  1442 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.004s
I20260709 13:13:28.069828  1126 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.119204  1126 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.121011  1321 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:28.116536   846 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.000s
I20260709 13:13:28.109835  1448 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:28.111532  1446 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.086s	user 0.004s	sys 0.000s
I20260709 13:13:28.127629 28158 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:28.128386   312 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.121555  1252 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.133456  1305 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.129914 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:13:28.134055  1426 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.551s	user 0.002s	sys 0.000s
I20260709 13:13:28.122043  1303 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.137447  1438 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:28.140158  1150 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:28.142264  1441 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.145s	user 0.001s	sys 0.001s
I20260709 13:13:28.143757  1337 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.145457  1140 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.145788 27979 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 9 to 11, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 11 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:28.147397  1346 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:28.147874  1346 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:28.131255   312 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:28.161584 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 98468.9us
I20260709 13:13:28.112826  1326 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.068225  1187 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.198375 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:28.205534  1305 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.206087  1305 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.208063  1305 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.212378   312 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.152815  1448 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.107864  1417 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.149987  1424 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:28.245486 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:28.157589  1199 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:28.203507  1423 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.004s
I20260709 13:13:28.112243  1173 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.192824  1140 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.249625 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:28.251272 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:28.252314 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:28.201905  1126 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.125281  1447 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.121s
I20260709 13:13:28.131227   846 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.244679  1305 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:28.253911  1384 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:28.270545 28077 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:28.272096  1451 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:28.272716  1447 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:28.273021  1448 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.253373  1423 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:28.273617  1439 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.206s	user 0.000s	sys 0.002s
I20260709 13:13:28.274266  1305 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:28.274771  1305 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:28.274925  1431 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.241s	user 0.000s	sys 0.007s
I20260709 13:13:28.275219  1305 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:28.281905  1451 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:28.282208  1233 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.810s	user 0.007s	sys 0.005s
I20260709 13:13:28.282678  1451 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:28.283149 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1210) took 1188 ms. Trace:
W20260709 13:13:28.285020  1427 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.712s	user 0.010s	sys 0.005s
I20260709 13:13:28.284365  1305 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:28.285984 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 902941us
I20260709 13:13:28.288165 28198 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:28.288486 28077 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:28.192440  1346 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.289021  1349 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:28.289471  1346 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:28.289592  1349 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:28.290661  1349 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.283241  1451 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.293774  1187 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.256155  1140 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:28.295780 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 919020us
I20260709 13:13:28.283603 28764 rpcz_store.cc:276] 0709 13:13:27.095069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.107404 (+ 12335us) service_pool.cc:224] Handling call
0709 13:13:27.136369 (+ 28965us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:27.189466 (+ 53097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:27.189499 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:27.189993 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:13:27.356948 (+166955us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:27.372051 (+ 15103us) log.cc:844] Serialized 14 byte log entry
0709 13:13:28.282490 (+910439us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b18004a2360. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:13:28.282677 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:28.282782 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:28.282830 (+    48us) raft_consensus.cc:1685] finished
0709 13:13:28.282949 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:28.283093 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57328,"prepare.queue_time_us":18643,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":53579,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":3480960,"thread_start_us":166660,"threads_started":1,"wal-append.queue_time_us":306373}
I20260709 13:13:28.282711  1233 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.296150  1234 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:28.275091  1440 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:13:28.296916 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:28.287285 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.253762  1199 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.315778 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:28.321286 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:28.296312  1270 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:28.258925  1126 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.148640 27979 catalog_manager.cc:5697] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:28.327790 28085 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:28.329744  1451 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.338438 28764 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:13:28.340778 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.329638  1305 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.314208  1350 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.349886 28764 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:28.294991  1384 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.359570 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.360256 28759 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:28.327080  1270 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:28.368898 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 116629us
I20260709 13:13:28.371026 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:28.378506 28227 raft_consensus.cc:2410] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:13:28.371987 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:28.379029 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:28.375444 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:28.380064 28235 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:28.393126 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:28.389803 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.355001  1350 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:28.348500  1437 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:28.407034  1321 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.410054  1355 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:28.410218 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:28.410797  1187 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:28.410880  1152 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:28.411475  1305 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:28.426182  1355 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.414963 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:28.407431  1438 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.417652  1384 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:28.419689 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 58157us
I20260709 13:13:28.455699 28049 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:28.377980  1303 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.339123 27979 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 10 to 11, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 11 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:28.385524  1270 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:28.441785  1321 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.426738  1252 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:13:28.418766  1258 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:13:28.414233  1435 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:13:28.450904 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:28.414456  1233 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.427628 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:28.495412 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 67861.9us
I20260709 13:13:28.495833   312 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:28.377458  1452 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.422293 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:28.460912 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:28.386615  1150 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.464413 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:28.530925 28084 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:28.464525 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.529158 28227 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:28.388883  1424 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.468137 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:28.437551  1320 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.473067 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:28.454932  1355 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:28.490046 28078 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:13:28.491793 28764 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:28.462428  1438 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.496368   312 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:28.542593   312 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:28.495862 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:28.498755 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:28.462790  1254 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:28.467207  1437 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:28.529232 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 107051us
I20260709 13:13:28.554302 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:28.533552 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:28.541507 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.463142  1177 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:13:28.500903  1173 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.503458   846 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.561672 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1214) took 1321 ms. Trace:
I20260709 13:13:28.562179 28760 rpcz_store.cc:276] 0709 13:13:27.240298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.241288 (+   990us) service_pool.cc:224] Handling call
0709 13:13:27.266587 (+ 25299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:27.266805 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:13:27.492809 (+226004us) log.cc:844] Serialized 14 byte log entry
0709 13:13:28.455407 (+962598us) log.cc:844] Serialized 14 byte log entry
0709 13:13:28.455864 (+   457us) log.cc:844] Serialized 14 byte log entry
0709 13:13:28.550266 (+ 94402us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b180036c7c0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:13:28.550410 (+   144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:28.550455 (+    45us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:28.550501 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:28.550579 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:28.561612 (+ 11033us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":904389,"prepare.queue_time_us":235546,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":6866,"raft.queue_time_us":805962,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":7303,"spinlock_wait_cycles":4087424,"thread_start_us":370819,"threads_started":2}
I20260709 13:13:28.532783  1303 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:28.454371  1451 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:13:28.569090 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:28.539413  1423 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:28.585467 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 93773.1us
I20260709 13:13:28.546509  1254 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.587868  1384 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:28.558398  1448 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.598111  1173 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:28.531519  1452 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.539086  1424 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.567847  1152 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:28.588212  1447 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:28.595943  1254 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:28.566020  1325 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:28.567881  1307 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:28.638826  1307 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.641178  1307 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.576251  1140 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.564921  1437 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.643205 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:28.643894  1437 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.627132  1152 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:28.645005 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.645314  1152 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:28.645587 28080 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:28.541379  1150 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.649024  1150 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:28.615603 28759 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:13:28.620426 28157 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:28.607563 28158 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:28.622449 28159 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:28.623524 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:28.626181 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 129794us
I20260709 13:13:28.651752 28154 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:28.608510 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:28.596174  1126 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.605273  1455 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.635742 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:28.659499 28156 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:28.637077 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.633279  1418 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.637746  1325 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.629594   312 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:28.634711 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.655659 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.612476  1417 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.613847 27979 catalog_manager.cc:5697] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 6 to 9, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 9 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:28.677037 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:28.677667 28161 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:28.681300   312 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.688526 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:28.612690  1448 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.690023 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:28.692109 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:28.694998 28078 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:28.694521 28083 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:28.698293   312 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:28.698786 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:28.699667 28154 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:28.613974  1453 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:13:28.676769  1177 consensus_queue.cc:1243] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:13:28.616921  1173 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:28.709474 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:28.717085 28159 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:28.713878 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:28.714037 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:13:28.704828  1171 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.666s	user 0.009s	sys 0.003s
I20260709 13:13:28.727068 28156 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:28.728742 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:28.722669 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:28.684506   846 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.729368 28234 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:28.729668   846 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:28.726287 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.687767  1336 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.729135 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:28.734638 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:28.736884 28230 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:28.738219 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:28.662181  1468 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:13:28.746106 28154 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:28.751670 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:28.714479  1424 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.722052  1126 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.675796  1418 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.723245  1305 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.702874  1046 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:13:28.671025  1455 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.775419 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:28.707377  1385 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:13:28.774987 28230 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:28.767690 28161 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:28.753539  1321 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.773880  1042 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:28.784802  1042 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:28.785630  1459 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:13:28.785977  1455 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.787042  1042 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:28.787185  1460 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:13:28.790092 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:28.792335 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.794188  1396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c654c99561449cb823642a1b85879ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:28.795284  1447 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.795825  1447 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.766322  1140 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:28.796475 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:28.796805  1140 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:28.797258 28155 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:28.797721 28157 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:28.797849  1452 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.798887  1447 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.783589  1177 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:13:28.801362 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.802284  1418 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:28.803491 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.804806 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.806576 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.806766 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.807282 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.808271 28763 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:28.809026 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.810218 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:28.819586  1454 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:28.820103  1454 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.820490  1454 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:28.810562  1325 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:28.839334 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:28.840222 28081 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:28.809283 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:28.798998 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:28.783401  1417 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:13:28.736306  1171 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.849555  1171 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.816097  1347 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:28.817803 28156 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:28.854158  1347 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:28.825279 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:28.855034 28230 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 11.
I20260709 13:13:28.826546 28154 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:28.855088  1347 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.826905  1199 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.824040  1350 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.856868 28235 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:28.797271  1140 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:28.858024  1350 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.833423  1448 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.811301 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.778270  1438 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.850915 28085 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:28.850854  1454 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.777989  1305 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.775338  1126 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.749153  1336 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:28.814217 27979 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 10 to 11, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 11 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:28.856560  1347 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:28.862689 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.898666 28759 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:28.855912 28158 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:28.946717 28759 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:28.923771 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:28.953540 28156 raft_consensus.cc:2368] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19.
I20260709 13:13:28.904466 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:28.955891 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 51517.8us
I20260709 13:13:28.911478 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:28.868914 28756 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:28.889793 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.872629 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:28.904066 28155 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:28.962689 28761 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:13:28.881438 28050 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:28.873018 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:28.893245 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:29.003504 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45973.5us
W20260709 13:13:29.005093 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 60299.4us
I20260709 13:13:29.009524 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.010205 28230 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:29.004367 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:29.015918 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:28.866221  1454 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:28.864074  1173 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.870028  1336 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:29.034670 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:28.867988  1126 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:29.018674 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 137326us
I20260709 13:13:28.944109  1303 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:29.024297 28199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.004s	sys 0.006s
W20260709 13:13:29.038568 28199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.004s	sys 0.006s
W20260709 13:13:29.038836 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 645732us
I20260709 13:13:29.039500 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:29.017964  1270 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.002s
I20260709 13:13:29.023618 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:28.866200  1305 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:29.003562  1448 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:13:29.030750 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.013690  1161 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:13:28.861169  1254 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.045104 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:29.015230  1337 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.042757 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:29.048120 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:29.023743 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 282858us
I20260709 13:13:28.897421  1152 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.016793  1470 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:29.024576  1320 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.024705  1029 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:29.077879 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:29.084870 28125 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:29.078987 28763 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:28.888136  1355 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.099033  1320 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.085165  1029 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.085047  1337 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.111824  1305 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:29.127333  1029 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:29.127707  1350 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:29.061067  1270 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.061857  1046 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:29.111042 28232 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:29.134403 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 130114us
I20260709 13:13:28.881599  1354 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.204s
I20260709 13:13:29.049057  1471 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:29.067618  1448 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.045898  1438 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:29.126543 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:28.881320  1396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:56420 (request call id 1208) took 2534 ms. Trace:
I20260709 13:13:29.083415  1435 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:29.134696 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:29.049207  1303 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.135550 28226 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:29.137514 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:29.130692  1459 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:13:29.119997  1455 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:29.187609 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:29.129124  1337 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:28.889372 27979 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 10 to 11, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 11 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:29.193230 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:29.195148 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 68680.4us
I20260709 13:13:29.193758  1448 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:29.199982  1476 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 13:13:29.201973 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 195544us
I20260709 13:13:29.202294 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:29.136907  1046 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:29.200392  1152 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:29.191860 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:29.215533 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:29.218683 28158 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:29.222052 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:29.224438 28153 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:13:29.222981 28157 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:13:29.134918  1140 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.231031 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.231069 28234 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:29.214032  1455 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.279143  1455 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:29.279449 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:29.279639 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:29.280890  1046 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.234642 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.284003 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.285135 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.286095 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.286478  1424 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:29.287493  1424 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:29.287907  1424 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:29.271299  1355 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:29.231695 28231 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:13:29.194169  1438 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:29.282620  1480 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:13:29.224923 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:29.286885 28229 raft_consensus.cc:2410] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:13:29.201682  1350 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:29.281190 28084 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:29.297408 28161 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:29.298631 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:29.293102 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 101310us
I20260709 13:13:29.189177 28764 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:29.280951 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.189411  1479 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:29.291900 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:29.247213 28227 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:13:29.304191 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 261501us
W20260709 13:13:29.304584 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 259882us
W20260709 13:13:29.304924 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.308470 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.307482 28152 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:29.307610 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.307019 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.311094 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:29.315760 28755 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:29.319083 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.323318 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:29.323940 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.324354 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:29.190255  1471 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.306600  1042 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:29.330077  1042 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.330489  1042 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:29.224717  1321 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 13:13:29.333293  1321 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:29.333647  1321 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:29.334496  1149 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:29.334968  1149 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:29.335824  1149 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.281754  1254 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:29.341264  1465 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.615s	user 0.000s	sys 0.003s
I20260709 13:13:29.342024  1042 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.325878  1307 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:29.343971   846 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:29.278210  1303 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:29.301738  1438 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:29.282620  1140 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:29.308518 28227 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:13:29.323197 28230 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:29.275655  1270 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:29.277083  1320 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:29.325096 28050 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:29.313040  1479 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.348161 28761 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:29.345613  1485 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:13:29.350121 28227 pending_rounds.cc:85] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 7
I20260709 13:13:29.358791 28227 pending_rounds.cc:107] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:13:29.351295 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.344237  1042 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:29.204290  1435 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.359200 28227 raft_consensus.cc:2889] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:29.332149 27979 catalog_manager.cc:5697] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:29.354027  1116 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:13:29.360584  1469 consensus_queue.cc:1175] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "11f10c12c03e47dcac88b050d4c1431b" responder_term: 11 status { last_received { term: 11 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 11 index: 7 } }
I20260709 13:13:29.360904 28728 leader_election.cc:400] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19.
I20260709 13:13:29.334492  1321 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.358170  1479 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:29.379189 28728 leader_election.cc:403] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:29.362334   846 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:29.379881  1321 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:29.380505  1042 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19.
I20260709 13:13:29.379321  1116 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:29.382782  1116 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.368192 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:29.385413 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:29.355433 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.193343  1396 rpcz_store.cc:276] 0709 13:13:26.358402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.441135 (+ 82733us) service_pool.cc:224] Handling call
0709 13:13:26.896442 (+455307us) ts_tablet_manager.cc:934] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
0709 13:13:28.650425 (+1753983us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:13:28.842077 (+191652us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:28.842240 (+   163us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:13:28.880248 (+ 38008us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:13:28.880669 (+   421us) tablet_replica.cc:206] Creating consensus instance
0709 13:13:28.881121 (+   452us) ts_tablet_manager.cc:989] Replica registered.
0709 13:13:28.881210 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":26064,"tablet-copy.queue_time_us":376328,"thread_start_us":256136,"threads_started":2}
I20260709 13:13:29.355388 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:29.327934  1424 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.355350  1337 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:29.372390 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:29.373826 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:29.386420  1303 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:29.378201 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:29.367385 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:29.340603  1029 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.387255 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:29.402030 28229 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:13:29.402494   829 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.402809 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:29.385440 28161 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:29.375869  1046 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.371371  1149 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:29.380121 27979 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:29.392182   846 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:29.399592 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:29.413971 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:29.352876  1471 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:29.400800  1424 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:29.390686 28231 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:29.389984 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:29.404909 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:29.399077  1337 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:29.417959 28229 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:29.439478 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:29.400300 28230 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:29.425094 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:13:29.426163 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.406088  1303 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:29.442641  1337 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.406873  1417 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:29.448154 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.452049 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:29.456298 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:29.453670 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 63778.6us
I20260709 13:13:29.419845  1485 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:29.427170  1355 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:29.457005 28086 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:13:29.457687  1497 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:29.406638   829 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:29.460888 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.447350  1152 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:29.430146  1396 tablet_copy_client.cc:806] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:29.463761  1320 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:29.462615  1492 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:29.452538   792 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:13:29.468408  1029 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:29.468585 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:29.476377 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:29.470585 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.475931  1320 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.478519  1320 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:29.480453  1435 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:29.481204  1435 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:29.481556  1435 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.481879  1435 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:29.481957  1384 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:29.471603  1497 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:29.486117  1046 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:29.487023  1046 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:29.486515  1497 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.487439  1046 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:29.498342  1396 tablet_copy_client.cc:670] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:13:29.472438 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.503885 28231 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:29.504990 28227 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:13:29.466233  1355 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.515280 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.477836 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:29.517598 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:29.518342 28083 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:29.516268 28086 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:29.521893 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:29.528880 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 71474.5us
I20260709 13:13:29.530088 28050 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:29.509112  1046 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.517966  1492 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:29.538163  1492 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.513926  1355 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:29.558296 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 153432us
I20260709 13:13:29.547765 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 1133) took 1066 ms. Trace:
I20260709 13:13:29.559247 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:29.565356 28082 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:13:29.566314 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.569105 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.569169 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:29.565131 28226 rpcz_store.cc:276] 0709 13:13:28.467331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.467493 (+   162us) service_pool.cc:224] Handling call
0709 13:13:29.533692 (+1066199us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b5400391cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:29.533903 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354448256}
I20260709 13:13:29.573663 28078 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:29.573868 28083 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:29.565158 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.576138 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.572535 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:29.578049 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:29.580032 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:29.584796 28079 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:13:29.585224 28079 pending_rounds.cc:85] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 12
I20260709 13:13:29.585520 28079 pending_rounds.cc:107] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:13:29.585788 28079 raft_consensus.cc:2889] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:29.583792 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.587456 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:29.583997 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 107688us
I20260709 13:13:29.590351 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:29.525501  1161 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:13:29.622035 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.622933 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.594269 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:29.566947  1252 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:29.626878 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:29.632763 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 199344us
I20260709 13:13:29.633505 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:29.503240   846 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.635396 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:29.579690  1417 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:29.644117 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:29.626941  1464 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.946s	user 0.004s	sys 0.000s
I20260709 13:13:29.657967 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:29.658317  1505 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:29.668785 28078 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 12: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:29.566890  1320 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:29.525243  1303 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:29.586447  1350 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:29.659899 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:29.620615   829 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:29.717916 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:29.672375 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.629508  1451 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:29.665071 28755 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:29.794018 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 161169us
I20260709 13:13:29.795464 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:29.795717 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 51821.5us
I20260709 13:13:29.822325 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:29.825820 28233 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:29.797026 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:29.797807 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:29.829391 28080 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:29.803442 28084 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:29.636376  1152 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:29.629240  1485 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.171s
I20260709 13:13:29.834252 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2867) took 1100 ms. Trace:
I20260709 13:13:29.832526 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35896 (request call id 1064) took 1027 ms. Trace:
I20260709 13:13:29.581991  1455 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:29.814873 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 155076us
I20260709 13:13:29.865813 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:29.867594 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:29.858793 28756 rpcz_store.cc:276] 0709 13:13:28.805375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.811029 (+  5654us) service_pool.cc:224] Handling call
0709 13:13:29.832310 (+1021281us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b54006d01d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:29.832448 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11612032}
I20260709 13:13:29.857438 28232 rpcz_store.cc:276] 0709 13:13:28.733940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.734064 (+   124us) service_pool.cc:224] Handling call
0709 13:13:28.736361 (+  2297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.137135 (+400774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.833971 (+696836us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b54001c70d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:29.834180 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":858516864}
I20260709 13:13:29.851577 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:29.580626  1478 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.229s
I20260709 13:13:29.854393 28086 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:29.670734  1504 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:29.938506 28160 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:13:29.808111  1385 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:13:29.919121  1470 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:29.928130  1354 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:29.847082  1505 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:29.901952  1350 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:29.955179  1152 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:29.911674 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 60229.9us
W20260709 13:13:29.875824  1464 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:13:29.873663  1417 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:13:30.068758 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:29.956610  1384 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:30.068974 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:30.066092 28755 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
W20260709 13:13:30.078835 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 231113us
I20260709 13:13:30.082785 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:30.137837 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 55121.9us
I20260709 13:13:30.144923 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:30.148216   829 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:30.149663   829 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:30.151132   829 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:30.179704  1046 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:30.186465 28081 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:30.187412  1171 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:30.187863  1171 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:30.188858  1171 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.190073  1171 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:30.193621 28730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.039s	sys 0.155s
W20260709 13:13:30.194036 28730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.039s	sys 0.156s
W20260709 13:13:30.194509 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 560830us
I20260709 13:13:30.195227 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:30.079538  1485 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:30.081748  1350 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:30.278108 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 82939.6us
I20260709 13:13:30.282286 28730 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:30.090822  1504 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:30.279384 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 137048us
W20260709 13:13:30.199440  1464 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:13:30.288695 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:30.273357 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 190957us
I20260709 13:13:30.135004  1305 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:30.067406  1152 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.277087  1352 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:30.319916 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:30.278613  1396 tablet_copy_client.cc:538] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:13:30.314110 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:30.354025 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:30.287125  1126 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:30.101326  1511 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.232s
W20260709 13:13:30.359501 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 491966us
I20260709 13:13:30.286177  1199 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:30.307026  1385 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:30.311106  1270 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:30.314152  1513 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:30.338245  1504 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:30.311692  1500 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:30.327102  1496 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:30.374878 28234 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:13:30.417001  1173 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.045s	sys 0.052s
I20260709 13:13:30.218479  1501 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.113s
I20260709 13:13:30.367053 28198 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:30.309613  1350 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.364502   829 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:30.360298  1417 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:30.374338  1503 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 13:13:30.519899  1126 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:30.452214  1305 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:30.530320  1513 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:30.602355  1523 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:13:30.520159  1385 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:30.603080  1509 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.559s	user 0.001s	sys 0.000s
I20260709 13:13:30.461288  1508 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:30.522200  1321 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.142s	user 0.012s	sys 0.002s
I20260709 13:13:30.520139  1485 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:30.401533  1483 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:13:30.630697  1350 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:30.631412 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 269151us
I20260709 13:13:30.522467  1455 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:30.553423  1173 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.046s	sys 0.053s
I20260709 13:13:30.461380  1470 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.160s
W20260709 13:13:30.641459  1499 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.592s	user 0.001s	sys 0.000s
I20260709 13:13:30.459839  1352 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:30.476107  1199 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.519716  1270 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:30.622473  1500 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:30.633659  1417 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:30.753823 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 120229us
W20260709 13:13:30.637779 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 270836us
W20260709 13:13:30.701177  1355 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.120s	user 0.013s	sys 0.000s
I20260709 13:13:30.775332  1515 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:30.828943  1350 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:30.830319  1350 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.833587  1350 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:30.649889  1126 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:30.777583  1435 consensus_meta.cc:287] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.295s	user 0.012s	sys 0.000s
I20260709 13:13:30.640066  1521 raft_consensus.cc:2764] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:13:30.754992  1518 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.229s	user 0.000s	sys 0.000s
I20260709 13:13:30.835440  1435 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.839777  1126 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:30.826278  1355 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.849454 28230 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:30.842015  1435 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:30.723300  1453 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.854440  1453 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:30.866475  1355 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:30.877051 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 52775.9us
W20260709 13:13:30.820487  1384 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.735s	user 0.004s	sys 0.007s
I20260709 13:13:30.885248  1384 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:30.822863  1424 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.382s	user 0.012s	sys 0.000s
I20260709 13:13:30.797358  1463 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:30.735817  1508 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:30.739006  1485 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:30.903504  1438 consensus_meta.cc:287] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.546s	user 0.005s	sys 0.009s
I20260709 13:13:30.913532  1453 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:30.917956  1352 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:30.920948  1417 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:30.923240  1448 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.610s	user 0.005s	sys 0.007s
W20260709 13:13:30.930377  1454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.001s	user 0.035s	sys 0.065s
I20260709 13:13:30.923679  1500 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:30.930785  1454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.035s	sys 0.065s
I20260709 13:13:30.930763  1448 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:30.919425  1479 consensus_meta.cc:287] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.540s	user 0.010s	sys 0.001s
I20260709 13:13:30.926570  1485 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:30.927826  1508 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:30.903266  1463 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:30.920972  1471 consensus_meta.cc:287] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.483s	user 0.012s	sys 0.001s
I20260709 13:13:30.929198  1513 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:30.455564 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 122532us
I20260709 13:13:30.971410 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:30.975648 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:30.994024 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2340) took 1377 ms. Trace:
I20260709 13:13:30.994815 28083 rpcz_store.cc:276] 0709 13:13:29.616703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.617890 (+  1187us) service_pool.cc:224] Handling call
0709 13:13:29.618285 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.954437 (+336152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.954471 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:29.955153 (+   682us) log.cc:844] Serialized 26 byte log entry
0709 13:13:30.087452 (+132299us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:30.087490 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:30.087558 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:30.992596 (+905038us) raft_consensus.cc:1685] finished
0709 13:13:30.992677 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:30.993742 (+  1065us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23088,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":21308,"thread_start_us":467685,"threads_started":2,"wal-append.queue_time_us":355765}
I20260709 13:13:30.773615 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:31.012801 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 239086us
W20260709 13:13:31.030987 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 146003us
W20260709 13:13:31.036612 28758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.380s	user 0.033s	sys 0.029s
W20260709 13:13:31.037262 28758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.381s	user 0.033s	sys 0.029s
I20260709 13:13:31.038647 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 959) took 1536 ms. Trace:
I20260709 13:13:31.043788 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2339) took 1470 ms. Trace:
I20260709 13:13:31.044076 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.045552 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.039105 28758 rpcz_store.cc:276] 0709 13:13:29.502682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.502823 (+   141us) service_pool.cc:224] Handling call
0709 13:13:29.522999 (+ 20176us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.640781 (+117782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.640817 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:29.656097 (+ 15280us) log.cc:844] Serialized 26 byte log entry
0709 13:13:31.038051 (+1381954us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:31.038138 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.038270 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:31.038343 (+    73us) raft_consensus.cc:1685] finished
0709 13:13:31.038353 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.038545 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2016,"raft.queue_time_us":45119,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":7180,"thread_start_us":1496176,"threads_started":2,"wal-append.queue_time_us":1257279}
I20260709 13:13:31.044349 28081 rpcz_store.cc:276] 0709 13:13:29.573106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.573261 (+   155us) service_pool.cc:224] Handling call
0709 13:13:30.186269 (+613008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:30.186949 (+   680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.043217 (+856268us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54003aa7d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:31.043597 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1408944384}
W20260709 13:13:31.052228 28084 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.089s	user 0.007s	sys 0.006s
W20260709 13:13:31.052934 28084 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.089s	user 0.008s	sys 0.006s
I20260709 13:13:30.916435  1199 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:30.904377  1152 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:31.056396  1233 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.056901  1152 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.057426  1355 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.057708  1233 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.057965  1152 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.919409  1270 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:31.058459  1355 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.059610   792 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.061160  1350 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.061978 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 943) took 2447 ms. Trace:
I20260709 13:13:31.062471  1270 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.062278  1350 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.063661  1355 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:31.064515  1126 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:31.064584  1173 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.065824  1126 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.066311  1350 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.065958  1173 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.067251  1350 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.066874  1126 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.068850 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1071) took 1645 ms. Trace:
I20260709 13:13:31.068183  1350 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.053059 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.071419  1350 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:31.071633  1152 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:31.069456 28761 rpcz_store.cc:276] 0709 13:13:29.409171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.411396 (+  2225us) service_pool.cc:224] Handling call
0709 13:13:29.429468 (+ 18072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.488694 (+ 59226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.488744 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:29.489308 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:13:30.173459 (+684151us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b1800382f00. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:13:30.532470 (+359011us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:30.532575 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.054072 (+521497us) spinlock_profiling.cc:243] Waited 58.8 ms on lock 0x7b54004c6b54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:31.054146 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:31.054222 (+    76us) raft_consensus.cc:1685] finished
0709 13:13:31.054241 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.054529 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1389,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":7518,"spinlock_wait_cycles":148684544,"thread_start_us":358662,"threads_started":1,"wal-append.queue_time_us":1261904}
I20260709 13:13:31.072046  1350 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.070149  1173 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.073526  1126 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.074364  1126 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.073601  1173 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.060673   792 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.075307  1126 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.076364  1355 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.077176  1355 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.077692   792 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:31.075378  1173 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.923537  1385 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.103019  1449 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.104133  1449 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.932550  1471 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.925937  1438 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:30.930795  1396 tablet_bootstrap.cc:492] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:13:31.079002  1384 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:31.079720 28082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.013s	sys 0.012s
I20260709 13:13:31.079763  1417 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.081001  1355 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:13:31.081859  1453 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.083261  1463 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.063334  1270 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.122005 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2344) took 1160 ms. Trace:
I20260709 13:13:31.123327  1270 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.062506 28764 rpcz_store.cc:276] 0709 13:13:28.614295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.627480 (+ 13185us) service_pool.cc:224] Handling call
0709 13:13:28.627777 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.309421 (+681644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.061270 (+1751849us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54007008d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:31.061717 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176557824}
I20260709 13:13:30.932236  1479 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.125108  1417 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.127614  1479 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.092880  1199 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.130260 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.130730  1199 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:31.070377  1046 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.885s	user 0.012s	sys 0.000s
I20260709 13:13:31.081135 28125 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:31.118117 28085 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:31.078377  1126 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.123869  1492 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.120005 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2321) took 1892 ms. Trace:
I20260709 13:13:31.131372 28235 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:31.093688  1140 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:31.081483  1448 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.119956  1438 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:31.137277 28082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.014s	sys 0.012s
I20260709 13:13:31.125808 28077 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:13:31.142485 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:31.143154   846 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:31.145304  1384 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.146780  1385 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:31.147735  1453 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.149138  1463 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.150647  1449 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.165696  1463 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.157431  1199 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.164060  1453 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:31.169751 28080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.011s	sys 0.010s
I20260709 13:13:31.170047  1492 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.138303 28233 rpcz_store.cc:276] 0709 13:13:29.227549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.229019 (+  1470us) service_pool.cc:224] Handling call
0709 13:13:29.229364 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.826401 (+597037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.119143 (+1292742us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54003caad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:31.119728 (+   585us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370636160}
W20260709 13:13:31.177958 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 50179.7us
W20260709 13:13:31.181756 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 678208us
I20260709 13:13:31.190495 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:31.191588 28159 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:31.122591 28084 rpcz_store.cc:276] 0709 13:13:29.961502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.962295 (+   793us) service_pool.cc:224] Handling call
0709 13:13:29.962790 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.963095 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:31.053767 (+1090672us) log.cc:844] Serialized 14 byte log entry
0709 13:13:31.121312 (+ 67545us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:31.121442 (+   130us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:31.121554 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.121680 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.121910 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36649,"prepare.queue_time_us":760046,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":12778,"raft.queue_time_us":43010,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":1812,"thread_start_us":1117038,"threads_started":2}
I20260709 13:13:31.134291  1350 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.199523  1140 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.200428  1140 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.201409  1350 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.201828  1468 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.202445 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2345) took 1149 ms. Trace:
I20260709 13:13:31.202694  1468 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.202986 28082 rpcz_store.cc:276] 0709 13:13:30.052866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.053532 (+   666us) service_pool.cc:224] Handling call
0709 13:13:30.053825 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:30.054040 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:30.482249 (+428209us) log.cc:844] Serialized 14 byte log entry
0709 13:13:31.201859 (+719610us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:31.201962 (+   103us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:31.202067 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.202162 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.202334 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36441,"prepare.queue_time_us":676060,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":38543,"thread_start_us":1030601,"threads_started":2,"wal-append.queue_time_us":628584}
I20260709 13:13:31.205744  1140 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.205955  1173 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:31.208432  1171 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.018s	user 0.005s	sys 0.020s
I20260709 13:13:31.213724  1384 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.213776 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:31.216310 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.216751 28161 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:31.217630 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 136839us
I20260709 13:13:31.219203  1463 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:31.221287 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.226944 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:31.155501  1417 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:31.228138 28230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 2881) took 1756 ms (client timeout 1644 ms). Trace:
I20260709 13:13:31.231031 28233 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:31.221894 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:31.233305 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2545) took 1642 ms. Trace:
I20260709 13:13:31.228029  1492 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.220265 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:31.249343 28231 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:31.251976  1320 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.254292 28085 rpcz_store.cc:276] 0709 13:13:29.590523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.052187 (+1461664us) service_pool.cc:224] Handling call
0709 13:13:31.232995 (+180808us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b54003a98d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:31.233174 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85643648}
I20260709 13:13:31.230022  1468 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.267247  1438 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.268496  1438 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:31.253877 28230 rpcz_store.cc:269] 0709 13:13:29.471925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.477587 (+  5662us) service_pool.cc:224] Handling call
0709 13:13:31.227531 (+1749944us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400288054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:31.227856 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3064000896}
W20260709 13:13:31.267316 28234 consensus_meta.cc:287] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.691s	user 0.011s	sys 0.014s
I20260709 13:13:31.275646 28764 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:13:31.277364 28234 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:31.280304  1506 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:31.280930  1506 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.281332  1506 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:31.287701 28050 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.644s (SENT) [suppressed 5 similar messages]
I20260709 13:13:31.289597  1455 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:31.290118  1455 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:31.291587  1455 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.293619 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:31.295017  1417 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:13:31.297716  1526 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.396s	user 0.003s	sys 0.001s
I20260709 13:13:31.301743  1417 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:31.302263 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 80886.5us
I20260709 13:13:31.302747  1417 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.303422 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:13:31.303797  1484 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.958s	user 0.000s	sys 0.013s
I20260709 13:13:31.305570 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:31.305570 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2549) took 1681 ms. Trace:
I20260709 13:13:31.306452 28077 rpcz_store.cc:276] 0709 13:13:29.624160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.634144 (+  9984us) service_pool.cc:224] Handling call
0709 13:13:29.634442 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:31.169018 (+1534576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.305230 (+136212us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540027bfd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:31.305508 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3428081920}
I20260709 13:13:31.305788  1355 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:31.307507 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:31.307327  1417 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.308005  1355 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:31.308801  1384 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:31.309351 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 89219us
I20260709 13:13:31.310702  1233 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:31.310130  1455 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:31.306452 28084 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:31.319481 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.297840  1438 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:31.323685   829 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.012s
I20260709 13:13:31.325454 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.327690 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.330888 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.312055 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.331959 28084 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 9.
I20260709 13:13:31.313660 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2846) took 2949 ms. Trace:
I20260709 13:13:31.332800 28228 rpcz_store.cc:276] 0709 13:13:28.364210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.365495 (+  1285us) service_pool.cc:224] Handling call
0709 13:13:28.365903 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:28.618911 (+253008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:28.618967 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:28.627966 (+  8999us) log.cc:844] Serialized 26 byte log entry
0709 13:13:29.118610 (+490644us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:29.118669 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.118761 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:31.302539 (+2183778us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:31.313124 (+ 10585us) raft_consensus.cc:1685] finished
0709 13:13:31.313154 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.313470 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49168,"raft.queue_time_us":59856,"raft.run_cpu_time_us":1180,"raft.run_wall_time_us":18996,"spinlock_wait_cycles":11085056,"thread_start_us":693634,"threads_started":2,"wal-append.queue_time_us":621369}
I20260709 13:13:31.334231  1466 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.310863 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:31.335072  1466 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.336014  1486 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:31.336325 28764 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:31.336460  1486 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.337033  1486 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:31.337702 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:31.338210  1320 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.339177 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.339071  1320 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.339584 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.340534  1521 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:31.341192 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.341815  1471 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.342568 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:31.342705  1471 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.343319  1326 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.343463  1471 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.344081  1326 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.345461  1479 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.346140  1479 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.346766 28764 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:13:31.341336 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.347990 28755 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:31.348109  1449 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:31.348762 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 64131.7us
I20260709 13:13:31.348884  1449 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.350869  1493 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:31.353106  1467 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:31.353683  1467 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:31.355355 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44610.5us
I20260709 13:13:31.355080  1467 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.357996 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1184) took 1267 ms. Trace:
I20260709 13:13:31.359202 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.358412 28080 rpcz_store.cc:276] 0709 13:13:30.090782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.090978 (+   196us) service_pool.cc:224] Handling call
0709 13:13:30.216673 (+125695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:30.216982 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:13:30.526634 (+309652us) log.cc:844] Serialized 14 byte log entry
0709 13:13:31.294530 (+767896us) log.cc:844] Serialized 14 byte log entry
0709 13:13:31.357450 (+ 62920us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:31.357545 (+    95us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:13:31.357626 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.357756 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.357941 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140594,"prepare.queue_time_us":532005,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":116,"raft.queue_time_us":96266,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":195,"thread_start_us":875146,"threads_started":2,"wal-append.queue_time_us":774051}
I20260709 13:13:31.343755 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:31.381376 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.360206  1486 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.387017 28161 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:31.391710 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.396903  1320 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.397490  1449 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.399720  1451 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:31.401962  1494 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.402899  1494 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.403888  1453 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:31.404349  1453 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:13:31.406144 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 118654us
I20260709 13:13:31.409803 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.412859 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.387183  1467 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:31.418921  1453 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.420671  1453 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:31.423063  1149 consensus_meta.cc:287] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 2.010s	user 0.013s	sys 0.009s
I20260709 13:13:31.378168  1417 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:13:31.432204 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.432469 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:31.408401  1350 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:31.436614 28763 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:31.450894 28161 raft_consensus.cc:2410] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:13:31.452570  1466 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.453469  1486 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.460567   312 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.466851 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:31.467016 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 127719us
I20260709 13:13:31.463901   312 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.464859  1466 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.469583 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.469499  1466 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.472071  1466 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.473994 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:31.475013  1478 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:31.475142   312 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.475494  1478 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.475589  1494 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.476047  1478 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:31.475962   312 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:31.468593  1157 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.477795 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.479642 28233 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:31.479571  1157 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.483538  1157 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.478386 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.485807  1506 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.489081 28764 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:31.485895  1478 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.493284  1478 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.499437  1479 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:31.503479 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 66874.7us
I20260709 13:13:31.504819  1233 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:31.505434  1233 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:31.506451  1233 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.514550 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:31.515556 28234 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:31.517755 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:31.518142 28228 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:13:31.518570 28228 pending_rounds.cc:85] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 8
I20260709 13:13:31.518898 28228 pending_rounds.cc:107] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:13:31.519313 28228 raft_consensus.cc:2889] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:31.518935 28226 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:31.521039 28077 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:31.517807 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:31.523232 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.524802 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.525319 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.525564 28227 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:31.526132 28231 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:31.450541  1417 consensus_queue.cc:1243] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:13:31.524438 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.527015 28228 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:31.529925 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.530676 28154 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:31.530788 28232 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:31.531463 28154 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:31.532606 28154 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.534101 28154 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:31.535547 28229 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 7. (term mismatch)
I20260709 13:13:31.536183 28229 pending_rounds.cc:85] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 6
I20260709 13:13:31.536437 28229 pending_rounds.cc:107] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 15.7
I20260709 13:13:31.536700 28229 raft_consensus.cc:2889] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:31.536819 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.534293 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.537369  1352 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.531308  1326 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.543757  1352 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.544956 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.546062  1187 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:31.547031  1352 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.547399  1417 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:31.530692  1471 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.547941  1417 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:31.548413 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.549449  1417 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.549917 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.550572 28080 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:31.550658  1417 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:31.552057 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.553488 28234 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:31.563895 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.589159 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.590080 28229 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:31.590143 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:31.594851 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:31.595639  1470 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.596297  1470 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.506867  1506 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:31.595872 28161 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:31.605365 28154 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:31.607023 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:31.620469  1471 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:31.622685  1471 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.622790 28232 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:31.623188  1350 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:31.623380 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:31.623682 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.624011  1384 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.624989  1453 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.624701  1384 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.625593  1453 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.627125  1384 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.627630  1453 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.628051 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:31.628983 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.629590 28764 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:31.633013 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:31.642171 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:31.642926 28234 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:31.645092 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.623137  1471 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:31.633179  1451 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:13:31.658200 28229 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:31.661369 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:31.659983 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.672961 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:31.677387 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.681069 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:31.682070 28156 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
W20260709 13:13:31.673810 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 52308.6us
I20260709 13:13:31.673583   829 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:13:31.689594 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:31.697604 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:31.698024  1467 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:31.699553  1149 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:31.699992  1149 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.700379  1149 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:31.699139 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:31.703402  1453 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:31.703476  1491 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:31.704026  1453 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.704460  1491 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.704334  1326 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:31.705315  1491 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:31.704695  1479 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:31.706986 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:31.708101  1326 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.708554  1326 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:31.710839  1470 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:31.688232 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 98231us
I20260709 13:13:31.726325 28154 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:31.726408   312 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.733455 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:31.704618  1453 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:31.737768 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:31.739732 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:31.741074 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.744840 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:31.750171 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:31.758960 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:31.760249 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:31.760411  1455 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:31.761194  1320 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:31.761471  1510 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:13:31.762558  1494 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.762773  1466 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.764917  1492 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:31.797222  1492 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.797667  1492 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:31.799023 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.765630 28233 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:31.757648   312 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.773849 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.775089 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.768005 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:31.806218 28232 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:31.750248  1478 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:31.782784 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.783069 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.783067 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:31.787755 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.779314 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.806612 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.752827  1449 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:31.796344 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:31.810701 28235 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:31.819146 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:31.774785  1354 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.819976 28084 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:31.771554  1493 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.821318 28234 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:31.771112  1467 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:31.821318 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:31.824081 28161 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:13:31.807395  1384 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:13:31.833969 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 73823.5us
I20260709 13:13:31.816215  1478 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.835253 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:31.838074 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.839380 28152 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:31.854843 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:31.856078  1350 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.856772  1350 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:31.859210  1350 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:31.837296   846 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:13:31.862325 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.863368 28764 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:31.864409 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:31.866444 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 112572us
I20260709 13:13:31.806380  1494 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.867219 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:31.877081   312 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:31.827065  1466 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.848644  1467 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:31.843715  1478 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:31.870849  1479 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:31.849159  1233 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:13:31.903184 28761 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:31.929792  1479 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:31.930426  1479 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:31.854393  1157 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:13:31.931793  1157 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:31.932296  1157 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:31.902249 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.933486  1157 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.934484 28764 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:31.901654 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:31.935175  1157 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:31.911464 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:31.936827  1455 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:31.937438  1455 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.937929  1455 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:31.943599  1479 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:31.945514 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 50171.6us
I20260709 13:13:31.948181 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:31.890272 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.838691  1449 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:31.922233 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:31.925923 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:31.955559 28083 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:31.973523  1470 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:31.888604  1510 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:13:31.985749 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 136358us
I20260709 13:13:31.866102  1493 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:31.998716 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:31.999817 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 101704us
I20260709 13:13:32.001327  1479 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:32.001652 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.002264 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:32.004160  1553 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.270s	user 0.000s	sys 0.002s
I20260709 13:13:32.005482  1521 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:31.835405  1354 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.038414  1521 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:32.038872  1521 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:31.995021  1485 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.923720  1467 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.040658  1354 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:32.041277  1485 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.043984  1485 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:32.033207 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:32.030680 28157 raft_consensus.cc:2376] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18. Re-sending same reply.
I20260709 13:13:32.037118 28153 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:32.037410  1451 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.047776 28758 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
W20260709 13:13:32.051216 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 52651.5us
I20260709 13:13:32.053283 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:32.054801  1141 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:32.053246 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:32.060642 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:31.992987  1463 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:31.986528  1187 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.996874  1470 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:31.905570  1326 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.067654 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.068425 28049 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:32.056457  1141 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:31.968461  1384 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.092s
I20260709 13:13:32.039976  1521 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.074836 28157 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:32.060199  1354 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:32.077073 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.083434 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.072853  1470 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:32.099260 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.074177  1141 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.081751  1140 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:32.115237 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.124027 28157 raft_consensus.cc:2410] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 13:13:32.134791  1559 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.440s	user 0.004s	sys 0.002s
I20260709 13:13:32.138252 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.139112 28159 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:32.142020  1466 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:32.145751  1493 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:32.108932  1141 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:32.072301  1451 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:32.152011 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.097805  1187 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.111534  1140 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.163312 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.166168 28152 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:32.168164 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.170320 28154 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:32.171520 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.176333 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.178781 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.190598 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.193634 28160 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:32.177543 28226 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.087s	sys 0.214s
W20260709 13:13:32.194469 28226 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.088s	sys 0.214s
I20260709 13:13:32.196458 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.197435 28152 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:32.174110  1140 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:32.119681  1354 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
W20260709 13:13:32.203285 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 144866us
I20260709 13:13:32.203646 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.202226 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.178243  1521 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:32.206162 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:32.209208 28082 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:32.210443 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.167256  1478 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.216701 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:32.220839 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.200193  1493 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:32.165604  1149 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.231585 28080 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:32.231709 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 171165us
I20260709 13:13:32.213008  1494 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:32.233829 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:32.236796 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 268726us
I20260709 13:13:32.244304 28081 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:32.244568  1510 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:32.288640  1510 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.242978 28082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:32.318770 28158 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:32.319250 28082 raft_consensus.cc:606] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:32.319617 28082 raft_consensus.cc:614] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:32.320693  1562 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:13:32.321823 28082 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:32.291339  1326 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:32.241472  1493 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.306643  1384 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:13:32.358418  1492 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.000s
I20260709 13:13:32.358989  1492 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.361110  1451 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:32.355391  1491 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.005s
I20260709 13:13:32.361840  1491 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.362507  1485 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.363200  1485 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:32.368607  1453 consensus_meta.cc:287] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.559s	user 0.006s	sys 0.007s
I20260709 13:13:32.378069 28763 raft_consensus.cc:2368] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19.
W20260709 13:13:32.373178  1471 consensus_meta.cc:287] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.699s	user 0.009s	sys 0.004s
W20260709 13:13:32.375780  1417 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.825s	user 0.008s	sys 0.007s
I20260709 13:13:32.369768  1187 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:32.392112  1455 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.388219  1471 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.394402 28234 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:32.395023  1471 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:32.396380 28078 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:32.384889 28084 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.555s	user 0.007s	sys 0.000s
I20260709 13:13:32.398990 28084 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:13:32.404079 28153 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:32.409382 28160 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
W20260709 13:13:32.411242 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.379724  1453 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.428025 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:32.400765  1455 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:32.425637 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.430830 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.424831 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.431931 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:32.432564 28158 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:32.433408 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.434860 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:32.442224 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:32.446674 28082 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:32.441190 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:32.417346  1384 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:13:32.468257 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:32.477952 28161 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 10: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:32.433159  1149 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:32.485164 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.485698 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:32.486935 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.468849  1486 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:32.480072  1326 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:32.495913 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:32.495198  1566 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:32.497279  1506 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:32.483170  1485 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:32.502017 28050 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:32.504204 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:32.492063  1486 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:32.484052  1471 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:13:32.488430  1233 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:13:32.520012  1233 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:32.520542  1233 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:32.521561  1233 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.523185  1233 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:32.506821  1493 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:32.482939  1157 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:13:32.525986  1486 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.488574  1467 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:13:32.534116  1493 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:32.534839  1493 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:32.535929  1493 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.545747  1573 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:32.546376  1573 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:32.548897 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.531275 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.498034  1326 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:32.537848 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:32.538353 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.539788 28158 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
W20260709 13:13:32.540336  1561 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:13:32.555325 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.508412  1492 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:32.566552 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.527034  1141 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.555886   792 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:32.547739  1573 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:32.567911 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.565975 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2908) took 1009 ms. Trace:
I20260709 13:13:32.572953 28226 rpcz_store.cc:276] 0709 13:13:31.557060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.557208 (+   148us) service_pool.cc:224] Handling call
0709 13:13:31.587796 (+ 30588us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:31.620919 (+ 33123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:31.620952 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:31.650455 (+ 29503us) log.cc:844] Serialized 52 byte log entry
0709 13:13:32.195060 (+544605us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:32.195121 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.195219 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:32.565756 (+370537us) raft_consensus.cc:1685] finished
0709 13:13:32.565776 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:32.565919 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18614,"raft.queue_time_us":32893,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":535182,"threads_started":1,"wal-append.queue_time_us":583986}
W20260709 13:13:32.575767 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72029.8us
I20260709 13:13:32.576068  1573 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:32.577230 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:32.578449 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:32.579224  1463 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:32.566586  1449 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:32.580895 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.575842 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:32.583668 28226 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:32.564641 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0311501000001044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:13:32.580777 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.556229  1326 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:32.597872 28081 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: [16.12-17.13]
I20260709 13:13:32.598230 28084 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:13:32.598655 28084 pending_rounds.cc:85] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 12
I20260709 13:13:32.598935 28084 pending_rounds.cc:107] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:13:32.599200 28084 raft_consensus.cc:2889] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:32.582226  1486 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:32.601053  1320 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:32.601461  1453 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:32.601613  1320 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:32.602613  1320 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.603668  1320 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:32.606922 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.607862 28154 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:32.609647   792 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:32.586330 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.591447 28155 raft_consensus.cc:2368] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:32.613603 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.614283 28155 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:32.615151 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.616284 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.593312 28231 raft_consensus.cc:1217] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Deduplicated request from leader. Original: 14.8->[15.9-15.9]   Dedup: 15.9->[]
I20260709 13:13:32.617007 28231 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:13:32.621170 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:32.621570 28230 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:32.622046 28231 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:32.605662 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.601075 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.630650 28086 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:32.608641 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.633030 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.633229  1140 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.635058 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.636391 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:32.637823 28160 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:32.637238 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 135669us
I20260709 13:13:32.638620 28761 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:13:32.639921 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:32.629351 28152 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:32.647786 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.665936 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:32.667014 28152 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
W20260709 13:13:32.653774  1470 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.535s	user 0.005s	sys 0.006s
I20260709 13:13:32.680667 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:32.678766 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:32.683697 28227 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:32.684386  1570 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:32.685482 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:32.686416 28234 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:13:32.687045 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:32.692024  1563 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.355s	user 0.002s	sys 0.000s
I20260709 13:13:32.702991  1478 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:32.711544 28159 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
W20260709 13:13:32.711851 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72048.5us
W20260709 13:13:32.716233 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 144559us
I20260709 13:13:32.713172 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:32.720026 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50064.8us
I20260709 13:13:32.722409 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:32.736626 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:32.739061 28155 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:32.741412 28227 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:13:32.716225  1577 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:32.716137  1140 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:32.757825 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:32.756721 28758 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:32.758740 28758 raft_consensus.cc:606] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:32.758989 28758 raft_consensus.cc:614] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:32.761703  1574 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:13:32.763819 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.726488  1506 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:32.727617  1478 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:32.771734 28234 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:32.774266 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:32.775075 28234 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:32.681705  1470 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:32.684160  1514 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:13:32.737648  1453 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:32.764402  1417 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:13:32.796013  1514 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:32.772878  1478 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:32.837010  1478 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:32.846866  1470 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:32.857990  1463 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:32.801851  1577 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:13:32.862041  1141 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:32.803608  1453 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:32.865151  1453 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:32.805467 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1181) took 1091 ms. Trace:
I20260709 13:13:32.871721 28229 rpcz_store.cc:276] 0709 13:13:31.714382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.714516 (+   134us) service_pool.cc:224] Handling call
0709 13:13:31.714820 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:31.737360 (+ 22540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:31.737392 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:31.783868 (+ 46476us) log.cc:844] Serialized 26 byte log entry
0709 13:13:32.193244 (+409376us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:32.193305 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.193395 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:32.691118 (+497723us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:32.798143 (+107025us) raft_consensus.cc:1685] finished
0709 13:13:32.798181 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:32.805346 (+  7165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8425,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7425,"spinlock_wait_cycles":10409216,"thread_start_us":376928,"threads_started":1,"wal-append.queue_time_us":455283}
I20260709 13:13:32.875743 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:32.881775  1587 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:13:32.804783  1514 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:33.208410 28226 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.000s
I20260709 13:13:33.208957 28226 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:33.152320  1521 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:33.213610 28231 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:33.155726 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:33.003850 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 199123us
W20260709 13:13:33.272352 28729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.027s	sys 0.028s
W20260709 13:13:33.272866 28729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.028s	sys 0.028s
I20260709 13:13:33.274463 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:33.275183 28226 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.206710  1514 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:33.285490  1514 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:33.301235 28227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.156s	sys 0.246s
W20260709 13:13:33.302462 28227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.156s	sys 0.246s
W20260709 13:13:33.311894 28234 consensus_meta.cc:287] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.001s
I20260709 13:13:33.275717   829 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:33.285532  1320 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.653s	user 0.003s	sys 0.007s
I20260709 13:13:33.316718 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1066) took 1177 ms. Trace:
I20260709 13:13:33.325310 28156 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:33.332290 28234 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:33.334649 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.345135 28159 rpcz_store.cc:276] 0709 13:13:32.139294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.139875 (+   581us) service_pool.cc:224] Handling call
0709 13:13:32.711355 (+571480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:32.712085 (+   730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.316460 (+604375us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b54000802d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:33.316618 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1313299840}
W20260709 13:13:33.354910 28232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.025s	sys 0.055s
W20260709 13:13:33.355616 28232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.026s	sys 0.055s
I20260709 13:13:33.316123  1478 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:33.359639 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.364331 28226 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:33.364867 28234 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:33.372062 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.372850 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.382131 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:33.329442  1521 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:33.397666 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.350921   829 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:33.398488 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.410043  1589 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.410702 28234 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:33.410799  1589 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:33.375145  1233 consensus_meta.cc:287] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.841s	user 0.013s	sys 0.000s
I20260709 13:13:33.399355 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:33.400655 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.413232 28228 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:33.413465 28226 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:33.414693 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.415912 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.416791 28226 raft_consensus.cc:2410] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:33.417639 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:33.418176 28226 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:33.421720 28228 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:33.423022 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.423878 28228 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:33.412205 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.425879 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.427330 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.428241 28234 raft_consensus.cc:2410] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 17 because replica has last-logged OpId of term: 16 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 13:13:33.431859 28234 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:33.432657  1478 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:33.432898 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.434927 28234 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:33.435789   846 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.009s	sys 0.030s
W20260709 13:13:33.448406   846 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.009s	sys 0.030s
W20260709 13:13:33.447899  1326 consensus_meta.cc:287] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.854s	user 0.014s	sys 0.000s
I20260709 13:13:33.462643 28234 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:33.458976 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.470798 28226 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
W20260709 13:13:33.481339 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.481802  1593 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:33.482374  1593 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:33.486491  1521 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:33.487141  1521 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:33.465816  1326 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:33.494731 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:33.495697 28761 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:33.496111  1491 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:33.503072 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 322674us
I20260709 13:13:33.484544  1593 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:33.504894 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:33.506031 28235 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:33.515935 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:33.516004 28761 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:33.509325  1589 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:33.523773 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:33.561745 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:33.562501 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 60104.1us
I20260709 13:13:33.562690  1593 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:33.560385  1577 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:33.563302  1593 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:33.571832 28235 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
W20260709 13:13:33.573129 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49471.3us
I20260709 13:13:33.575094 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.574254 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:33.568696  1593 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:33.618430 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.576822  1577 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:33.587358  1141 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:33.608498  1326 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:33.640875 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:33.636626  1141 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:33.647267  1141 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:33.653735  1577 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:33.635419  1600 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:33.657152  1600 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:33.659591  1600 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:33.660199  1600 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:33.663239  1564 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.298s	user 0.011s	sys 0.013s
I20260709 13:13:33.705529 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1085) took 1211 ms. Trace:
I20260709 13:13:33.712209  1605 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:13:33.713920 28156 rpcz_store.cc:276] 0709 13:13:32.493755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.507233 (+ 13478us) service_pool.cc:224] Handling call
0709 13:13:33.705166 (+1197933us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54000a28d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:33.705391 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":873895168}
I20260709 13:13:33.676287  1600 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:33.717681  1600 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:33.704536  1593 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:33.721444  1593 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:33.735491  1396 tablet_bootstrap.cc:492] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:33.736574  1396 tablet_bootstrap.cc:492] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:13:33.737699  1396 ts_tablet_manager.cc:1403] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 2.807s	user 0.056s	sys 0.032s
I20260709 13:13:33.741986  1396 raft_consensus.cc:359] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:33.743041  1396 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:13:33.744521  1396 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:33.746948  1396 ts_tablet_manager.cc:1434] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:13:33.767525 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2362) took 1984 ms. Trace:
I20260709 13:13:33.768191 28079 rpcz_store.cc:276] 0709 13:13:31.782903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.795591 (+ 12688us) service_pool.cc:224] Handling call
0709 13:13:32.741196 (+945605us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:13:32.756811 (+ 15615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:33.764130 (+1007319us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:13:33.765885 (+  1755us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.766950 (+  1065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:33.767005 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:33.767052 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.767125 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:33.767272 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1433,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"spinlock_wait_cycles":26856832,"thread_start_us":1486,"threads_started":2,"wal-append.queue_time_us":1019}
I20260709 13:13:33.786602  1608 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:33.787590  1608 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:33.790587  1608 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:33.831687  1600 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:33.832177  1600 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:33.832985  1600 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:33.833949  1600 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:33.841313   829 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:33.847939  1612 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.849246  1612 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:33.851894  1612 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:33.847939 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:33.856266  1612 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:33.856436 28235 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:33.860065  1612 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:33.861181  1612 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:33.863734 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2346) took 1240 ms. Trace:
I20260709 13:13:33.864283 28231 rpcz_store.cc:276] 0709 13:13:32.622924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.623573 (+   649us) service_pool.cc:224] Handling call
0709 13:13:33.213315 (+589742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:33.222008 (+  8693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.863480 (+641472us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54003bfbd0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:33.863659 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1304641920}
I20260709 13:13:33.875025 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:33.876612 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:33.877414 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.877871 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:33.882443  1612 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:33.896133 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2922) took 2010 ms. Trace:
I20260709 13:13:33.896322  1612 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:33.896632 28233 rpcz_store.cc:276] 0709 13:13:31.886096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.886340 (+   244us) service_pool.cc:224] Handling call
0709 13:13:31.929178 (+ 42838us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:31.929693 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:31.929722 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:31.930406 (+   684us) log.cc:844] Serialized 26 byte log entry
0709 13:13:32.200909 (+270503us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:32.200966 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.201060 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:33.662481 (+1461421us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:33.895791 (+233310us) raft_consensus.cc:1685] finished
0709 13:13:33.895813 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:33.896068 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14223,"raft.queue_time_us":30302,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":14523,"spinlock_wait_cycles":7046400,"thread_start_us":270025,"threads_started":1,"wal-append.queue_time_us":363045}
I20260709 13:13:33.903072  1612 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:33.903901  1612 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.912987 28086 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:33.909672 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:33.913932 28080 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:13:33.910777 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:33.905279 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:33.916162  1613 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:33.917857 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:33.918677 28077 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:33.919595 28079 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:33.920581 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 1.20297e+06us
I20260709 13:13:33.921401 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:33.926285 28080 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:33.922219  1612 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:33.928865  1615 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:33.935227  1616 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:33.938366  1616 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:33.944177  1616 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:33.940291  1615 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:33.952278  1618 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:33.953469  1615 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:33.951694 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:33.960669 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:33.963413 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 58334.1us
I20260709 13:13:33.967566 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:33.970181 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:33.969017  1619 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:33.969667  1613 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:33.972995  1613 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:33.957114  1618 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:33.980639 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2943) took 1104 ms. Trace:
W20260709 13:13:33.981644 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 56955.4us
I20260709 13:13:33.981971 28234 rpcz_store.cc:276] 0709 13:13:32.876322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.470576 (+594254us) service_pool.cc:224] Handling call
0709 13:13:33.474965 (+  4389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:33.475209 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:33.633073 (+157864us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.980247 (+347174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:33.980305 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:33.980363 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.980438 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:33.980577 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":167477,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":115,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":470861,"threads_started":2,"wal-append.queue_time_us":75419}
I20260709 13:13:33.985759 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:33.985924 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:33.993822 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.986720  1619 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:33.995293  1619 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:33.988687  1449 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:33.995869  1619 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:33.996147  1449 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:33.996821  1449 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:33.997015  1619 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.002461 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:33.984565  1617 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:34.010111 28085 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:34.013231 28127 connection.cc:256] RPC call timeout handler was delayed by -1.1495236460000342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
I20260709 13:13:34.014585  1600 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.002815  1623 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:34.017098  1607 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.017441  1600 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.017797  1607 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:34.015623 28127 connection.cc:256] RPC call timeout handler was delayed by -1.0829236459999265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
I20260709 13:13:33.979712  1618 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.015316  1608 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.026449 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.019764  1605 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.031104  1605 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.032130  1616 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.033772  1605 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.034657  1605 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:34.035152  1616 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:34.020334  1617 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:34.019237  1615 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:13:34.048126 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.065217 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2347) took 1441 ms. Trace:
I20260709 13:13:34.069687  1629 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:34.072629  1629 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.051685  1607 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.053151  1600 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:34.043776  1608 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:34.086351 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.035182  1605 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:34.086855 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.092211  1608 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.094007  1593 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:34.094872  1593 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.096767  1617 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.097909  1593 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:34.099179  1593 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:34.100214  1569 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.738s	user 0.004s	sys 0.002s
I20260709 13:13:34.099836  1593 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.102839  1593 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:34.103180  1612 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:34.106364 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:34.106938 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.106179 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:34.095098  1449 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.111403  1600 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:34.110468 28125 connection.cc:256] RPC call timeout handler was delayed by -1.1027754050001022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
I20260709 13:13:34.106914  1593 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:34.112083  1600 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.112761  1593 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.113811  1605 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:34.114431  1600 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.114943  1593 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:34.111944  1612 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.116904  1449 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:34.119445  1608 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.124257 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:34.130914 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.133661 28159 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:34.134024 28226 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:34.128474 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.129489 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.134961 28156 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:34.129532 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:34.129917 28234 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:13:34.130298 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:34.135993 28227 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:34.130431  1605 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:34.130934 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:34.135092 28155 raft_consensus.cc:2368] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:34.129748 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.138238 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:34.139199 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:34.139779 28233 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:34.066661 28230 rpcz_store.cc:276] 0709 13:13:32.623308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.623504 (+   196us) service_pool.cc:224] Handling call
0709 13:13:32.645421 (+ 21917us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:32.650549 (+  5128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:32.650598 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:32.669740 (+ 19142us) log.cc:844] Serialized 26 byte log entry
0709 13:13:32.759598 (+ 89858us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:34.012672 (+1253074us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b1800409a80. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:13:34.044655 (+ 31983us) log.cc:844] Serialized 14 byte log entry
0709 13:13:34.044916 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.044993 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:34.045034 (+    41us) raft_consensus.cc:1685] finished
0709 13:13:34.064959 (+ 19925us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:34.065117 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":873903,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":782,"raft.queue_time_us":778,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":11592,"spinlock_wait_cycles":14510976,"thread_start_us":106396,"threads_started":2,"wal-append.queue_time_us":77068}
I20260709 13:13:34.138887 28232 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:34.152271  1593 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.152961  1593 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.154292  1605 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.155140  1593 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:34.155889 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.157099 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.157711 28158 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:34.120239  1623 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:34.147562 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:34.172924 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.131147  1600 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:34.179409 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:34.180315 28232 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:34.146376  1608 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.183636  1608 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.184101 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:34.184746 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.185567 28155 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:34.188856  1352 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.189802  1352 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.190006  1600 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.192854  1352 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.193070  1600 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:34.193435 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.194011  1608 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:34.194562  1608 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.195022  1608 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:34.195775 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.196131 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.196513 28077 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:34.197214  1600 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:34.197394 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.198709 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.200145 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.195390 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:34.202852 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.203833 28080 raft_consensus.cc:2410] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:13:34.204309 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.205621  1629 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:34.207214 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.195250 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.210213 28085 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:34.209731 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 206364us
I20260709 13:13:34.211975 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.213869 28079 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:34.217234 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:34.218619 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.219709 28127 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:34.221501  1600 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:34.222236  1600 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:34.222589  1600 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.222967  1600 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:34.224907  1607 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:34.225801  1607 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:34.226639  1521 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:34.231926 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.232796 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.233560 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:34.233980 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.234279 28077 raft_consensus.cc:2376] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17. Re-sending same reply.
I20260709 13:13:34.233594 28080 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:34.239377  1600 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.241031 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.242336 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.242558 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:34.243259 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:34.248476 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.253633 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:34.264627 28232 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:34.256621 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:34.266682 28233 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:34.269151  1611 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.270740 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:34.271355 28228 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:34.257704 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.273249 28155 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:34.277688 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.278812  1600 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.210739  1629 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:34.282382 28079 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:34.266709 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.236060  1608 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.304378 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:34.305392  1623 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:34.305879  1623 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.306262  1623 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:34.327461 28233 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:34.269755  1611 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:34.336444 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 116796us
I20260709 13:13:34.337728 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:34.337904 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.338590 28155 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:34.340193 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 167347us
I20260709 13:13:34.341602 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.341732 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.342561  1629 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:34.343600  1629 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:34.344712 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.345461 28156 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:34.346028 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.327972  1605 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:34.350503 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:34.351260 28228 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:34.354023 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:34.360070  1635 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:34.360540  1635 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.360867  1521 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:34.360903  1635 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:34.352157  1600 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:13:34.362186  1611 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.366534  1600 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:34.367000  1600 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:34.367817  1600 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.369338  1600 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:34.370151 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.370842 28155 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
W20260709 13:13:34.371752 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.371407  1605 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:34.379631 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:34.383176 28228 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:34.387307 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:34.389186 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:34.390573  1617 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:34.391189 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:34.392446  1633 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:34.385110  1635 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.372601 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.397398 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:34.410754 28161 raft_consensus.cc:2410] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:13:34.410768 28156 raft_consensus.cc:2410] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:13:34.351081  1629 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:34.432410 28755 raft_consensus.cc:1217] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 14 LEARNER]: Deduplicated request from leader. Original: 14.10->[14.11-14.11]   Dedup: 14.11->[]
I20260709 13:13:34.427037 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:34.430611 28158 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:34.432502 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:34.431684 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:34.459141 28758 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:34.457981 28763 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.432883 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:34.459985 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.461092 28761 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:13:34.461982 28227 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:34.393814  1617 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:34.458468 28764 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:34.411257  1605 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:34.480440 28049 connection.cc:256] RPC call timeout handler was delayed by -1.1394713590000265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.169s for a response
I20260709 13:13:34.481128 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:34.481698 28229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.011s	sys 0.029s
I20260709 13:13:34.482002 28158 raft_consensus.cc:2410] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:13:34.467279 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:34.470198 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:34.475081 28096 tablet_copy_service.cc:342] P 0faaace2b36c4bcebf2fa6835235e92b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-5c654c99561449cb823642a1b85879ac received from {username='slave'} at 127.0.0.1:36768
I20260709 13:13:34.477267 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:34.482075 28229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.011s	sys 0.030s
I20260709 13:13:34.491492 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:34.467721 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:34.484601 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.423708  1633 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:34.499477 28049 leader_election.cc:400] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19.
I20260709 13:13:34.515645 28049 leader_election.cc:403] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:34.499861 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:34.498625  1605 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.506062 28082 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:34.508459 28096 tablet_copy_service.cc:434] P 0faaace2b36c4bcebf2fa6835235e92b: ending tablet copy session 70996712c3c942468a6006602de6477f-5c654c99561449cb823642a1b85879ac on tablet 5c654c99561449cb823642a1b85879ac with peer 70996712c3c942468a6006602de6477f
I20260709 13:13:34.511523 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.552459 28155 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:34.513952 28161 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:13:34.552955 28161 pending_rounds.cc:85] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 8
W20260709 13:13:34.553233 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 165988us
I20260709 13:13:34.553242 28161 pending_rounds.cc:107] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:13:34.553990 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:34.554143 28161 raft_consensus.cc:2889] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:34.556185 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:34.556831 28158 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:34.559594 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:34.568451 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:34.546262  1521 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:34.570387 28199 leader_election.cc:400] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:34.570813 28199 leader_election.cc:403] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:34.526705  1613 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:13:34.576071 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 2945) took 1423 ms. Trace:
I20260709 13:13:34.578449 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:34.524818 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:34.542306  1514 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
W20260709 13:13:34.528134 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 46567.2us
I20260709 13:13:34.578524 28229 rpcz_store.cc:276] 0709 13:13:33.153147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.473380 (+320233us) service_pool.cc:224] Handling call
0709 13:13:33.473779 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:33.475643 (+  1864us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:33.629829 (+154186us) log.cc:844] Serialized 14 byte log entry
0709 13:13:34.575726 (+945897us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:34.575774 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:34.575818 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.575904 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:34.576017 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33048,"prepare.queue_time_us":167367,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"thread_start_us":1058936,"threads_started":3,"wal-append.queue_time_us":85481}
I20260709 13:13:34.531442 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:34.532083 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:34.532544 28152 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:34.530680  1615 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19.
I20260709 13:13:34.586220 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:34.586859 28152 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:34.490674  1607 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:34.544842 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:34.591241 28156 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:34.498734  1569 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:13:34.584748 28159 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 12: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:13:34.596364 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 111810us
I20260709 13:13:34.590497 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.597258 28155 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:34.609539 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.611033 28155 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:34.614162 28155 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:34.576344  1521 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.607291 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:34.597838 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:34.580114  1593 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:34.619982 28159 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:34.627852 28159 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:34.623669  1521 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:34.580952  1608 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.569409  1233 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:13:34.649647 28156 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:34.619849 28155 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:13:34.629424 28159 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.596822  1644 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.661566 28159 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:34.653661 28158 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
W20260709 13:13:34.653182 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 69645.2us
W20260709 13:13:34.657485 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 110474us
I20260709 13:13:34.617892  1630 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:34.632570  1352 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.677119 28161 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
W20260709 13:13:34.682165 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 74756.4us
I20260709 13:13:34.658663  1608 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:34.686394 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 120533us
I20260709 13:13:34.580000  1633 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.687704 28156 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:34.683729 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:34.582299  1617 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.699563 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:34.662345  1643 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:34.679176  1635 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:34.665155  1354 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:13:34.714692 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:34.722831 28158 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:34.678720  1651 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 12. Current term is 13.
I20260709 13:13:34.728559 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:34.727348 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:34.741338 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:34.742985 28159 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:34.743021 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:34.718734 28182 debug-util.cc:398] Leaking SignalData structure 0x7b08002dbe20 after lost signal to thread 26962
W20260709 13:13:34.747133 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 162719us
W20260709 13:13:34.749755 28182 debug-util.cc:398] Leaking SignalData structure 0x7b0800233020 after lost signal to thread 28018
W20260709 13:13:34.751740 28182 debug-util.cc:398] Leaking SignalData structure 0x7b08001337c0 after lost signal to thread 28110
I20260709 13:13:34.752301 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:13:34.738083  1647 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:13:34.736806  1642 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:13:34.714221  1643 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.777127 28234 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:13:34.777144 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.737659  1605 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:34.726143  1649 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:34.676124  1644 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.706853  1608 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.729370  1654 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:34.744700  1635 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:34.774175  1611 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:34.802294 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49907.3us
I20260709 13:13:34.687043  1352 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.809762 28763 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:34.786298  1651 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:34.786155  1643 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:34.758052  1657 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:13:34.824681 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 72455.5us
I20260709 13:13:34.824941 28764 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
W20260709 13:13:34.824695 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 83120.9us
W20260709 13:13:34.824867 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 81928.4us
I20260709 13:13:34.813104 28050 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:34.826707 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:34.809309 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 211724us
I20260709 13:13:34.803599  1649 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.827908  1649 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:34.818639  1653 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:34.829515  1656 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:34.819664  1651 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:34.830936  1651 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:34.834573  1352 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:34.842439  1644 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:34.846635  1645 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:34.847314  1645 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:34.847699  1645 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:34.848675  1650 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:34.850389  1608 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:34.856297  1623 consensus_meta.cc:287] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.549s	user 0.004s	sys 0.005s
W20260709 13:13:34.810277 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 252006us
I20260709 13:13:34.744876   846 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:13:34.807615  1635 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.857023  1613 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:34.857666  1660 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:34.818115  1611 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.821476  1657 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:34.859195  1605 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:13:34.861783 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.856441  1645 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:34.788992  1449 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:34.867231 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:34.825922 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:34.924623 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 57459.7us
I20260709 13:13:34.924822 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.867586 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:34.886389 28152 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:34.925729 28153 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:34.901129  1593 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:34.929008 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:34.929247  1593 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:34.930306  1644 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:34.907035 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1259) took 1577 ms. Trace:
I20260709 13:13:34.920648  1611 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:34.912567  1618 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:34.913020  1664 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:34.953421 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 85875us
I20260709 13:13:34.887676  1623 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:34.916114  1641 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.443s	user 0.002s	sys 0.000s
I20260709 13:13:34.909544  1636 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.911463  1635 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:34.891691 28182 debug-util.cc:398] Leaking SignalData structure 0x7b08000c3680 after lost signal to thread 28185
W20260709 13:13:34.962909 28182 debug-util.cc:398] Leaking SignalData structure 0x7b08001c00a0 after lost signal to thread 28186
W20260709 13:13:34.964249 28182 debug-util.cc:398] Leaking SignalData structure 0x7b08003ac8e0 after lost signal to thread 28259
W20260709 13:13:34.966001 28182 debug-util.cc:398] Leaking SignalData structure 0x7b080019b080 after lost signal to thread 28789
I20260709 13:13:34.928119  1605 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:34.928709 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:34.969471 28156 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:34.971057  1617 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:34.842718  1649 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:13:34.936285 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47649.8us
I20260709 13:13:34.911271  1521 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:34.920516  1613 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:34.923522  1608 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:34.926987  1449 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:34.959787 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 133969us
I20260709 13:13:34.932156 28756 rpcz_store.cc:276] 0709 13:13:33.318854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.319956 (+  1102us) service_pool.cc:224] Handling call
0709 13:13:33.320408 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:33.444770 (+124362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:33.444816 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:33.455537 (+ 10721us) log.cc:844] Serialized 26 byte log entry
0709 13:13:33.606421 (+150884us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:33.606475 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.896173 (+1289698us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54004c5ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:34.896259 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:34.896331 (+    72us) raft_consensus.cc:1685] finished
0709 13:13:34.896351 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:34.896701 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3017,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":2493,"spinlock_wait_cycles":281368448,"thread_start_us":148410,"threads_started":1,"wal-append.queue_time_us":226906,"wal-append.run_cpu_time_us":2316,"wal-append.run_wall_time_us":1017736}
I20260709 13:13:34.985893 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:34.921396  1657 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:34.961391 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:34.968153 28182 debug-util.cc:398] Leaking SignalData structure 0x7b080023d7c0 after lost signal to thread 28790
I20260709 13:13:34.971565  1617 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:34.971793  1661 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:34.976545 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 163558us
I20260709 13:13:34.969096  1605 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:34.955348  1644 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:34.980717 28230 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:35.007398 28156 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:34.941682  1618 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:34.956557  1660 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:34.991258 28182 debug-util.cc:398] Leaking SignalData structure 0x7b08003d65c0 after lost signal to thread 1557
I20260709 13:13:34.994481 28049 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:35.001433 28229 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:13:34.994277 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:34.977726 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 88922.1us
I20260709 13:13:34.988286  1449 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:34.983115  1608 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:34.991732  1352 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.978056  1320 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:13:34.997404  1617 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:34.996008  1669 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:13:35.053120 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 77163.9us
I20260709 13:13:35.054550 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:35.009481  1646 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:34.957685  1664 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:35.057173 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 62763.8us
I20260709 13:13:35.046504 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:34.974311  1636 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.000129  1600 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:13:34.995846  1643 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.056581  1141 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:35.057767  1660 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:35.058660  1646 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:35.056619  1352 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.089907  1666 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:35.093389  1666 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.095453  1655 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:13:35.092000  1663 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:35.100939  1608 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.087037  1660 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:35.102983  1660 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:35.082078  1669 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:35.105443  1623 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:35.006152  1661 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.106110  1636 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:35.107285  1678 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
W20260709 13:13:35.116946 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 56757.7us
W20260709 13:13:35.117494 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 123276us
I20260709 13:13:35.119452   792 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:35.119933   792 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:35.087250  1618 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.122469  1618 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:35.087287  1646 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:35.106123  1671 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 13:13:35.126709 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 66397.8us
I20260709 13:13:35.128697 28081 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:35.129729 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:35.130620 28235 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:35.122326  1660 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.133905 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:35.136541 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.138077 28156 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:35.139755 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.141016 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.142108 28077 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:35.142380 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:35.142735 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.143008 28232 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:35.144048 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:35.135293 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.136677 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:35.145680 28235 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:35.144954 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:35.148047 28078 raft_consensus.cc:2368] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18.
I20260709 13:13:35.152422 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:35.163183 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:35.168767  1661 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:35.169306  1674 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:35.169904  1674 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.172348  1674 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:35.172562  1449 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.108071  1613 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.165331  1666 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:35.131544  1644 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.167514  1649 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:35.173067  1674 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:35.225183 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.164201  1664 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:35.177580 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:35.226509 28235 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:35.185720 28228 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:35.197091 28232 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:35.178793 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.227945 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.198916 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:35.131295  1645 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:35.223143 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 164513us
I20260709 13:13:35.228345 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.231137 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.231719 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:35.232442 28152 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:13:35.232185 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:35.234798 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:35.235605 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.236447 28758 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:13:35.175392  1449 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:35.238780 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.245471 28156 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:35.249429 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.183430  1521 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:35.178081  1669 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:35.252305 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.252663 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:35.260324 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.261297 28154 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:35.253121 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 74449.4us
I20260709 13:13:35.256170 28756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:35.262441 28756 raft_consensus.cc:606] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:35.262971 28756 raft_consensus.cc:614] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:35.263761 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.266357 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.269608 28152 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:35.273674 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.274367 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.241485  1666 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:35.273849  1645 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:35.291590 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:35.286646 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49014.9us
I20260709 13:13:35.142838  1605 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.295660 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.258096  1654 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.296895 28235 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:35.268175  1685 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:35.298440 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.309842 28152 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:35.311574 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:35.312803 28156 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 17: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:35.302102 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:35.317162 28235 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:35.322835 28233 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:35.324796  1654 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.329803  1669 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.333798  1660 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:35.296221  1655 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:13:35.338840  1643 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:35.336141  1657 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.238178  1674 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.347561 28083 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:35.301122  1649 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:35.342139  1449 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:35.341292  1650 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.380378 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.324560 28233 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:35.349659 28758 raft_consensus.cc:2410] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:13:35.315246 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:35.387075 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 71929.7us
I20260709 13:13:35.389667  1660 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:35.390507  1660 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:35.391224  1660 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:35.380456  1649 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.397523  1449 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:35.398077  1449 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:35.398882  1449 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.353272  1521 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:35.370182 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.400370  1521 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.376403 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.378511 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:35.399458  1617 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.407084 28154 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:35.410208 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.412827  1605 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:35.414526 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.415616  1645 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:35.377651  1352 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:35.391343  1657 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:35.418057  1646 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:35.420922  1637 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.886s	user 0.008s	sys 0.000s
I20260709 13:13:35.420970  1611 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.423098  1635 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.423826  1611 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:35.423107 28082 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:35.427042 28085 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 10.
I20260709 13:13:35.428823 28083 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:13:35.429287 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.385043  1650 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:35.396457  1649 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:35.445843 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 210276us
I20260709 13:13:35.438401 28228 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:35.435202 28235 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:35.447779 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.446910 28085 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:35.452009 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.354760  1623 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:13:35.453961 28078 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:35.454077 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.455710 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.456969 28083 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:35.454070 28077 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
W20260709 13:13:35.449180  1633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.016s	sys 0.027s
I20260709 13:13:35.467618 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.405709  1593 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.449631  1657 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.452495  1685 raft_consensus.cc:2764] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:13:35.472931 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.478631 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.456786  1491 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:35.475003 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:35.467092  1633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.016s	sys 0.027s
I20260709 13:13:35.474988 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:35.481837 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.466070  1649 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:35.482399 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:35.482548 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:35.496032 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.496134 28232 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:35.469980  1521 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:35.477546 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 156417us
I20260709 13:13:35.469740  1645 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:35.389228  1654 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:35.511593 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.516498 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:35.517927 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.519932 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.521221 28077 raft_consensus.cc:2410] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:13:35.518543 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:35.523233 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:35.508538  1607 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:35.525163 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.525851 28084 raft_consensus.cc:2376] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18. Re-sending same reply.
I20260709 13:13:35.527983 28084 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:35.529063 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.531128 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:35.531801 28227 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:35.533085  1667 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:13:35.534907  1684 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.535519  1684 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.538116  1684 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:35.538905 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:35.540223 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:35.540689  1670 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:35.541328  1670 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:35.542337  1670 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:35.548091 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 43882.9us
W20260709 13:13:35.549705  1690 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:13:35.552392 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:35.553797  1593 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:35.554200  1633 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:35.554545  1630 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:35.554421  1663 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:35.554848  1633 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:35.555164  1630 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:35.555732  1630 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:35.549784  1623 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:13:35.555905  1633 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.565188  1635 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:13:35.566920  1617 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:35.568321  1696 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:13:35.495002  1691 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:13:35.480023  1657 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.572515  1674 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:35.597705 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:35.598060 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 74897.9us
I20260709 13:13:35.504396  1688 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:13:35.609951  1697 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:35.610466  1697 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:35.613253 28084 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:35.599412 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:35.590867  1635 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:35.592473 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:35.595270 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.615353  1635 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:35.592753 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:35.592630 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:35.593809 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:35.617046  1593 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:35.594269  1649 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.591536  1657 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:35.593231 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:35.595291  1651 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.764s	user 0.014s	sys 0.000s
I20260709 13:13:35.594587 28235 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:35.583112  1694 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:13:35.621039  1651 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.596181 28079 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:35.525224 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.623389  1651 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:35.601471  1660 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.613837 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.626374  1660 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:35.626791 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.627426 28077 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:13:35.602837  1646 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:35.603827 28155 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:35.600021 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:35.564606  1618 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:13:35.607411  1684 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:35.611462  1697 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:35.615572 28154 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:35.632000 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:35.594112 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:35.620236 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 145435us
I20260709 13:13:35.621678  1694 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:13:35.595840  1693 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:13:35.623409  1649 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.601285 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:35.616487  1635 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.633998 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.619858 28227 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:13:35.634236 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:35.640638 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:35.636687 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:35.651224  1654 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:35.651851  1697 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:35.655601 28125 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:35.659384 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:35.671310 28231 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:35.671957 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:35.672552 28233 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:35.674995 28230 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:35.676127 28125 leader_election.cc:400] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18.
I20260709 13:13:35.676476 28125 leader_election.cc:403] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:35.678541 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.683948 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.686148 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.688491 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:35.678828 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44933.7us
I20260709 13:13:35.693049 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.695173 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.696058 28082 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:35.697284 28158 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:13:35.697340 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.697461 28157 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:35.700166 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.700624 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:35.705176 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:35.706017 28154 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:35.707082 28160 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:35.668960  1654 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.713398 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:35.667238  1645 raft_consensus.cc:2764] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:13:35.722277 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.638293  1694 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:35.674451  1651 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.667810  1660 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:35.719760 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.720126 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:35.720583 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:35.688038  1646 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:35.687808  1684 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:35.727450 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.731588 28760 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:13:35.730041 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.736383 28077 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:35.737183 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.732184 28755 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:35.735687 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.732932 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.744042 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.745350  1644 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:35.745532  1692 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.745750  1680 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.746327 28757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:35.746244  1692 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.747036 28757 raft_consensus.cc:606] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:35.715088   846 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:13:35.690105  1605 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:13:35.659356  1650 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.760807 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:35.751338 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.726876  1636 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18.
I20260709 13:13:35.770464 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.772248 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:35.712293  1667 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:35.721782  1654 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:35.776961 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:35.778700 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:35.779789 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.780341 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.789143 28154 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:35.726408  1577 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:13:35.745601  1624 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:35.826752 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 54554.6us
I20260709 13:13:35.734748  1651 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.830874 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:35.779292  1667 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.777456  1593 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:35.831985 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.796411  1605 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:35.795569  1670 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:35.835060 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.837005  1692 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:35.746361  1680 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.805039  1657 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:35.838680 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 162666us
I20260709 13:13:35.844529 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:35.843276 28756 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 17: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:35.832410 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.847847 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:35.849993 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.851804 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:35.852946 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:35.854282 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:35.834208  1667 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:35.837759  1663 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.860529 28082 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:35.859303 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:35.862349 28086 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:35.804036  1681 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.156s
I20260709 13:13:35.864987 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:35.840919  1670 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:35.851728  1701 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:13:35.856292  1703 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:13:35.847730  1660 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:35.828922  1624 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:35.798826  1694 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.917644  1630 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:35.958670  1660 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.856841  1593 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:35.857247  1644 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:35.949780  1712 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:35.869707  1521 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:35.859192  1674 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:35.870920  1605 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:36.048389 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:35.932813  1596 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.207s
I20260709 13:13:35.882391  1663 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:36.052925 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 182924us
I20260709 13:13:36.059512 28758 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
W20260709 13:13:36.052882 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 189526us
I20260709 13:13:36.062017 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:36.065979 28755 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:36.073042 28154 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:36.080788 28229 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:36.060343  1514 raft_consensus.cc:3055] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:36.061049  1660 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:36.084007 28077 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:36.085035 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:36.086069  1680 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.088820  1624 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.070603  1694 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:36.097360 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 247543us
I20260709 13:13:36.102310 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:36.066881  1674 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:36.105414  1705 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:13:36.105551  1674 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:36.078271  1593 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:36.109076  1593 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.109495  1593 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:36.084147  1649 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:36.112886  1702 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.216s	user 0.005s	sys 0.000s
I20260709 13:13:36.056201  1636 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:13:36.093302  1141 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
W20260709 13:13:36.104214  1700 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.222s	user 0.005s	sys 0.000s
I20260709 13:13:36.105083 28158 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:13:36.103230 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 157670us
I20260709 13:13:36.100028 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:36.093647  1514 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:36.113868  1630 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:36.083287   846 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:13:36.149230 28158 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:36.094867  1506 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:13:36.134229  1665 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:13:36.152872 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 58305.7us
W20260709 13:13:36.158290 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:36.158947 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.162357 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:36.142620  1650 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:36.168202  1650 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:36.171703  1663 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:36.177381  1714 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:13:36.094291  1670 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.156177  1651 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:36.156565  1665 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:36.218230 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 58109.5us
I20260709 13:13:36.221225  1663 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:13:36.219108 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:36.223387  1663 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:36.223522 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:36.223886  1663 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:36.225111  1691 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:36.225594  1691 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:36.226441 28758 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:36.226521  1691 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.228681 28758 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:36.228789  1663 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.229372 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:36.228694 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2385) took 1094 ms. Trace:
I20260709 13:13:36.229919 28228 rpcz_store.cc:276] 0709 13:13:35.134715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.144608 (+  9893us) service_pool.cc:224] Handling call
0709 13:13:36.228524 (+1083916us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b54003566d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:36.228639 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88450432}
I20260709 13:13:36.230183  1663 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:36.233367 28234 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:36.235035 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:36.235874 28755 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 17: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:13:36.238940 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.239500  1719 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:36.244458  1719 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:36.245508  1719 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.246089   792 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.247241   792 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:36.229017 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:36.248041 28757 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:36.251917  1326 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:13:36.260635 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 348693us
I20260709 13:13:36.261369 28226 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:36.262543 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:36.258792 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:36.267138 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:36.268457 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:36.269145 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 102397us
I20260709 13:13:36.271875 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:36.280148 28085 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:36.283541 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:36.284185 28233 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:36.286187 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:36.286339 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:36.286948 28228 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:36.182638  1681 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 13:13:36.208667  1670 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:36.293751 28226 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:36.285739  1649 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:36.301132 28234 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:13:36.276925  1651 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:36.308182 28234 pending_rounds.cc:85] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 9
I20260709 13:13:36.312480 28234 pending_rounds.cc:107] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:13:36.203954  1720 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:13:36.286238  1683 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:13:36.305363 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.310101 28226 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:36.312861 28234 raft_consensus.cc:2889] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:36.201515  1514 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.314203 28228 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:36.325356 28233 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:36.275760  1633 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:13:36.339736 28198 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:36.324111  1683 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.340726  1683 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:36.345757 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:36.304064  1649 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:36.294261  1644 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:13:36.351276 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:36.351552 28198 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:36.354308 28235 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
W20260709 13:13:36.367163 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 82307.6us
I20260709 13:13:36.308600  1667 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.356119  1649 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.333307   846 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:13:36.366513  1521 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:36.336902  1651 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.381906  1662 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:36.387612  1662 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.388077  1662 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:36.391094 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:36.392268 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:36.370699  1685 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:36.335629  1678 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:13:36.385865  1722 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:13:36.348016  1683 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.409260  1685 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.410758  1683 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.401506  1651 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.423952  1685 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:36.420154  1697 consensus_meta.cc:287] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.749s	user 0.006s	sys 0.007s
I20260709 13:13:36.424150  1660 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.427474 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:36.432749 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:36.437438 28233 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:36.443471 28231 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:36.413269  1655 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:36.420285  1680 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:36.420354  1645 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.445372   846 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:36.454692   846 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:36.455574   846 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.455579  1686 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:13:36.461941  1667 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:36.462249   846 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:36.445356  1697 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:13:36.448048  1655 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.464303  1655 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:36.465334  1449 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:36.472151 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2873) took 1339 ms. Trace:
I20260709 13:13:36.472685 28155 rpcz_store.cc:276] 0709 13:13:35.132807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.145999 (+ 13192us) service_pool.cc:224] Handling call
0709 13:13:35.146297 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.628057 (+481760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.471855 (+843798us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b5400238a54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:36.472074 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051327744}
I20260709 13:13:36.475169 28155 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:36.475862  1660 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.490630  1593 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.492089 28226 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:13:36.501375  1593 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.483139  1645 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.503427 28152 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:36.503892 28152 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:36.505788  1645 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:36.440284  1633 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:13:36.509409 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 111708us
I20260709 13:13:36.511898 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:36.496076  1674 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.512909 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 1228) took 1015 ms. Trace:
I20260709 13:13:36.525180  1605 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.525837  1605 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.527978  1605 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:36.514843  1654 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.716s	user 0.008s	sys 0.004s
I20260709 13:13:36.529106  1636 consensus_queue.cc:1243] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:13:36.528990  1654 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.530053 28159 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:13:36.530099 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 430124us
I20260709 13:13:36.531574 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:36.532400  1654 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:36.532943  1630 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:36.533216  1716 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:36.533931  1716 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:13:36.534251  1730 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.182s	user 0.001s	sys 0.002s
I20260709 13:13:36.535152  1646 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:36.535059  1716 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.535826  1646 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.537489  1596 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.538331  1646 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.539005  1646 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:36.516057 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2579) took 1288 ms. Trace:
I20260709 13:13:36.542176 28081 rpcz_store.cc:276] 0709 13:13:35.227892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.228046 (+   154us) service_pool.cc:224] Handling call
0709 13:13:36.515789 (+1287743us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54001aead4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:36.515946 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179760384}
I20260709 13:13:36.542688 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:36.521358 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:36.524984 28078 rpcz_store.cc:276] 0709 13:13:35.497719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.497944 (+   225us) service_pool.cc:224] Handling call
0709 13:13:36.511737 (+1013793us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5800002d98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:13:36.512833 (+  1096us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4456192}
I20260709 13:13:36.538055  1596 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.520367  1679 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:36.519625  1649 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.545066 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:36.567649 28082 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:36.545607 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:36.570142 28125 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:36.547725 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:36.571946 28125 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:36.563774  1679 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:36.566452  1712 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:36.574942  1636 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:36.575059  1679 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.575661  1712 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.576846  1630 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:36.519778  1662 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.582891 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:36.582940 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:36.584829 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:36.528663  1605 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:36.586423 28081 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:36.588403 28086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:36.589159 28086 raft_consensus.cc:606] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:36.559563 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:36.589745 28086 raft_consensus.cc:614] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:36.589953 28758 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:36.591216 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:36.591744  1662 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:36.560793 28083 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:36.562964  1645 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:36.586704  1605 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.576610  1679 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:36.587253 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:36.601254 28085 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:36.589270  1593 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:36.603135  1593 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.605806  1593 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.588436  1636 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:36.606920 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:36.607489  1636 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:36.607941  1635 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:36.610407  1635 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.564556  1654 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.612618  1354 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:36.613145  1354 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:36.621389  1354 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.622499 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2390) took 1150 ms. Trace:
I20260709 13:13:36.622642  1735 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:36.623128  1735 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:36.608469  1636 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.626492 28757 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:36.626919 28757 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:36.627693 28757 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.624120  1735 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.599936  1645 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:36.629454  1735 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:36.629870  1645 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:36.632128  1712 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:36.632691 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 1226) took 1178 ms. Trace:
I20260709 13:13:36.633337  1605 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.633258 28079 rpcz_store.cc:276] 0709 13:13:35.454557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.455286 (+   729us) service_pool.cc:224] Handling call
0709 13:13:36.632536 (+1177250us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54002ad854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:36.632637 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240344960}
I20260709 13:13:36.635708 28086 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:36.623025 28083 rpcz_store.cc:276] 0709 13:13:35.472424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.472620 (+   196us) service_pool.cc:224] Handling call
0709 13:13:36.622300 (+1149680us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400296dd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:36.622428 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2337385728}
I20260709 13:13:36.636502 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:36.638152 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:36.638547  1661 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:36.638986  1661 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:36.639376  1661 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:36.634584  1649 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.642419  1665 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.622903  1354 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:36.643458 28152 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.645668 28152 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:36.539559  1646 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.651369 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:13:36.662410 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.664047 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:36.530197  1674 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:36.597585 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.666139 27982 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 11 to 18, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 18 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:36.643699  1665 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:36.667845 27982 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:36.670204 27982 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 14 to 17, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 17 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:36.673430 28086 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:36.673936 28086 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:36.674778 28086 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.675518  1646 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.600764  1718 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:36.503895 28084 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:36.678206 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:36.655980 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 84150.9us
I20260709 13:13:36.675854 28086 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:36.671729 27982 catalog_manager.cc:5697] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 13, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 13 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:36.681464 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:36.684059 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:36.599165 28758 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:13:36.613646  1635 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:36.690829 28085 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:36.661674  1630 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:36.693868 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:36.698599  1718 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:36.699314  1718 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:36.699782 28125 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:36.701395  1605 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:13:36.680675  1661 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.713214 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:36.702931 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:36.700384  1718 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.687932  1646 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:13:36.613226  1654 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.709829 28152 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:36.702198  1746 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:36.722457  1746 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:36.722525 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2886) took 1041 ms. Trace:
I20260709 13:13:36.723372 28152 rpcz_store.cc:276] 0709 13:13:35.681422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.681928 (+   506us) service_pool.cc:224] Handling call
0709 13:13:35.698759 (+ 16831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:36.503332 (+804573us) spinlock_profiling.cc:243] Waited 80.2 ms on lock 0x7b5400203da0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5eedc0 00007f59ce6791c0 00007f59ce67a231 00007f59ce676d8f 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c
0709 13:13:36.643068 (+139736us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b5400203da0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5f506f 00007f59ce66df60 00007f59ce66f7de 00007f59ce679b49 00007f59ce67a416 00007f59ce676d8f 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62
0709 13:13:36.722163 (+ 79095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.722452 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245305344}
I20260709 13:13:36.717291 28763 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:36.717257 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:36.723668  1746 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:36.732508 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.743077 28082 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:13:36.744869 28078 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:13:36.711468  1629 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:36.746753 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:36.751088 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:36.759246 28152 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:36.747457  1684 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
W20260709 13:13:36.771763 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 376413us
I20260709 13:13:36.692015  1655 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.746569  1716 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:36.777971 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:36.760969  1692 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.778445 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:36.752271  1646 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:36.783589 28152 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:36.765422  1635 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:36.783286 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:36.788901  1687 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:13:36.790351  1635 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:36.791273  1635 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.793825  1692 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:36.794790 28086 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:13:36.795358 28086 pending_rounds.cc:85] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 8
I20260709 13:13:36.795851 28086 pending_rounds.cc:107] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:13:36.796516 28763 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
W20260709 13:13:36.796469 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.796586 28086 raft_consensus.cc:2889] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:36.809618 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:36.809859 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:36.811425 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 94282.7us
I20260709 13:13:36.812222 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:36.795729  1629 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:36.814705 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:36.800626  1656 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:36.780767  1665 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:13:36.816565 28125 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:36.817739 28127 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:36.819847 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:36.800714  1716 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:36.825402  1661 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:36.828837  1655 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:36.829272  1716 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:36.829367 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:36.814002  1674 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:13:36.831403 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:36.833231 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:36.834424 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:36.802640  1687 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:36.840641 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:36.844947 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:36.851316 28083 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:36.845892 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:36.844949 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:36.847098 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:36.852430 28081 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:36.848202 28085 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:36.848937 28233 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:36.849103  1716 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:36.813567  1646 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:36.850157 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:36.868176 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:36.818992  1607 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:36.869060 28161 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:36.869650 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:36.866948 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:36.868201 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:36.873060 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.874763 28081 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:36.980182 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:36.981545 28161 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:36.982931 28228 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:13:36.986114 28084 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2587) took 1299 ms (client timeout 1683 ms). Trace:
W20260709 13:13:36.986500 28084 rpcz_store.cc:269] 0709 13:13:35.686376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.694507 (+  8131us) service_pool.cc:224] Handling call
0709 13:13:36.985953 (+1291446us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b54003a93d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:36.986059 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665852160}
I20260709 13:13:36.987818 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:36.989241 28158 raft_consensus.cc:2410] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:13:36.988909 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:36.991957 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:36.979446 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:36.991634 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:36.995270 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:36.995504 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 145429us
W20260709 13:13:36.985720 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 114429us
W20260709 13:13:37.010867 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 142730us
I20260709 13:13:37.011865 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:37.012574 28159 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:37.012367 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:37.011883 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:37.015556 28230 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:13:37.018105 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:37.012667 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:36.860790  1629 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:37.028764 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:37.031287 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:37.025470 28234 raft_consensus.cc:2393] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 12: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:37.026336 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:37.027148 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:37.039592 28083 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:13:36.998135  1593 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:37.045765 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:37.134509 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:37.048050 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:37.137208 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:37.138309 28234 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:37.049789 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:37.140125 28233 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
W20260709 13:13:37.113603 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 115347us
I20260709 13:13:37.142508 28198 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:37.022902  1665 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:37.117519 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:37.030954  1644 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:37.027066  1654 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:37.042446  1722 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:37.127453 28153 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:13:37.126219 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 128841us
I20260709 13:13:37.108709  1718 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.160427 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:37.109323  1521 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:37.162993 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.114892  1692 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:37.132030 28159 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:13:37.145351 28155 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:37.104524  1352 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:37.144277 28158 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:37.129228  1674 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:37.148203  1754 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.129019  1685 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:13:37.148782  1655 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:37.149330  1747 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:37.150105  1721 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.289s
I20260709 13:13:37.151232  1593 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:37.153878  1678 raft_consensus.cc:993] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:13:37.155324   792 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:37.161767 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:37.014658  1726 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.112s
I20260709 13:13:37.130910 28154 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:37.229385 28158 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:37.129858 28160 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:13:37.203178  1636 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:37.121840  1646 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:13:37.164651  1679 consensus_meta.cc:287] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.534s	user 0.006s	sys 0.005s
I20260709 13:13:37.144048  1722 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:37.169899 28199 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.571s (SENT) [suppressed 27 similar messages]
W20260709 13:13:37.212198 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.113098  1727 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.095s
I20260709 13:13:37.166052 28153 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 17: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:37.202483 28232 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
W20260709 13:13:37.231355 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69656.3us
I20260709 13:13:37.238451 28159 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19. Ops: []
W20260709 13:13:37.242079 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 92030.6us
I20260709 13:13:37.254057 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:37.249851  1722 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:37.258152 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:37.167312  1718 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.207590  1650 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
W20260709 13:13:37.278306 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 286379us
I20260709 13:13:37.205166  1521 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.241366  1607 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:37.221753  1674 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.126641  1687 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.286697 28125 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:37.228823  1593 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.278105  1636 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:37.293035  1636 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:37.293563  1636 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.294477  1674 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:37.295641  1645 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.665s	user 0.010s	sys 0.002s
I20260709 13:13:37.286127  1736 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:37.296835  1736 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:37.275394  1678 raft_consensus.cc:1081] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:13:37.226812  1747 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.275579  1716 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:37.214696  1754 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.296063  1636 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:37.301015  1736 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.322391  1736 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:37.286022  1615 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:13:37.326015  1615 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:37.326814  1615 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:37.327989  1716 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.329057  1593 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:37.302697  1624 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:13:37.287928  1759 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:37.331738  1759 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:37.332386  1759 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:37.287070  1665 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:13:37.333591  1665 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:37.334044  1665 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:37.335028  1665 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.338291  1636 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:37.327834  1615 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.339991  1615 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:37.296134  1645 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.302210 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:37.352253 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 50176.5us
I20260709 13:13:37.359730  1636 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:37.360822  1521 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:37.361152  1636 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.361590  1636 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:37.360536  1656 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.532s	user 0.005s	sys 0.005s
I20260709 13:13:37.362898  1754 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:37.363214  1656 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:37.305054  1619 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:37.304865  1674 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:37.368345 28230 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:37.372200 28234 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 15. Ops: []
I20260709 13:13:37.374864 28229 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:37.386018 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 243550us
I20260709 13:13:37.387708 28198 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:37.389348 28198 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:37.366082  1593 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.351285  1758 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:37.369796  1665 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:37.364152  1619 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.418130 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:37.395931  1521 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:37.375221  1674 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.450816 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:37.400094  1758 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.400300  1593 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.487375 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:37.487936 28232 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:37.491376 28230 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:44070
I20260709 13:13:37.500864 28230 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:37.501366 28230 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:37.425053  1681 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:37.504436 28232 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:37.518440 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:37.520129 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:37.497895  1521 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:37.526206 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:37.503822  1768 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:37.461858  1722 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.507499  1718 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:37.513787  1661 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:37.541555 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:37.558115 28082 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:13:37.515331  1635 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:37.558715 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:37.559408 28758 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
W20260709 13:13:37.560160 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 390287us
I20260709 13:13:37.564123 28758 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:37.556015 28230 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.569234 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:37.571923 28081 raft_consensus.cc:2368] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18.
I20260709 13:13:37.549705  1685 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.515520  1681 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.528386  1667 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:13:37.566322  1649 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:37.573503  1619 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:37.575580  1656 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.578794  1661 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:37.607542  1719 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:37.604645 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:37.622726 28084 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 10.
W20260709 13:13:37.608911 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.574479  1646 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:37.604049  1765 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:13:37.579495  1645 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:37.641500 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.653736 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:37.656529 28153 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:37.580137  1760 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:13:37.669317  1747 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:37.636366  1759 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.626770  1719 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:37.664500 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:37.694162 28153 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 11.
W20260709 13:13:37.670722 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 120853us
I20260709 13:13:37.587926   792 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:13:37.661490  1605 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.722453  1605 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:37.725165 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:37.725669 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:37.695452  1767 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:13:37.686527  1596 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:37.621140  1685 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.628193  1774 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:13:37.690049  1722 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:37.690670  1674 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:37.589545  1727 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:13:37.626055  1646 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:37.691964 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:37.696780 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:37.633366  1739 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:37.699733 28229 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:37.703449  1719 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.704556  1760 consensus_queue.cc:1243] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:13:37.705585 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:37.626194  1656 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.713930 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:37.662113  1521 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.718436  1654 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:37.720697  1759 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:37.760752  1654 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.762481  1656 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:37.753836   846 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.763813   846 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.764447  1685 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:37.721647  1718 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.765324  1593 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:37.765532  1718 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.765969   846 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:37.752008  1722 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:37.766615  1722 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.767620  1514 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:37.768929  1521 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.769284  1758 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:37.770462  1681 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:37.770962  1521 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:37.691275  1661 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:37.777246  1732 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:37.777513  1629 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.734s	user 0.008s	sys 0.004s
I20260709 13:13:37.778025  1629 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:37.777846  1732 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.781153  1732 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:37.749464  1739 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:37.772199  1661 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.783540  1778 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:37.783027  1739 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.783944  1778 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:37.784873  1778 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.786345  1636 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.749881  1772 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:37.787717  1772 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.790594  1608 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:37.791656  1772 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:37.797163  1607 consensus_meta.cc:287] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
I20260709 13:13:37.738657 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:37.729592  1605 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:37.756486  1655 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:13:37.844230 28232 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:37.844769   792 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:37.639158  1577 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:37.845485  1577 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:37.845324   792 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.846448  1577 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.846903  1768 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:37.847522  1577 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:37.745530 28758 raft_consensus.cc:2368] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18.
I20260709 13:13:37.850243  1747 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:37.850628  1685 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:37.851246  1685 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.852245  1630 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:37.784494  1739 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:37.853401  1685 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:37.847492  1768 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.856035  1768 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:37.847144  1676 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:37.833850  1772 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:37.856709  1676 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:37.857064  1772 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.858001   792 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:37.857656  1676 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.858807  1676 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:37.859213  1772 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:37.834906 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 75292.4us
I20260709 13:13:37.860595 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:37.862002  1655 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:37.862509  1655 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:37.863369  1655 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:37.840008 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 126157us
I20260709 13:13:37.841756  1624 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:37.870389 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:37.843612  1784 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:37.860666  1636 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:37.863754  1654 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:37.835294  1722 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:37.871943 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:37.793210  1651 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:37.714356 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:37.852666  1630 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:37.876012 28755 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:13:37.875466 27982 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:37.890285 27982 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:37.877761 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:37.879144 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:37.893884 28077 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:37.880939 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:37.892340 27982 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 17. New cstate: current_term: 17 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:37.897444 27982 catalog_manager.cc:5697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:37.843645  1514 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:37.850690  1747 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:37.876488  1784 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:37.886945  1719 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:37.905858 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.907033  1514 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.960695  1514 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:13:37.910355 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.977860  1691 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:37.884999  1651 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:37.978499  1691 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.983099  1354 raft_consensus.cc:2955] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Committing config change with OpId 10.8: config changed from index 3 to 8, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:37.986161 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2398) took 1122 ms. Trace:
I20260709 13:13:37.986703 28234 rpcz_store.cc:276] 0709 13:13:36.863793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.139078 (+275285us) service_pool.cc:224] Handling call
0709 13:13:37.372023 (+232945us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:37.373195 (+  1172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.985900 (+612705us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54001f59d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:37.986079 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":534660352}
I20260709 13:13:37.991657  1772 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:37.991755  1646 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.992326  1772 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:37.994721  1772 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:37.995270  1772 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.997109 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:37.998688 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:37.878424 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:37.875759  1607 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:37.919979 28125 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:37.918287 28757 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:37.906814  1768 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.905042  1722 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.918278  1754 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:13:37.941303  1630 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.957247  1747 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:37.959379  1784 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:38.010597 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 132045us
I20260709 13:13:37.912147  1665 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:38.005874 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 86027.4us
I20260709 13:13:38.007148  1593 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:38.012135 28082 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:38.041175 28161 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:38.015713 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.042341 28077 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:38.042433 28086 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:38.029955 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.032294 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.039974 28125 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:37.998584  1691 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.046716 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:38.050414 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.045168 28082 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:13:38.045042 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.055975 28233 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:38.057476 28232 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:13:38.057862 28232 pending_rounds.cc:85] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 7
I20260709 13:13:38.058194 28232 pending_rounds.cc:107] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:13:38.006052  1644 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:38.075948 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.076783 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.076606 28154 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:38.058523 28232 raft_consensus.cc:2889] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:38.078071 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.078881 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:38.014780  1605 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:38.079429 28085 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:38.084066 28233 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:38.073748 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:38.028987  1320 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:38.083740 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:37.999805  1772 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.095144 28155 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:38.084337 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:37.999752  1685 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:13:38.088542 28198 leader_election.cc:400] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18.
I20260709 13:13:38.118335 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:38.089604 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.090271 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.079968 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.040794  1758 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:38.084146 28079 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:38.122366 28758 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:13:38.007968  1768 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:38.110874 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.014424  1722 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.101534 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:38.117723 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:38.118644 28198 leader_election.cc:403] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:13:38.122830 28758 pending_rounds.cc:85] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 14
I20260709 13:13:38.123206 28085 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:13:38.045259  1600 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.123723 28154 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:38.084695  1630 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:38.126355 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:38.127056 28764 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:13:38.116420 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:38.148857 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:38.136230 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.079698  1674 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:38.140830 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 52430.4us
I20260709 13:13:38.094583  1784 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:38.141142 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:38.124307 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:38.080166  1684 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:38.081557  1760 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:38.079843  1759 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:38.094388  1624 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.152397 28232 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:38.096112  1577 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.136221 28758 pending_rounds.cc:107] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:13:38.165799 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.166957 28154 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:38.167309 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:38.094782  1320 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:38.160956 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:38.074858   846 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:13:38.166198 28758 raft_consensus.cc:2889] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:38.118525  1141 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.167749 28234 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:38.123082  1521 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:38.057142  1723 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:13:38.171980 28228 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 13:13:38.175096 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.144876  1605 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:38.120630  1607 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:13:38.145071  1593 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18.
I20260709 13:13:38.156635  1644 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:38.173875 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.053014  1665 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.227460 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.152113  1600 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.261567 28232 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:38.158020  1629 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 13:13:38.227530 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3010) took 1381 ms. Trace:
I20260709 13:13:38.124847  1655 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:13:38.103453  1691 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:13:38.293200  1449 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:38.294042  1449 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.177325  1320 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:38.188068  1716 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.302371 28080 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
W20260709 13:13:38.302459 28229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3009) took 1457 ms (client timeout 1760 ms). Trace:
W20260709 13:13:38.303578 28229 rpcz_store.cc:269] 0709 13:13:36.845311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.845551 (+   240us) service_pool.cc:224] Handling call
0709 13:13:38.302223 (+1456672us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54003b3654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:38.302348 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877066112}
I20260709 13:13:38.145833  1758 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:38.172755  1759 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.310375  1665 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.179026  1645 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:38.162719  1732 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:13:38.263408  1674 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:38.272104  1600 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.267802 28226 rpcz_store.cc:276] 0709 13:13:36.846177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.846332 (+   155us) service_pool.cc:224] Handling call
0709 13:13:36.871965 (+ 25633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:36.872164 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:37.174591 (+302427us) log.cc:844] Serialized 14 byte log entry
0709 13:13:37.510832 (+336241us) log.cc:844] Serialized 14 byte log entry
0709 13:13:37.517397 (+  6565us) log.cc:844] Serialized 14 byte log entry
0709 13:13:37.901901 (+384504us) spinlock_profiling.cc:243] Waited 7.94 ms on lock 0x7b1800472fc0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:13:38.190736 (+288835us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:38.190791 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:38.190839 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.190932 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:38.227434 (+ 36502us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124832,"prepare.queue_time_us":431309,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":2042,"raft.queue_time_us":94039,"raft.run_cpu_time_us":448,"raft.run_wall_time_us":24656,"spinlock_wait_cycles":36925952,"thread_start_us":754425,"threads_started":3,"wal-append.queue_time_us":416127}
I20260709 13:13:38.074990  1651 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:38.281625 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.281932  1772 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:13:38.287143  1768 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:38.228631  1141 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.290060  1722 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:38.173045  1760 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.269305  1593 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
W20260709 13:13:38.232465 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 110830us
I20260709 13:13:38.314822  1759 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.315560 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.231416  1605 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:38.177417  1624 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:38.321512 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 157909us
I20260709 13:13:38.235584  1661 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:38.321956 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 129385us
I20260709 13:13:38.385002 28730 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:38.385071 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:38.386093 28758 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:38.386392 28758 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:38.366811  1600 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.321336  1514 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:38.318347  1608 consensus_meta.cc:287] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.003s
I20260709 13:13:38.178503  1577 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.394783 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:38.362171 28230 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.365362  1646 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.321751  1645 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:38.401158 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:38.392495 28758 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.402088 28758 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:38.332818 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 197048us
I20260709 13:13:38.356616  1449 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.378463  1605 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:38.380702 28757 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:13:38.410701 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1283) took 2142 ms. Trace:
I20260709 13:13:38.411002 28156 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:38.350419  1521 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
W20260709 13:13:38.394476 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 227234us
I20260709 13:13:38.318645  1716 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.407460 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.414059 28083 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:38.407426 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:38.411106 28757 rpcz_store.cc:276] 0709 13:13:36.268665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.268823 (+   158us) service_pool.cc:224] Handling call
0709 13:13:36.293385 (+ 24562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:36.626382 (+332997us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b540043a6a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5eedc0 00007f59ce6791c0 00007f59ce67a231 00007f59ce676d8f 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c
0709 13:13:37.917768 (+1291386us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b58003140d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5d30aa 00007f59ce655447 00007f59ce66dfe7 00007f59ce66f7de 00007f59ce679b49 00007f59ce67a416 00007f59ce676d8f 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4
0709 13:13:38.410427 (+492659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.410646 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1194129024}
I20260709 13:13:38.398216  1798 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:38.425158 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.425064 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.426012 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:38.395709  1726 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:38.427163 28760 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:38.438318 28758 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:38.440089 28198 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:38.406484  1645 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:38.413204  1691 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:38.411886  1724 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.411803  1712 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.443883 28154 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:38.426144  1732 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:38.427671  1665 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.394137  1661 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.471371 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.418857  1722 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:38.428337  1674 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:38.423261  1608 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:38.448149  1676 consensus_meta.cc:287] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.589s	user 0.015s	sys 0.000s
I20260709 13:13:38.401764  1600 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.422554  1768 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:38.477373 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.481590 28230 raft_consensus.cc:2955] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Committing config change with OpId 10.8: config changed from index 3 to 8, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:38.410578  1759 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:38.448062  1691 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.423491  1449 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:13:38.445066  1798 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.473588  1802 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.516973 28229 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:38.472110  1619 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:13:38.477823  1805 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:38.477926  1724 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.567214  1759 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.588744  1805 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.590165  1759 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.567749  1691 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:38.591958  1800 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:13:38.490401  1806 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:38.490453  1793 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.486606  1797 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.490437  1644 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:38.563442 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.491580  1732 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:38.568300 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 140315us
I20260709 13:13:38.584213  1320 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.528959  1758 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.528844  1665 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.473956  1723 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:38.516804  1819 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:38.471930  1712 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.582732  1802 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:38.584939  1141 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:38.587157  1716 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.530009  1722 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:38.609419 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:38.648224 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.634013 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:38.600024  1491 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:13:38.541867  1608 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.533273  1674 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.588605  1760 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.657023 28199 leader_election.cc:400] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18.
I20260709 13:13:38.596910  1759 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.564534  1768 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.670601  1674 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:38.644579  1719 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:13:38.671627  1768 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:38.630226  1793 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.672773  1320 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.674198  1793 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.644522 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.677759 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.678032  1722 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:38.678890 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.680029 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.633339  1732 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:38.680867  1805 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:38.649310 28232 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:38.685428 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.635641  1772 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.631212  1806 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:38.655275 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.637964  1723 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.638592  1685 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:38.659686 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:38.660120  1600 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.642918  1514 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:38.645959  1819 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:38.662276  1803 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:13:38.701499  1819 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:38.702502  1630 consensus_meta.cc:287] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.000s
I20260709 13:13:38.691872  1723 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:38.707429 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.712679 28157 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:38.706022  1605 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.714612 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.715776  1645 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.716374 28160 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:38.662178  1608 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.663700  1661 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:38.669503  1759 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.685707 28086 raft_consensus.cc:2393] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:38.669526 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:38.686379 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.687126 28078 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:38.642685  1624 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.688071 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.690497 28232 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:38.629981  1797 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:38.661641  1801 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:13:38.694454  1798 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.693158  1354 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:13:38.698712 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.699235  1685 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.708134 28153 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
W20260709 13:13:38.727582 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.783540 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.780759 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:38.780874 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 128582us
I20260709 13:13:38.786761 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:38.790273 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:38.793233 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.794042 28153 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:38.794754 28157 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:38.737458  1608 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:38.689579  1772 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.795547  1630 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:38.801878 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.802263 28228 raft_consensus.cc:2410] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:13:38.782204  1624 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.809448  1624 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.811645  1624 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.801234  1819 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:38.814289  1819 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:38.815184  1819 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.817296 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.818081 28152 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:38.818259  1797 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.818884 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.818998 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.819566 28154 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:38.819983 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.820359 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.821201 28154 raft_consensus.cc:2410] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:13:38.821444 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.821935 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:38.822590 28154 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:38.822724 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.823587 28159 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:38.795962  1760 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:38.825723  1645 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.822028 28152 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:38.833137 27982 catalog_manager.cc:5697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:38.835364 27982 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:38.837889 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:38.838804  1656 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:38.839201  1656 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:38.839576  1656 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:38.844133  1822 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:38.846769  1723 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.849764  1691 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:38.850929 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.851912  1691 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:38.853721  1724 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:38.856933 28760 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:38.860433 28124 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:38.860987 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:38.857184  1661 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.863016 28756 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:13:38.864440 28082 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:38.867779 28232 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 11. Current term is 12. Ops: []
I20260709 13:13:38.868476  1636 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:38.867969  1661 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.862133 28763 raft_consensus.cc:2955] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Committing config change with OpId 10.8: config changed from index 3 to 8, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:38.871037  1661 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:38.881917 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3022) took 1144 ms. Trace:
I20260709 13:13:38.904870 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.906068 28079 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:38.886139 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.907289 28082 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:38.709206  1806 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:38.890650 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.921905 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.928359 28230 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:38.926682 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:38.927680 28232 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:38.937668 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.940258 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.941123 28230 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:38.935281 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:38.944064 28233 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:38.892359 28227 rpcz_store.cc:276] 0709 13:13:37.737404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.739136 (+  1732us) service_pool.cc:224] Handling call
0709 13:13:37.739619 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:37.838147 (+ 98528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:37.838181 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:37.838668 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 13:13:38.079309 (+240641us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:38.385867 (+306558us) log.cc:844] Serialized 14 byte log entry
0709 13:13:38.423031 (+ 37164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.881340 (+458309us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b5400288054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:38.881421 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:38.881485 (+    64us) raft_consensus.cc:1685] finished
0709 13:13:38.881611 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:38.881836 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41389,"prepare.queue_time_us":229666,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":82110,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":15536,"spinlock_wait_cycles":46509952,"thread_start_us":492102,"threads_started":2,"wal-append.queue_time_us":439140}
I20260709 13:13:38.946246 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:38.947003 28082 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:38.891561 27982 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:38.835397 28154 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:38.952322 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:38.803128  1608 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:38.934544 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:38.958145 28759 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:38.956187 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:38.887344  1822 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:38.969843 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:38.970786 28159 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:38.977654 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.978686 28227 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:38.980079 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:38.980959 28230 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
W20260709 13:13:38.991523 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.880673  1665 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.001078 28155 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:38.999687 28052 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:39.005812 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:39.008222 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.009752 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.010567 28154 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:39.011631 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.012341 28154 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:39.013644 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:38.949760  1735 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:13:38.997574  1798 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.998109  1784 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:39.016711  1636 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:39.023044  1823 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.048436 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.077754 28154 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:39.078888 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.071120 28759 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:39.081257 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:39.084570 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:39.084956 28755 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.085597 28154 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.080204  1665 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.090593  1665 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.053670  1320 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:39.080025 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:39.076772  1624 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:39.074604  1784 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:39.117996 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:39.102427  1320 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.131112 28154 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:39.132251 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.133013 28154 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:39.104692  1685 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:39.079161  1798 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:39.114665  1624 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:39.139438 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 133754us
I20260709 13:13:39.141078 28755 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
W20260709 13:13:39.142540 28156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2603) took 3643 ms (client timeout 1571 ms). Trace:
I20260709 13:13:39.146538 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:39.146600 28156 rpcz_store.cc:269] 0709 13:13:35.481670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.482129 (+   459us) service_pool.cc:224] Handling call
0709 13:13:39.125066 (+3642937us) spinlock_profiling.cc:243] Waited 2.91 s on lock 0x7b54000807d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:39.125182 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6697354624}
I20260709 13:13:39.147284 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:39.118834  1784 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.080099  1823 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:39.080255  1636 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.148985 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.149601 28049 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.160679 28156 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:13:39.161316 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2919) took 1004 ms. Trace:
I20260709 13:13:39.161597 28156 rpcz_store.cc:276] 0709 13:13:38.157238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.160082 (+1002844us) service_pool.cc:224] Handling call
0709 13:13:39.160365 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.161099 (+   734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.161268 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:39.162873 28156 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:13:39.126008  1772 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.157976   846 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:39.170841  1819 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:39.171550  1819 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:39.172468  1819 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:39.177386  1825 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:13:39.185784  1823 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.146639  1624 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.199322 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2918) took 1053 ms. Trace:
I20260709 13:13:39.199585  1636 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.199790 28160 rpcz_store.cc:276] 0709 13:13:38.145735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.167129 (+ 21394us) service_pool.cc:224] Handling call
0709 13:13:39.199138 (+1032009us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b5400053554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:39.199240 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1039812992}
I20260709 13:13:39.200745  1491 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:39.202766 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.201261  1491 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.150126 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:39.147261 28762 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
W20260709 13:13:39.145172 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 171319us
I20260709 13:13:39.124478  1320 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:39.217753 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 67720.5us
I20260709 13:13:39.219424 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:39.216677 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.224438 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.225569 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.226434 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:39.222036  1808 log.cc:927] Time spent T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.566s	user 0.004s	sys 0.000s
W20260709 13:13:39.208442 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 208812us
W20260709 13:13:39.210304 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 178658us
I20260709 13:13:39.222895  1734 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:13:39.223263  1810 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:39.242012  1810 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:39.242470  1810 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:39.206707  1656 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:39.227487  1352 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:39.246572  1352 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.246727  1651 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:39.246982  1352 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:39.245128 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.239923  1691 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:39.248337  1691 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.240614  1734 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:13:39.248940 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.249137  1734 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:39.249625 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:39.250080  1734 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.250921 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.251327  1734 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:39.241441 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.244423  1735 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.228592  1818 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:39.253268  1735 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:39.261829  1735 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:39.262172  1732 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
I20260709 13:13:39.262564  1732 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:39.253329  1691 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:39.267976  1810 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:39.268676 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.268946  1656 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.269106 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.269826 28233 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:39.270533 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.270648 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:39.271134 28235 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:39.271364 28230 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:39.272176 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.273029 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.273216 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.273835 28226 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:39.274587 28234 raft_consensus.cc:2393] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:39.255990  1778 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:39.274776 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.278105 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:39.278662 28231 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:39.278903 28227 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:39.259136 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2916) took 1138 ms. Trace:
I20260709 13:13:39.261003 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.264536  1732 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.279076 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.286162 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:39.279803 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.286803 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.279529 28152 rpcz_store.cc:276] 0709 13:13:38.120769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.827412 (+706643us) service_pool.cc:224] Handling call
0709 13:13:39.258463 (+431051us) spinlock_profiling.cc:243] Waited 58.9 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:13:39.258667 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.258831 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:39.258854 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:39.258890 (+    36us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:39.258917 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.258990 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.259085 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135417728}
I20260709 13:13:39.252957 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.287781 28082 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:39.282403 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.283315  1712 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.289564 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.283404 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.256946  1665 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.290720 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.291944 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:39.257802  1491 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.292742  1712 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:39.292953  1141 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:39.293166 28077 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:39.289361  1797 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:13:39.295043  1784 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.295719  1786 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:13:39.296160  1786 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:39.297986 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.287638 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:39.288610 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.299963 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:39.256306  1818 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.290513 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.255880  1608 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.281591 28756 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:39.301805 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.303839 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.310343 28233 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:39.257651  1798 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.310770 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.311753 28083 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:39.312152 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:39.284132  1636 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:13:39.309432 28082 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:39.314554 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.315788 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.335558 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.336140 28083 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:39.317037 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:39.337666 28084 raft_consensus.cc:2376] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15. Re-sending same reply.
I20260709 13:13:39.277594  1630 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:13:39.291532  1837 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:39.291743  1759 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.323689 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.293177  1712 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.327667 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.313019  1665 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:39.314280  1608 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:39.344873 28082 raft_consensus.cc:2410] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:13:39.348551 28152 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:39.352636 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.353515 28152 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:39.358328 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.359236 28082 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:39.344236  1491 raft_consensus.cc:2764] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:13:39.357144 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 73814.5us
I20260709 13:13:39.365576 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:39.366802 28082 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:39.346926  1837 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.349323  1712 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.315987  1354 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:13:39.324498  1735 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.362715 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.359376 28160 raft_consensus.cc:2955] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Committing config change with OpId 10.8: config changed from index 3 to 8, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:39.374218 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.378995 28084 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:39.350436  1352 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.374176  1837 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:39.389000 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.409235 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:39.409523 28762 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:39.413132 28084 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:39.420306 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:39.334322  1786 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.430645 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.432449 28756 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:13:39.437654 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:39.410172  1326 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.438864 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:39.442726  1772 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:39.443341 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:39.444773 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:39.429983  1806 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:39.449931  1593 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:39.443367  1772 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.446841 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:39.452347  1821 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.669s	user 0.003s	sys 0.001s
I20260709 13:13:39.453248  1685 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:39.454069  1806 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:39.500953  1772 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:39.444077  1624 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.561461  1854 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.557528 28160 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:39.540863  1724 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.544369  1786 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:39.448549  1326 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.548341  1636 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:39.578594 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1110) took 1098 ms. Trace:
I20260709 13:13:39.548715  1685 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.490685  1320 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:39.573300  1844 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:13:39.579046 28758 rpcz_store.cc:276] 0709 13:13:38.479751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.480417 (+   666us) service_pool.cc:224] Handling call
0709 13:13:38.481281 (+   864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:38.481725 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.481752 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:38.482320 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:13:38.643639 (+161319us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:13:38.643675 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.643740 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:39.356130 (+712390us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:39.578320 (+222190us) raft_consensus.cc:1685] finished
0709 13:13:39.578339 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.578537 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":5027,"spinlock_wait_cycles":13372160,"thread_start_us":160982,"threads_started":1,"wal-append.queue_time_us":240379}
I20260709 13:13:39.581568  1352 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:39.583832  1691 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.585547 28085 raft_consensus.cc:2368] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15.
I20260709 13:13:39.562336  1854 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.554189  1806 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:39.555397  1818 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.574014  1593 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:39.600617  1593 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:39.609104 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1121) took 1599 ms. Trace:
I20260709 13:13:39.586393  1685 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.609654 28161 rpcz_store.cc:276] 0709 13:13:38.009670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.009799 (+   129us) service_pool.cc:224] Handling call
0709 13:13:38.010196 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.041633 (+ 31437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.608781 (+1567148us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b5400255654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:39.609029 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70843136}
I20260709 13:13:39.587878  1718 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.579685  1724 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.588002  1636 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:39.642328  1840 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.210s	user 0.001s	sys 0.001s
I20260709 13:13:39.643029  1326 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.627885  1735 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:39.645026  1854 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.628463  1651 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.645860  1651 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:39.646407  1651 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.650207  1724 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:39.650897  1847 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:13:39.652506  1636 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.633834  1834 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.653515  1834 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.656100  1834 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:39.641117  1842 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:13:39.627022  1853 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:39.667603 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.676962 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2634) took 1148 ms. Trace:
I20260709 13:13:39.687057  1605 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:39.690743 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.695930  1577 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:39.697767  1854 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.698364  1854 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.699970 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:39.700554 28153 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:39.700721  1593 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:39.701326  1593 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.700568  1854 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.709882 28153 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:39.710601 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2621) took 1296 ms. Trace:
I20260709 13:13:39.712984  1605 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:39.713580  1605 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.716861 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.717273  1605 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.717885 28762 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:39.719287 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:39.720360 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.718040  1593 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.721880 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.722748 28227 raft_consensus.cc:2410] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:13:39.710649 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:39.725077  1856 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:39.725709  1856 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.726224  1856 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.729324 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:39.729489  1654 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:13:39.730232 28086 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:39.730063 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 299496us
I20260709 13:13:39.730469  1654 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:39.730917  1654 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:39.731086  1354 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:39.731596  1354 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.731999  1354 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:39.731812  1654 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.733229  1608 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:13:39.733434  1654 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:39.733695  1608 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:39.734635  1608 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.726382 28159 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:39.737030 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 293739us
I20260709 13:13:39.738713 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:39.740424  1326 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:39.740854  1326 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.713012 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.741305  1326 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:39.741499 28761 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:39.742828 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.743752   792 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:39.744194 28759 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:39.744578   792 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.744922  1697 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:39.745046   792 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.745385  1697 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.745795  1697 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:39.747187 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.747802  1744 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:13:39.748179 28755 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:39.749536  1798 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:39.749675 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.750598 28762 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:39.752462 28758 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:39.754099 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.754841 28761 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:39.757555 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:39.759235  1636 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:39.759694  1636 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:39.760076  1636 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:39.760946 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.737098 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:39.762531 28082 raft_consensus.cc:2368] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15.
I20260709 13:13:39.757579 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.725735 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:39.769213 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 43532.5us
I20260709 13:13:39.769321  1600 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:39.772131  1600 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.772589  1600 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:39.773182 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:39.773816 28228 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:39.775998   792 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.774304 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:39.779407  1723 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.779407 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:39.780933 28230 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 13:13:39.857075 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:39.857249 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 83013.9us
I20260709 13:13:39.857717 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.771057 28159 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:13:39.858079 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:39.858521 28230 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:39.785230  1784 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:39.785504 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:39.867581 28155 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2630) took 1404 ms (client timeout 1715 ms). Trace:
I20260709 13:13:39.788038 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.869652 28234 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
W20260709 13:13:39.868242 28155 rpcz_store.cc:269] 0709 13:13:38.463743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.691276 (+227533us) service_pool.cc:224] Handling call
0709 13:13:39.867300 (+1176024us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54003773d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:39.867494 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502860672}
I20260709 13:13:39.793100 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:39.780185  1354 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.762538 28762 raft_consensus.cc:1217] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Deduplicated request from leader. Original: 12.10->[14.11-18.13]   Dedup: 18.13->[]
I20260709 13:13:39.797590  1608 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:39.787503  1636 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.798835 28226 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:39.791992  1856 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:39.810434  1810 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.760095  1747 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:39.817634 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.930729 28227 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:39.819218  1822 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:39.821235 28198 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.864818  1709 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:39.823326 27979 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 261) took 48173 ms (client timeout 15000 ms). Trace:
I20260709 13:13:39.823930 27982 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:39.943282 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 122115us
I20260709 13:13:39.945259 27982 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 15 to 16, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 16 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:39.824246 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 68831.3us
W20260709 13:13:39.828907 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 58305.7us
W20260709 13:13:39.830745 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:39.947376 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.831489  1676 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:13:39.834276  1697 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:39.828474 28228 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:39.948323 28198 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.842876 28728 leader_election.cc:400] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15.
I20260709 13:13:39.955101 27982 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:39.956424 28728 leader_election.cc:403] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:13:39.851238 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.677616 28154 rpcz_store.cc:276] 0709 13:13:38.528119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.166646 (+638527us) service_pool.cc:224] Handling call
0709 13:13:39.167046 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.167489 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:39.167530 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:39.203999 (+ 36469us) log.cc:844] Serialized 26 byte log entry
0709 13:13:39.378001 (+174002us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:39.378044 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.378115 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:39.676625 (+298510us) raft_consensus.cc:1685] finished
0709 13:13:39.676657 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.676873 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16483,"raft.queue_time_us":2103,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":16641,"thread_start_us":150639,"threads_started":1,"wal-append.queue_time_us":174631}
I20260709 13:13:39.957952 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.957367 27982 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 8 to 16, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 16 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:39.711095 28153 rpcz_store.cc:276] 0709 13:13:38.414591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.800894 (+386303us) service_pool.cc:224] Handling call
0709 13:13:39.699812 (+898918us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:13:39.710543 (+ 10731us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44491136}
I20260709 13:13:39.769215 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:39.859472 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.959656 28230 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:39.780997  1600 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:39.963848  1352 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:39.963997  1600 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.939654  1691 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.953933  1709 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:39.975684  1709 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.882169 28159 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:39.984925 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:39.959312 27982 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 11 to 17, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 17 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:39.895355 28762 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: [14.11-18.13]
I20260709 13:13:39.856540  1754 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:39.856818  1778 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.004s
I20260709 13:13:39.906399  1850 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.908999  1846 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:40.002559 28764 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:39.909658  1718 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.942286  1605 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:13:39.917027  1608 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:39.878485  1736 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.878638  1797 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:39.822449 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:39.888063  1320 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:39.888284  1759 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:39.948006 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.808120  1326 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:40.003722  1577 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:40.034586 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 86670.4us
I20260709 13:13:40.035758 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:39.958525 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:39.961238 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:40.040422 28153 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:40.035602 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:39.961273 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 118572us
I20260709 13:13:39.662982  1853 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.002612  1754 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:39.977807  1636 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:39.982156  1806 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:39.988181 27982 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:39.950043  1593 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:39.909871  1810 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:40.080366 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:40.006721 28199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.037s	sys 0.041s
I20260709 13:13:39.985217  1822 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:39.984472  1855 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:40.001094  1712 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.617s	user 0.006s	sys 0.006s
I20260709 13:13:40.021476 28229 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:39.961323  1723 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:13:40.025242 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:40.109761 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 62812us
I20260709 13:13:40.162200 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:40.083451  1630 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:40.077782 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:40.109972 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 74472.7us
I20260709 13:13:40.164559 28232 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:40.043370 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 84903us
I20260709 13:13:40.144454  1593 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:40.077406 28764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 1298) took 1925 ms (client timeout 1789 ms). Trace:
W20260709 13:13:40.157198  1651 consensus_meta.cc:287] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
I20260709 13:13:40.043869  1823 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:40.173306 28764 rpcz_store.cc:269] 0709 13:13:38.152106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.160653 (+  8547us) service_pool.cc:224] Handling call
0709 13:13:40.077179 (+1916526us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54005f0fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:40.077310 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4205969280}
I20260709 13:13:40.024776  1608 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.004544  1850 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:40.044174 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.143275 28153 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:40.143270 28199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.038s	sys 0.041s
I20260709 13:13:40.143942 27982 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:40.191864 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 745034us
I20260709 13:13:40.192833 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:40.193921 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:40.193303 27982 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 9 to 11, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 11 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:40.010257  1718 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.081974  1854 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:40.046195  1619 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:40.177605  1823 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.128623 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1300) took 1268 ms. Trace:
I20260709 13:13:40.204156 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:40.158530  1712 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.178081 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:40.147022  1855 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.147608  1691 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:40.146384  1856 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:40.179036 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:40.030598  1736 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.006371  1846 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.080865  1732 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 13:13:40.163229  1630 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:40.083547  1577 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.176453  1593 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:40.167241  1836 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:40.235834 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:40.173864  1644 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.196506 27982 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:40.233970  1772 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.002s
I20260709 13:13:40.138849 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2929) took 1169 ms. Trace:
I20260709 13:13:40.187469  1798 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:13:40.174319 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 263492us
I20260709 13:13:40.216225 28228 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:13:40.203634  1854 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.181264  1651 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:40.224627 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:40.227361 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:40.359234 28756 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:40.359562 28758 raft_consensus.cc:2368] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:13:40.229130 28157 raft_consensus.cc:2368] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15.
I20260709 13:13:40.229637 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:40.361232 28159 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:40.208379  1823 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:40.208700  1778 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.207067  1619 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:40.212845  1855 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:40.315754 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:40.225973  1864 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:13:40.319929 28154 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:40.212639  1691 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:40.322139 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 86437.9us
I20260709 13:13:40.205101 28763 rpcz_store.cc:276] 0709 13:13:38.860606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.861555 (+   949us) service_pool.cc:224] Handling call
0709 13:13:38.861809 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.862023 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:38.917804 (+ 55781us) log.cc:844] Serialized 14 byte log entry
0709 13:13:39.757407 (+839603us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b18004b2e00. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:13:40.063240 (+305833us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.128244 (+ 65004us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.128302 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:40.128347 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.128419 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.128558 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62696,"prepare.queue_time_us":101257,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":72264,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":21567,"spinlock_wait_cycles":7071104,"thread_start_us":487061,"threads_started":2,"wal-append.queue_time_us":272832}
I20260709 13:13:40.390264  1651 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:40.391063  1651 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:40.380700  1778 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.355305  1326 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:40.393278 28231 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:40.219893  1860 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:40.330513 28161 rpcz_store.cc:276] 0709 13:13:38.969342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.617818 (+648476us) service_pool.cc:224] Handling call
0709 13:13:39.618142 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.618516 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:39.675250 (+ 56734us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.138123 (+462873us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.138387 (+   264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.138456 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:40.138517 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.138610 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.138774 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":202726,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":39,"raft.queue_time_us":150,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":12989,"spinlock_wait_cycles":57307776,"thread_start_us":515004,"threads_started":2,"wal-append.queue_time_us":523222}
I20260709 13:13:40.333971 28125 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:40.322261  1821 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:13:40.318866  1863 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:13:40.366434 28228 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:40.316133  1449 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.370266  1856 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.322563  1845 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:40.391363  1837 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.964s	user 0.009s	sys 0.000s
I20260709 13:13:40.369345  1697 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:40.220085  1354 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.327394 27982 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 14 to 17, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 17 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:13:40.415115 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 245157us
I20260709 13:13:40.415740 27982 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 12, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 12 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:40.221800   792 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:40.377710  1685 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.001s
I20260709 13:13:40.360124  1854 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:40.357380  1772 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:40.368100  1726 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:40.371001 28153 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:40.424428 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:40.425485 28153 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 15: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:40.406392 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 72558.2us
I20260709 13:13:40.426828 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:40.428393 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:13:40.406939 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 197455us
I20260709 13:13:40.408867 28124 leader_election.cc:400] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:13:40.432613 28124 leader_election.cc:403] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:13:40.411070 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:40.434262 28161 raft_consensus.cc:2410] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:13:40.435400 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:40.380759  1619 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:40.413322 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 183679us
I20260709 13:13:40.328814  1810 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:13:40.393652  1691 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:40.419077 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:40.333642  1644 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:40.420202 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 226395us
I20260709 13:13:40.417342 27982 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:40.441072 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:40.441555 27982 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 9 to 17. New cstate: current_term: 17 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:40.443208 27982 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 12 to 16, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 16 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:13:40.334630  1802 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:40.330929  1836 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.402009  1654 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:40.445068  1636 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:40.418300  1449 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:40.448851  1861 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:13:40.402334  1860 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:40.449270  1644 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.450024 28077 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:40.452170  1836 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.452989  1718 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.499704 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:40.500790 28082 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:40.453197 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:40.452209  1691 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:40.460206  1870 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:13:40.461493  1712 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.445591  1853 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:39.950004 27979 rpcz_store.cc:269] 0709 13:12:51.649836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:51.650361 (+   525us) service_pool.cc:224] Handling call
0709 13:13:39.823107 (+48172746us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:30.988664 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:30.989577 (+   913us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:30.989658 (+    81us) write_op.cc:435] Acquired schema lock
0709 13:13:30.989694 (+    36us) tablet.cc:662] Decoding operations
0709 13:13:30.991317 (+  1623us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:30.991445 (+   128us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:30.991494 (+    49us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:13:30.998728 (+  7234us) tablet.cc:701] Row locks acquired
0709 13:13:30.998782 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:13:30.999029 (+   247us) write_op.cc:270] Start()
0709 13:13:30.999271 (+   242us) write_op.cc:276] Timestamp: P: 1783602810998933 usec, L: 0
0709 13:13:30.999329 (+    58us) op_driver.cc:348] REPLICATION: starting
0709 13:13:31.000787 (+  1458us) log.cc:844] Serialized 3627 byte log entry
0709 13:13:31.523072 (+522285us) op_driver.cc:464] REPLICATION: finished
0709 13:13:31.689211 (+166139us) write_op.cc:301] APPLY: starting
0709 13:13:31.689300 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:31.913287 (+223987us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:31.913498 (+   211us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:32.131770 (+218272us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:32.132273 (+   503us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 13:13:32.132345 (+    72us) write_op.cc:312] APPLY: finished
0709 13:13:32.155708 (+ 23363us) log.cc:844] Serialized 89 byte log entry
0709 13:13:32.167757 (+ 12049us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:32.167946 (+   189us) write_op.cc:454] Released schema lock
0709 13:13:32.168022 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:32.168444 (+   422us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:32.168472 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":923001,"reported_tablets":48,"spinlock_wait_cycles":6400,"child_traces":[["op",{"apply.queue_time_us":165794,"apply.run_cpu_time_us":19935,"apply.run_wall_time_us":479223,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41873,"num_ops":9,"prepare.queue_time_us":26334,"prepare.run_cpu_time_us":46757,"prepare.run_wall_time_us":165964,"raft.queue_time_us":353437,"raft.run_cpu_time_us":21766,"raft.run_wall_time_us":121423,"replication_time_us":523548,"spinlock_wait_cycles":100608,"thread_start_us":440337,"threads_started":4,"wal-append.queue_time_us":163838,"wal-append.run_cpu_time_us":19353,"wal-append.run_wall_time_us":2109098}]]}
I20260709 13:13:40.463217 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2922) took 1736 ms. Trace:
I20260709 13:13:40.430830  1726 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:40.424027  1834 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:40.424312  1845 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.464344  1869 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:40.413556  1600 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:40.419480  1856 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.465618  1850 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:40.562673  1636 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:40.467357  1772 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:13:40.419198  1806 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:40.468904 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:40.470157  1747 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 13:13:40.473969  1778 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:40.654121 28078 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:13:40.474838  1352 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
W20260709 13:13:40.475260  1786 consensus_meta.cc:287] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.897s	user 0.012s	sys 0.000s
I20260709 13:13:40.431416  1685 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:40.476156  1624 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.478492  1619 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.479242  1784 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:40.481678 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:40.658687  1861 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:13:40.484102 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:40.487620 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:40.489245 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:40.518930 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.560949 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:40.558409 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:40.563975 28080 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:13:40.462515  1736 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:40.566257 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:40.568364 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 115246us
I20260709 13:13:40.568281 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:40.576411 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 80795.1us
I20260709 13:13:40.648418  1869 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:40.484715   792 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:13:40.423514  1837 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:40.498854  1860 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.562397  1644 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:40.649209 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:40.572834  1802 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:40.657701 28158 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:13:40.698371 28084 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:40.496971  1654 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:40.700660  1654 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:40.700963 28157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1134) took 1830 ms (client timeout 1711 ms). Trace:
I20260709 13:13:40.701324  1760 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:40.701510 28157 rpcz_store.cc:269] 0709 13:13:38.870657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.874462 (+  3805us) service_pool.cc:224] Handling call
0709 13:13:39.769074 (+894612us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:13:40.700692 (+931618us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400343fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:40.700868 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647573632}
I20260709 13:13:40.701936  1760 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.703094 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:40.703931 28157 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 17: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:40.705200  1871 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:13:40.705302 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:40.706677 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:40.707348 28157 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:13:40.708809 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2925) took 1969 ms. Trace:
I20260709 13:13:40.710330  1837 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:40.709210 28152 rpcz_store.cc:276] 0709 13:13:38.739198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.354302 (+615104us) service_pool.cc:224] Handling call
0709 13:13:39.369571 (+ 15269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.431262 (+ 61691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:39.431310 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:39.436937 (+  5627us) log.cc:844] Serialized 26 byte log entry
0709 13:13:40.004557 (+567620us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:40.081203 (+ 76646us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.081451 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.081571 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:40.708561 (+626990us) raft_consensus.cc:1685] finished
0709 13:13:40.708586 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.708752 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68099,"prepare.queue_time_us":78199,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":3055,"raft.queue_time_us":65411,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":11981,"spinlock_wait_cycles":52841216,"thread_start_us":517627,"threads_started":2,"wal-append.queue_time_us":714693}
I20260709 13:13:40.711367 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:40.711591   792 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:40.712014   792 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:40.714757  1845 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.715452 28157 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:40.717988  1619 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.567088  1798 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:40.661532 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 233211us
I20260709 13:13:40.721040 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:40.578593  1726 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:40.671643 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:40.722132 28153 raft_consensus.cc:2410] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
W20260709 13:13:40.670621 28232 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.001s
I20260709 13:13:40.674769 28159 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:40.722688 28232 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:40.463435 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2927) took 1520 ms. Trace:
I20260709 13:13:40.723277 28160 rpcz_store.cc:276] 0709 13:13:38.943433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.578387 (+634954us) service_pool.cc:224] Handling call
0709 13:13:39.591444 (+ 13057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.639751 (+ 48307us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:13:39.855856 (+216105us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.205980 (+350124us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:40.206470 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:13:40.207768 (+  1298us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:13:40.207803 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.207871 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:40.463232 (+255361us) raft_consensus.cc:1685] finished
0709 13:13:40.463252 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.463381 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":176670,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":33094,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":9478,"thread_start_us":590890,"threads_started":3,"wal-append.queue_time_us":524843}
I20260709 13:13:40.676203  1784 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:13:40.678243 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.678282 28083 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:40.676014 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 257004us
I20260709 13:13:40.678442 28086 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:40.575325 28156 rpcz_store.cc:276] 0709 13:13:38.726611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.163533 (+436922us) service_pool.cc:224] Handling call
0709 13:13:39.163788 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.165823 (+  2035us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:39.165851 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:39.702901 (+537050us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b18004dd880. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a
0709 13:13:39.703320 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 13:13:39.927090 (+223770us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:39.927151 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.927263 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:40.462913 (+535650us) raft_consensus.cc:1685] finished
0709 13:13:40.462936 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.463156 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1734,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":34050,"spinlock_wait_cycles":4998272,"thread_start_us":223213,"threads_started":1,"wal-append.queue_time_us":340971}
I20260709 13:13:40.680472 28155 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:40.671255  1352 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15.
I20260709 13:13:40.683815 28052 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:40.680747  1786 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:40.697407  1449 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.697271 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:40.740579 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:40.677265 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 243773us
I20260709 13:13:40.741354 28160 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 11.
I20260709 13:13:40.745688  1848 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:40.746209  1848 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:40.746608  1848 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:40.749089 28755 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:40.732484 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:40.736059 28159 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
W20260709 13:13:40.757215  1879 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:13:40.756848  1869 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.765362 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:40.766381 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:40.737111 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53402us
I20260709 13:13:40.738664 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:40.729046  1654 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:40.613333  1834 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.749621  1691 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.750329  1798 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:40.752405  1822 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.596s	user 0.006s	sys 0.005s
I20260709 13:13:40.752991  1600 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:40.686858  1593 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.753927  1685 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:40.755492  1760 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.712863   792 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.744539 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:40.726373 28161 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:40.766286  1352 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:13:40.768675 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 71488.3us
W20260709 13:13:40.770155 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 361481us
I20260709 13:13:40.770206 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:40.804637 28125 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:40.775627 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:40.805657 28228 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:40.691102  1644 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:40.776928 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:40.778043 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:40.780236 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:40.780710  1834 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:40.768996 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:40.842185 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73263.3us
I20260709 13:13:40.792078 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:40.796545 28079 raft_consensus.cc:2410] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:13:40.776811  1873 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:40.780786  1449 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:40.808070 28755 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:40.768774  1818 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:40.859750  1818 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:40.861209  1818 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:40.863466  1860 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:40.808859 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:40.786703  1854 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.849622 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:40.797802  1856 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.790267  1654 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.807587  1352 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:40.865298 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 56580.5us
I20260709 13:13:40.816548 28125 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:40.781076  1855 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:40.867383 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 124916us
I20260709 13:13:40.841562  1760 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:40.797351  1600 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:40.843755  1605 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:40.885542 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 69101us
I20260709 13:13:40.857061  1449 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.882819 28198 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:40.852123  1873 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.866066  1822 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:40.894387  1760 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:40.895308  1874 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:40.895896  1449 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:40.893741  1856 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:40.889184 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:40.924153 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2939) took 1012 ms. Trace:
I20260709 13:13:40.924643 28157 rpcz_store.cc:276] 0709 13:13:39.911954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.719091 (+807137us) service_pool.cc:224] Handling call
0709 13:13:40.719369 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.719559 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:40.771794 (+ 52235us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.923793 (+151999us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.923867 (+    74us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:40.923920 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.923988 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.924097 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6791,"prepare.queue_time_us":49664,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":151602,"threads_started":1}
I20260709 13:13:40.898295 28125 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:40.903810  1855 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:40.904548 28235 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:40.905983  1754 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:40.953594 28199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.003s	sys 0.024s
W20260709 13:13:40.953999 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55855.9us
I20260709 13:13:40.906226 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:40.955847 28755 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:40.856207 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:40.958339 28232 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:40.958566 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:40.857313  1744 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.941300 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:40.887799  1816 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.002s	sys 0.024s
I20260709 13:13:40.959543 28230 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:40.877327  1736 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:40.884268  1352 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:40.954032 28199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.003s	sys 0.024s
I20260709 13:13:40.965807 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:40.911293  1873 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:40.984428 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:40.965615 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 525039us
I20260709 13:13:40.959172 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:40.924885  1874 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.997531 28228 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:40.997988 28199 leader_election.cc:400] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15.
I20260709 13:13:40.988250 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:40.989498 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:40.991443 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:40.978072 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:41.000223 28234 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:40.998353 28199 leader_election.cc:403] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:40.953796  1883 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:40.963039  1816 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.002s	sys 0.025s
I20260709 13:13:41.005215 28232 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:41.019622 28230 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:41.021286 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2941) took 1108 ms. Trace:
I20260709 13:13:41.016208 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:41.048041 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:41.051818 28080 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:41.044855 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:41.053901 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:41.051995 28155 rpcz_store.cc:276] 0709 13:13:39.912454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.732333 (+819879us) service_pool.cc:224] Handling call
0709 13:13:40.745202 (+ 12869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.745423 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:13:40.781718 (+ 36295us) log.cc:844] Serialized 14 byte log entry
0709 13:13:41.020827 (+239109us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:41.020903 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:41.020969 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.021059 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.021211 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23825,"prepare.queue_time_us":41581,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":258,"thread_start_us":238294,"threads_started":1}
I20260709 13:13:41.056779 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:40.963119  1845 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:40.991676  1605 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:41.065073 28077 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:13:41.067734 28084 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:40.974156  1822 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:40.997653  1884 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:40.973464  1736 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.017364  1874 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:41.006765 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:40.962745  1744 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.098306 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:41.103044 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 145449us
I20260709 13:13:41.049482  1855 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15.
I20260709 13:13:40.910741  1644 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:41.106133 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2084137980000378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
I20260709 13:13:41.107167 28157 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:41.068876  1882 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
W20260709 13:13:41.074204  1868 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:13:40.974392  1886 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:41.117000 28155 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:41.119182 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1904137980000087s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 13:13:41.126312 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0608137980000265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260709 13:13:41.116361 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:41.061609  1888 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.068634  1856 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:41.093984  1806 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.109591  1736 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:41.139618 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:41.141867 28755 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:41.142839 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.143683 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.144868 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.145053 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:41.145802 28228 raft_consensus.cc:2393] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:41.145921 28229 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:41.146849 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.148317 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.149785 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:41.142510 28230 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:13:41.151516 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:41.152402 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.153884 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.154781 28230 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:41.155635 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.142923 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.158282 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.167557 28228 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:41.121901  1754 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:41.186292 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:41.190124 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 132826us
I20260709 13:13:41.172989 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:41.175215 28755 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:41.191179 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:41.191943 28155 raft_consensus.cc:2368] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15.
I20260709 13:13:41.192420 28198 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.191671 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:41.142410  1823 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.003s
I20260709 13:13:41.097177  1845 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.195703 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:41.140348  1885 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:41.196434 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:41.193683 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:41.202322 28232 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:41.199054 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.138249  1593 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.215967 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.220247 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:41.221109 28755 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:41.223517 28762 raft_consensus.cc:2393] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:41.194782  1855 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:41.201220  1822 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.118698  1760 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.201169  1665 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:13:41.240509 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54056.1us
I20260709 13:13:41.194242  1860 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:41.228331 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2953) took 1308 ms. Trace:
I20260709 13:13:41.203290  1685 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:41.244872 28159 rpcz_store.cc:276] 0709 13:13:39.920012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.759237 (+839225us) service_pool.cc:224] Handling call
0709 13:13:40.759650 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.760157 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.760205 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:40.760849 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 13:13:41.097548 (+336699us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:41.097609 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.097714 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:41.228054 (+130340us) raft_consensus.cc:1685] finished
0709 13:13:41.228082 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.228254 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2627,"raft.queue_time_us":7435,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":19160,"thread_start_us":336165,"threads_started":1,"wal-append.queue_time_us":371542}
I20260709 13:13:41.250159 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.211862  1778 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.252641 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:41.168880  1806 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.221951  1772 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:41.220544  1890 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:41.140659  1888 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.229944  1854 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.209949  1823 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:41.263027 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 70682.3us
I20260709 13:13:41.230192  1593 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:13:41.262567  1726 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.000s
W20260709 13:13:41.277448  1896 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:13:41.269598  1772 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:41.276304 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.277390 28762 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:13:41.278034 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:41.254793  1885 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:41.287919 28756 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:41.259892  1665 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:41.272607 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.287043 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.303201 28758 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:41.302762  1665 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.267408  1786 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:41.289244  1636 consensus_meta.cc:287] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.003s
I20260709 13:13:41.312700  1636 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:13:41.312592  1786 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.315107  1786 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:41.315207 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:41.315932 28229 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:41.323879 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.325063  1654 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.274982  1593 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:41.326000  1593 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:41.325845  1772 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.327975 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.260496  1778 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.288412  1854 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:41.329312  1854 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.330442  1778 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:41.296108  1806 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.331000 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:41.331672 28157 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:41.332829  1577 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.297680  1845 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.333153 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.334012 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1127) took 3300 ms. Trace:
I20260709 13:13:41.334431 28158 rpcz_store.cc:276] 0709 13:13:38.033722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.035491 (+  1769us) service_pool.cc:224] Handling call
0709 13:13:38.076353 (+ 40862us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.698285 (+2621932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.333721 (+635436us) spinlock_profiling.cc:243] Waited 2.58 s on lock 0x7b54003430d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:41.333910 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5935655808}
I20260709 13:13:41.335749 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:41.336530 28158 raft_consensus.cc:2368] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17.
I20260709 13:13:41.337301 28762 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:13:41.337666 28762 pending_rounds.cc:85] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 13
I20260709 13:13:41.337859 28762 pending_rounds.cc:107] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:13:41.338078 28762 raft_consensus.cc:2889] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:41.341871 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.332320  1854 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.300132  1885 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.286412  1890 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:41.286728  1860 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:41.271772  1600 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.306888 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2958) took 1087 ms. Trace:
I20260709 13:13:41.308671 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.369925 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.371016 28757 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:13:41.309728  1848 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.000s
I20260709 13:13:41.311560   792 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:41.311952  1888 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.326404  1900 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.283782  1726 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.341979  1846 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.299431  1823 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.299311 28756 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
W20260709 13:13:41.299837 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 104201us
W20260709 13:13:41.291276  1898 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:13:41.347528 28156 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:41.350142 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:41.350658 28764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:41.350904 28728 leader_election.cc:400] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15.
I20260709 13:13:41.386456 28761 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:41.386802 28764 raft_consensus.cc:606] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:41.387377 28764 raft_consensus.cc:614] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:13:41.353600 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 67506.1us
I20260709 13:13:41.355650 28229 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:41.390782 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.391515 28233 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:41.392385 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.332150  1874 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.343250  1873 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.396195 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.373880 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.348651  1654 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:41.354267  1904 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.353796  1636 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:13:41.379995 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:41.381222 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:41.382768 28759 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:41.380380 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.488530 28756 raft_consensus.cc:2410] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:13:41.385134 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:41.393122 28764 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:41.369710 28154 rpcz_store.cc:276] 0709 13:13:40.219095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.908282 (+689187us) service_pool.cc:224] Handling call
0709 13:13:40.908642 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.912477 (+  3835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.912526 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:40.913167 (+   641us) log.cc:844] Serialized 26 byte log entry
0709 13:13:41.118354 (+205187us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:41.119434 (+  1080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.119564 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:41.306650 (+187086us) raft_consensus.cc:1685] finished
0709 13:13:41.306672 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.306825 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1994,"raft.queue_time_us":2991,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":30193,"thread_start_us":119668,"threads_started":1,"wal-append.queue_time_us":292110}
I20260709 13:13:41.349303  1449 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.378439 28157 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:41.490257 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:41.490648 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:41.490896 28154 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:41.462584 28760 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
W20260709 13:13:41.481628 28231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3060) took 1699 ms (client timeout 1815 ms). Trace:
I20260709 13:13:41.491788 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:41.492046 28231 rpcz_store.cc:269] 0709 13:13:39.782295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.785196 (+  2901us) service_pool.cc:224] Handling call
0709 13:13:41.481396 (+1696200us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54000d34d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:41.481557 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1212815744}
W20260709 13:13:41.493326 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 113400us
I20260709 13:13:41.493350 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:41.494712 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:41.483006 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 86903.4us
I20260709 13:13:41.498611 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.465620  1352 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.500396 28758 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:41.501806  1352 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.379257  1900 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.502959  1726 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:41.465975  1654 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.503926  1654 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:41.504204  1900 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.465909  1854 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 13:13:41.505563 28761 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:41.507069  1910 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:13:41.508138 28786 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
I20260709 13:13:41.510993 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.511907 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:41.502987 28763 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
W20260709 13:13:41.463727  1913 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:13:41.518388 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.499366 28764 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:13:41.513914 28158 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:41.519117 28232 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:41.519562 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:41.515759 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:41.519762 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.520313 28233 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:41.516649 28730 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.520489 28764 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:41.373426  1860 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.522158 28755 raft_consensus.cc:2368] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19.
I20260709 13:13:41.521960 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:41.523177 28228 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:13:41.526902 28757 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:41.529353  1726 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:41.529822  1726 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.530190  1726 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:41.532305  1744 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.533481 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.534426 28764 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
W20260709 13:13:41.534922  1894 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.325s	user 0.000s	sys 0.002s
I20260709 13:13:41.536227  1908 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:41.537037 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:41.537966 28757 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:41.538232  1685 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:41.538650  1685 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:41.539467  1685 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:41.540928  1854 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:41.540952  1910 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:41.526768 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.550786 28233 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:41.546149 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.552320 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:41.542215  1910 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.376058  1848 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.554603 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:41.555604  1891 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:41.556273  1891 raft_consensus.cc:2764] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:13:41.556483  1848 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.556816  1908 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:41.512665  1904 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.512246  1654 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.557312  1908 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:41.558099 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:41.570173 28760 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:41.395275  1577 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.156s
I20260709 13:13:41.542564  1644 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:41.573661  1908 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:41.578630  1608 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.465549  1736 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.553267  1884 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:41.551934  1900 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:41.564373  1449 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.562083  1885 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.559327  1904 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:41.577257  1577 consensus_queue.cc:1243] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:13:41.562867  1593 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.657166 28083 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:41.389650 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:41.736459  1778 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:41.737830  1778 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.739341  1878 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:41.740216  1878 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:41.669910 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.672658 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:41.673655 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:41.675158 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:41.689972 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 131744us
W20260709 13:13:41.691401 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 145198us
I20260709 13:13:41.721843 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.600595  1654 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.750442  1630 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.751289  1873 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:41.752110 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 82572.4us
I20260709 13:13:41.543227  1890 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.752918 28154 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:41.754920 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:41.588917  1900 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.730700   792 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:41.756953  1608 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:41.731442 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:41.757467  1736 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.758570 28762 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:41.738386 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 348673us
I20260709 13:13:41.676436 28159 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:41.724798  1605 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.745421  1874 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.607820  1860 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.766263  1605 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.767578 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:41.767921 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:41.757622  1608 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.759101 28157 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:41.744184  1806 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:41.736585  1772 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:41.770083  1910 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.783617  1772 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.784198  1772 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:41.774847 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.757308  1900 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:41.780445 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:41.757543   792 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.768937 28158 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:41.781343  1891 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:41.787103 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:41.791020 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:41.782691  1608 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:41.790807 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.797552 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:41.789031  1891 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:41.813109  1449 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.813342 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.818415  1449 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.819303 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:41.820084 28232 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:41.795001  1778 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.820106 28229 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:41.799546  1593 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:41.825003 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.825073 28762 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 19: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:41.825230  1854 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:41.827041 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:41.820883  1806 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:41.784255  1910 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.815914  1891 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:41.835983 28157 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:41.838676 28154 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 19
W20260709 13:13:41.841586 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 96085.9us
W20260709 13:13:41.844437 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45981.2us
I20260709 13:13:41.835711  1806 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:41.854691 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 74456.3us
I20260709 13:13:41.852226 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.857565  1786 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.850593 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:41.860093 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:41.861585 28231 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:13:41.814714  1878 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:41.841692  1887 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:41.869029  1887 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.872870 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1147) took 1146 ms. Trace:
I20260709 13:13:41.873621   792 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:41.878203 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:41.886081 28226 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 19: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:41.874521  1605 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.880090  1778 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:41.891446  1778 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:41.891222  1786 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.881927  1449 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.873456 28156 rpcz_store.cc:276] 0709 13:13:40.721542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.732220 (+ 10678us) service_pool.cc:224] Handling call
0709 13:13:41.858800 (+1126580us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b540011b054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:41.868306 (+  9506us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280513920}
I20260709 13:13:41.898495  1784 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.900362 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:41.896718  1786 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:41.901335 28157 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:41.907111  1887 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:41.901322   792 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:41.907930   792 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:41.908380   792 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:41.911859  1778 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.917515 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2437) took 1427 ms. Trace:
I20260709 13:13:41.917972 28078 rpcz_store.cc:276] 0709 13:13:40.490021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.490658 (+   637us) service_pool.cc:224] Handling call
0709 13:13:40.561240 (+ 70582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:40.654694 (+ 93454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.916982 (+1262288us) spinlock_profiling.cc:243] Waited 89.9 ms on lock 0x7b54000c80d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:41.917275 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206731008}
I20260709 13:13:41.924254  1910 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:41.901417 28232 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:41.925374  1900 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.931149 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2434) took 1443 ms. Trace:
I20260709 13:13:41.931591 28159 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:41.931897 28081 rpcz_store.cc:276] 0709 13:13:40.487164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.487292 (+   128us) service_pool.cc:224] Handling call
0709 13:13:41.930720 (+1443428us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b540017e154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:41.930968 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1055567232}
I20260709 13:13:41.932349 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2960) took 1210 ms. Trace:
W20260709 13:13:41.935945 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 1.13198e+06us
I20260709 13:13:41.941174 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:41.941740 28159 rpcz_store.cc:276] 0709 13:13:40.721812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.930779 (+1208967us) service_pool.cc:224] Handling call
0709 13:13:41.931228 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.932038 (+   810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.932239 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:41.942607 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:41.943559 28083 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:13:41.944024 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2961) took 1222 ms. Trace:
I20260709 13:13:41.944423 28159 rpcz_store.cc:276] 0709 13:13:40.722053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.942890 (+1220837us) service_pool.cc:224] Handling call
0709 13:13:41.943334 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.943682 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:41.943704 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:41.943758 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:41.943801 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.943855 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.943963 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:41.946789 28152 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:13:41.938506 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:41.938824 28079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2610) took 2719 ms (client timeout 2902 ms). Trace:
I20260709 13:13:41.949268  1772 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:41.949792 28079 rpcz_store.cc:269] 0709 13:13:39.218904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.219078 (+   174us) service_pool.cc:224] Handling call
0709 13:13:41.938364 (+2719286us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54002ace54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:41.938619 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3619344128}
I20260709 13:13:41.952160  1772 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:41.942126 28728 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.961402 28756 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:13:41.961926 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 1269) took 1188 ms. Trace:
I20260709 13:13:41.962427 28235 rpcz_store.cc:276] 0709 13:13:40.773464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.776599 (+  3135us) service_pool.cc:224] Handling call
0709 13:13:41.961296 (+1184697us) spinlock_profiling.cc:243] Waited 97 ms on lock 0x7b54003917d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:41.961717 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223076736}
I20260709 13:13:41.963891 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:41.964951  1883 raft_consensus.cc:2764] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:13:41.954272 28161 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1139) took 2047 ms (client timeout 2009 ms). Trace:
I20260709 13:13:41.959059 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:41.959766 28154 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:41.961865  1577 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:41.957134 28156 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:41.975384   792 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:41.976987 28229 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
W20260709 13:13:41.966748 28161 rpcz_store.cc:269] 0709 13:13:39.906893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.436544 (+529651us) service_pool.cc:224] Handling call
0709 13:13:41.953820 (+1517276us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54001978d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:41.954027 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523626880}
I20260709 13:13:41.900506  1784 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:41.982245 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2645) took 1010 ms. Trace:
I20260709 13:13:41.985172   792 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:41.985347 28156 rpcz_store.cc:276] 0709 13:13:40.972082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.899934 (+927852us) service_pool.cc:224] Handling call
0709 13:13:41.982180 (+ 82246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:41.990908  1655 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.994781 28049 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:41.974692  1577 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:42.003131 28157 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:42.003877 28158 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:13:41.991783 28079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:41.932613  1891 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.009584 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:42.011157 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 151302us
I20260709 13:13:42.017665 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2954) took 2097 ms. Trace:
I20260709 13:13:42.022729 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.026665 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.032014 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.014122  1449 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:42.020171 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.020629 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:42.033380 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:42.039636 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:42.042632 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:42.041545 28124 leader_election.cc:400] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19.
I20260709 13:13:42.044500 28124 leader_election.cc:403] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:42.049608  1577 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.050961 28756 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:42.052999 28154 raft_consensus.cc:2368] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16.
I20260709 13:13:42.015220  1655 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.058350 28079 raft_consensus.cc:606] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.058748 28079 raft_consensus.cc:614] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:42.061640 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.062157  1655 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.062638 28085 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
W20260709 13:13:42.062947 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 70559.3us
I20260709 13:13:42.046406  1449 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:42.065728 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.065848 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.069651  1835 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:42.018172 28160 rpcz_store.cc:276] 0709 13:13:39.920275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.845063 (+924788us) service_pool.cc:224] Handling call
0709 13:13:40.845462 (+   399us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:40.865438 (+ 19976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.865471 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:40.886984 (+ 21513us) log.cc:844] Serialized 52 byte log entry
0709 13:13:41.130694 (+243710us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:13:41.168578 (+ 37884us) log.cc:844] Serialized 14 byte log entry
0709 13:13:41.530317 (+361739us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b180044bc00. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a522 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:13:41.967375 (+437058us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b180044bc00. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:13:42.016516 (+ 49141us) log.cc:844] Serialized 14 byte log entry
0709 13:13:42.017043 (+   527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.017163 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:42.017232 (+    69us) raft_consensus.cc:1685] finished
0709 13:13:42.017247 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:42.017443 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":835772,"prepare.queue_time_us":902355,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":115,"raft.queue_time_us":22469,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":17027,"spinlock_wait_cycles":14919296,"thread_start_us":168863,"threads_started":1,"wal-append.queue_time_us":355404}
I20260709 13:13:42.073580 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:42.064940 28083 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:42.077874 28756 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:42.079766 28232 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:42.080912 28760 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:13:42.082442  1605 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19.
W20260709 13:13:42.083148  1908 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.010s
I20260709 13:13:42.083665  1908 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.077142  1655 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:13:42.069576  1922 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:13:42.105998 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:42.106758 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.107668 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:42.108008 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:42.142386 28160 raft_consensus.cc:2410] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:13:42.107259 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.109223 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:42.111162  1883 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.146183  1883 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.111469  1655 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:42.149268  1655 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:42.117156 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 122356us
I20260709 13:13:42.117944 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:42.122737 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:42.104633 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66351.5us
I20260709 13:13:42.129151 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:42.113847 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:42.112937 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:42.152633 28158 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:42.112396  1747 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.153942  1747 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.116168 28154 raft_consensus.cc:1217] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 12.10->[14.11-19.14]   Dedup: 16.12->[17.13-19.14]
I20260709 13:13:42.155092 28154 pending_rounds.cc:85] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 12
I20260709 13:13:42.155411 28154 pending_rounds.cc:107] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:13:42.113669  1806 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:13:42.156136 28154 raft_consensus.cc:2889] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:42.156275  1747 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:42.117352 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:42.091671  1449 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.157641 28155 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:42.104903  1900 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.148470  1883 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.124899 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:42.165242 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:42.123565 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:42.106967 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:42.167009 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53254.5us
I20260709 13:13:42.169301 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:42.170583 28233 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:42.095638  1726 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.565s	user 0.014s	sys 0.005s
I20260709 13:13:42.130523  1754 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.088686  1608 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.172569  1900 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:42.122714  1878 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:13:42.173375 28081 raft_consensus.cc:2410] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:13:42.173259 28764 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:42.123818  1845 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
W20260709 13:13:42.174331 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 50879.2us
W20260709 13:13:42.175359 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 65238.8us
I20260709 13:13:42.176210  1352 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:13:42.179632  1352 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:42.181978 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 69332.9us
I20260709 13:13:42.182119  1352 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:42.183743 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:42.185145  1837 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:42.181401  1754 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.187165  1772 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:42.187413  1644 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:42.187855  1772 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:42.188522  1747 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:42.188961  1772 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.181349  1806 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:42.189495  1747 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:42.191566  1900 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:42.192654  1900 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.195662  1845 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:42.196143  1845 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:42.197372 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 79556.8us
I20260709 13:13:42.197294  1845 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.198282 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:42.116351  1685 consensus_meta.cc:287] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.016s
I20260709 13:13:42.201150  1619 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:42.106135  1908 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:42.219545  1608 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:42.206286 28158 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:42.111999  1856 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:42.104089  1891 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:42.201807  1845 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:42.201567  1754 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.237694 28230 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:42.240674 28232 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:42.231316 28728 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.222185  1837 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:42.254340 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.248713  1878 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:42.202904  1726 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.230784  1449 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:42.246176  1619 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:42.264077 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 80467.7us
W20260709 13:13:42.264523 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 99444.8us
I20260709 13:13:42.274706  1726 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.276278 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:42.277083 28235 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:42.282299 28231 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:42.284883 28233 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
W20260709 13:13:42.285625 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 87486us
I20260709 13:13:42.286705 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:42.287364 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:42.288749 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65472.3us
I20260709 13:13:42.258070  1754 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:42.218293  1685 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.281344  1608 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:42.317118  1608 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.266479  1837 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:42.325428  1883 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.326596  1619 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.327927  1608 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.330981  1747 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:42.331920  1605 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:13:42.293248  1878 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:42.333712  1665 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:13:42.344903  1754 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.281453  1577 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:42.347378 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 54066.7us
I20260709 13:13:42.263677  1856 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.349414  1784 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.350471 28199 leader_election.cc:400] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17.
I20260709 13:13:42.420555 28199 leader_election.cc:403] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:13:42.351429 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 120217us
I20260709 13:13:42.352324 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.358484  1654 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.360011  1449 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.318888  1835 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:42.378588 28229 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:42.381089  1630 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:42.382341  1685 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:42.267084  1655 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.293680 28232 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:42.383473  1352 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.409783 28155 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:42.422940  1784 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.442188 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:42.442866  1784 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.443501  1784 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:42.443902  1784 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:13:42.445421 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 93305.5us
I20260709 13:13:42.445914  1449 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:42.447365  1630 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.419492  1878 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.439056  1654 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.424057  1754 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:42.426115 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 103218us
I20260709 13:13:42.451267  1654 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:42.451587 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:42.452099 28232 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:42.453274 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:42.455412  1593 raft_consensus.cc:2764] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:13:42.455528 28229 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:42.455886  1910 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:42.437192  1891 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17.
I20260709 13:13:42.456691 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.457281 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:42.457577 28230 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:42.458096  1891 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:42.458510  1856 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.421710  1778 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.459780  1665 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:42.459582  1778 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.460357  1665 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:42.462200  1778 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:42.421262  1883 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.458593  1845 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:42.459007 28233 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:13:42.461668  1665 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:42.462286 28226 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:13:42.464187 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:42.447554 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:42.471323 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.412954 28235 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:42.474686 28730 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:42.292266  1736 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:42.463941  1891 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:13:42.496282  1630 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:42.481848  1910 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:42.496980  1736 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:42.473551  1577 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:13:42.485337 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:42.500628 28154 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:42.485837 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:42.501711 28155 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:42.503021 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:42.485972  1778 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:42.486809  1810 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:42.487752  1449 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:42.487705 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:42.465906  1656 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.471231  1845 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:42.464928  1816 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:13:42.494091  1655 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:42.494216  1904 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:42.516599 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:42.464828  1835 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:13:42.497613  1910 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.481900 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:42.485513 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.538254 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:42.444289  1784 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:42.507522 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:42.511713 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:42.509832  1810 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:42.495182 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 50464.5us
I20260709 13:13:42.519834 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:42.535346 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:42.515064 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:42.541427 28158 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:42.541102 28157 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:42.515965  1778 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:42.552784 28155 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:42.539901 28160 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:42.554456 28159 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:42.550413  1656 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.556896 28156 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:42.557739  1656 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.570384 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.547025  1810 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:42.585001 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65325.2us
W20260709 13:13:42.585038 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 103383us
W20260709 13:13:42.575675 28199 connection.cc:256] RPC call timeout handler was delayed by -1.451417180000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2902s for a response
I20260709 13:13:42.587198 28158 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:13:42.597805  1835 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:42.598217  1835 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:42.599118  1835 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.600281  1835 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:42.602658 28159 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:42.604199 28155 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:42.514750  1837 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.609740 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:42.618386 28154 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:42.621435 28161 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:13:42.622133 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 62079.4us
I20260709 13:13:42.606083  1900 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:13:42.631253  1608 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:42.631839  1608 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:42.632074  1910 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:42.632949  1747 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:42.633594  1747 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.634351  1856 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:42.634511 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.636000 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.637599 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.595399  1845 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.655457 28156 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:42.668336 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.664037 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.677855 28233 raft_consensus.cc:2368] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13.
I20260709 13:13:42.623888  1449 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:42.637763 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.677191 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.693885 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:42.689554 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:42.729211 28234 raft_consensus.cc:2368] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18.
W20260709 13:13:42.731511 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 93803.4us
I20260709 13:13:42.624136  1938 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:42.669096  1910 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.670048  1900 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:42.737864 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:42.586258  1736 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.739343 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.740299 28230 raft_consensus.cc:2368] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:13:42.677237  1845 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:13:42.739333 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 153476us
W20260709 13:13:42.741833 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 104323us
I20260709 13:13:42.742029 28235 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:42.691293  1655 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:42.690832  1449 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:42.661265  1944 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:13:42.632829  1608 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.733453  1352 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:42.735163  1726 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:42.764604  1619 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:42.757252 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:42.759325 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:42.766085 28157 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:42.768975 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.769340 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:42.769764 28235 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:42.769933 28081 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:42.768147  1630 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:42.771474  1816 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:13:42.759882 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 259147us
I20260709 13:13:42.761060 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.761726 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.766335 28085 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:13:42.759532  1946 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:13:42.780251  1605 raft_consensus.cc:2764] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:13:42.678939  1778 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.750409  1655 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:42.758339  1449 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:42.765295  1883 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.743655  1910 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.791893  1619 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:42.759797  1939 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:42.795769 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.797022 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.799036 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.801163 28230 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:42.804445 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:42.805873 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48703.2us
W20260709 13:13:42.794955 28125 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.790s (SENT) [suppressed 12 similar messages]
I20260709 13:13:42.808024 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.809495 28231 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:42.783062  1856 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:13:42.811759 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:42.812608 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:42.831029 28235 raft_consensus.cc:2368] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
W20260709 13:13:42.817348  1945 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:13:42.818086  1910 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:42.837957 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.839717  1904 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.840346  1904 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:42.836045 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 63961.5us
I20260709 13:13:42.842578  1904 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.822566 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.823925 28082 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:13:42.823604  1810 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.825058 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.825320 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:42.809134 28080 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:42.833756 28085 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:13:42.835562 28081 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:42.797842  1619 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:42.838198 28235 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:13:42.821537 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:42.849413 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.853864 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.848035 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.861985 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.862598 28228 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: []
W20260709 13:13:42.862663 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 103240us
I20260709 13:13:42.865216 28083 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:13:42.874903 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.807960  1577 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:13:42.877928 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.878655 28232 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:42.878873 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:42.880014 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 85105.2us
I20260709 13:13:42.881181 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.881434 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.888922 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:42.889213 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.889573 28080 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:42.889755 28231 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:42.886379 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.891206 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:42.891853 28756 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:42.893554 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.894779 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:42.880913  1726 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:13:42.896587 28086 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:42.896885 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:42.897665 28758 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:42.897873 28086 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:42.898459 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:42.899094 28762 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:13:42.899212 28086 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [19.15-21.16]
I20260709 13:13:42.901018 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:42.881289  1655 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.902182  1655 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:42.904920 28761 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
W20260709 13:13:42.904804  1944 consensus_queue.cc:1175] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "611ec64ec58244fe82058d6130e4e6f3" responder_term: 15 status { last_received { term: 15 index: 8 } last_committed_idx: 7 last_received_current_leader { term: 15 index: 8 } }
I20260709 13:13:42.879906  1630 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:13:42.906406 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 76308.7us
I20260709 13:13:42.909472 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:42.911674 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:42.911906 28759 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:13:42.912032 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.912747 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.913887  1747 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:42.914829 28078 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:42.919178 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.917908 28755 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:42.925275 28077 raft_consensus.cc:2410] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 13:13:42.932272 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.933844 28761 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:42.932205 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:42.938709  1810 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:42.944866  1948 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:42.947808  1908 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:42.949256 28758 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:13:42.949551  1608 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:42.898037  1619 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.958973 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.950459  1956 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:13:42.955859 28226 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:42.958263 28080 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:13:42.948453  1908 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:42.970074 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:42.957737  1810 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
W20260709 13:13:42.982147 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48653.6us
I20260709 13:13:42.976573  1630 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:42.983819  1744 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:13:42.984030 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51292.5us
I20260709 13:13:42.984349  1744 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:42.985088 28127 leader_election.cc:400] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13.
I20260709 13:13:42.985391  1744 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:42.986105 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:42.986925  1744 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:42.987318  1630 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:42.987694 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:42.992954  1630 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:42.994801  1908 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:43.002104 28052 leader_election.cc:400] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18.
W20260709 13:13:43.002278 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 75465us
I20260709 13:13:43.002641 28052 leader_election.cc:403] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:13:42.996891  1619 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:43.029367 28127 leader_election.cc:400] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:13:42.962313  1948 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:13:42.980340  1655 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:13:43.032253  1784 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:43.046821 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:43.048338  1947 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:13:43.050606  1956 raft_consensus.cc:3055] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:43.051092  1956 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:43.051970  1956 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:43.033219 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 55717.1us
I20260709 13:13:43.034947 28127 leader_election.cc:403] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:13:43.033222  1810 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18.
I20260709 13:13:43.063606 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:43.061491 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:43.066471 28226 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
W20260709 13:13:43.066026 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 107616us
W20260709 13:13:43.067162 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 63644.9us
I20260709 13:13:43.067860 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:43.071348 28757 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:43.072615  1630 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:43.073068  1956 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:43.074911  1608 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:43.075554  1608 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:43.077864  1608 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:43.071159  1655 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:13:43.053490  1939 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:43.093470  1784 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:43.093703 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46959.7us
W20260709 13:13:43.094413 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46837.9us
I20260709 13:13:43.095062  1736 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:43.095208 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:43.153395 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 85599.9us
I20260709 13:13:43.082258  1953 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:43.070108  1654 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:43.083282 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:43.126657 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:43.113770  1962 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:43.158218 28085 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:43.159575 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:43.092374  1449 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:43.164181 28081 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:43.164811 28728 leader_election.cc:400] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16.
I20260709 13:13:43.165163 28728 leader_election.cc:403] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:13:43.168795 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:43.158638  1953 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:43.189239 28082 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 11.
I20260709 13:13:43.190492 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:43.190752 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:43.195103 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:43.171911  1736 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:43.188809  1953 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:43.199261  1784 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16.
W20260709 13:13:43.201789 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.187345  1654 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.162611  1938 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:13:43.209424 28081 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:43.192417  1939 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:43.221496 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 153641us
I20260709 13:13:43.211189 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:43.224215 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:43.224916  1593 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:13:43.225172  1964 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:13:43.206398  1962 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:43.238289 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:43.239197 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 51581.2us
W20260709 13:13:43.258535  1970 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:13:43.262343 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 111194us
I20260709 13:13:43.227489  1654 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:43.227361  1962 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:43.225338  1772 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:13:43.219821  1910 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:43.227942 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:43.310992 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53193.5us
I20260709 13:13:43.208858  1908 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:43.229045  1806 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:13:43.207295  1736 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:43.329604 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.329464  1593 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:43.355542 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:43.356122 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.328284 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:43.331313  1962 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:43.366920 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:43.334751 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 106930us
I20260709 13:13:43.337337 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.368080 28081 raft_consensus.cc:2410] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 13:13:43.332685 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 132156us
W20260709 13:13:43.352185  1965 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.320s	user 0.002s	sys 0.000s
I20260709 13:13:43.333467 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:43.379303 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.381672 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:43.387079 28077 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:43.388973 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:43.391714 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.335532  1908 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:43.392537 28085 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:43.394881 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.397992 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:43.396709 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.377962  1891 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:13:43.398872 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.399286 28084 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:43.347934  1939 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:43.400638 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:43.462918 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:43.463987 28757 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:43.400105  1726 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:43.368803  1449 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:43.469611  1968 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.308s	user 0.006s	sys 0.000s
W20260709 13:13:43.470614 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 137230us
I20260709 13:13:43.387044  1900 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:13:43.491015 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.483843 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:43.492106 28085 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:43.506701 28077 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 19: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:43.397370  1577 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:13:43.400467  1908 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:43.514366 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:43.506738 28756 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 14 LEARNER]: Advancing to term 16
I20260709 13:13:43.532338 28759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:43.533139 28759 raft_consensus.cc:606] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:43.533592  1778 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:43.534250  1778 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:43.550266 28084 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:43.493641  1938 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:43.558238 28764 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 11: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:43.491408  1891 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:43.564723 28229 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:43.538647  1856 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:43.540578  1778 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:43.649468 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 280381us
I20260709 13:13:43.651615 28127 leader_election.cc:400] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
W20260709 13:13:43.562161  1936 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:13:43.652788 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:43.650498 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 162073us
I20260709 13:13:43.661123 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.662101 28078 raft_consensus.cc:2410] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:13:43.507227  1608 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:13:43.666513  1726 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:43.678074 28756 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 16 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:13:43.556087  1939 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.108s
I20260709 13:13:43.682786 28757 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 16 LEARNER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:43.698609 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.701555  1726 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:43.710100 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:43.648566  1910 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:43.711400 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 221912us
I20260709 13:13:43.689250 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:43.683823  1938 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:43.717363  1744 consensus_meta.cc:287] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.730s	user 0.015s	sys 0.005s
W20260709 13:13:43.734814 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 242097us
W20260709 13:13:43.711921  1956 consensus_meta.cc:287] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.011s
I20260709 13:13:43.718281  1608 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:13:43.716790  1619 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.670s	user 0.009s	sys 0.006s
W20260709 13:13:43.742447 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 153846us
W20260709 13:13:43.737347 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48164us
W20260709 13:13:43.733213  1630 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.003s
I20260709 13:13:43.754941 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.755723 28198 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:43.759419 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:43.755152 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:43.655035  1593 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:43.745929  1654 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:43.774574  1654 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:13:43.775763  1654 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:43.776161  1654 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:43.779240  1962 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:43.761904  1891 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:43.780630 28233 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
W20260709 13:13:43.897805 28199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.010s	sys 0.015s
W20260709 13:13:43.900714 28199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.011s	sys 0.016s
W20260709 13:13:43.795603 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:43.901211 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 677008us
I20260709 13:13:43.800923 28760 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:43.817214 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.786103 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:43.903244 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 138277us
I20260709 13:13:43.858806 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.907102 28078 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
W20260709 13:13:43.859521  1979 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 13:13:43.865518  1993 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:13:43.920425 28078 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:13:43.921204 28082 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:13:43.815528  1971 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.307s	user 0.000s	sys 0.000s
I20260709 13:13:43.901947 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:43.770540  1630 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:13:43.772078  1744 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.847550  1577 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:13:43.917771  1953 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.691s	user 0.012s	sys 0.000s
I20260709 13:13:43.858218  1987 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:13:43.814771  1816 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.915624  1990 raft_consensus.cc:993] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:13:43.780874  1991 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:43.858819  1983 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:43.899608  1962 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:43.936466 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:43.925050  1939 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:13:43.958881 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 290748us
I20260709 13:13:43.959932 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:43.962396 28085 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:43.964037 28082 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:13:43.964476 28082 pending_rounds.cc:85] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 9
I20260709 13:13:43.964710 28082 pending_rounds.cc:107] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:13:43.966722 28082 raft_consensus.cc:2889] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:43.945381  1991 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:43.942363  1980 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:43.989624 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:43.902155  1910 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:43.984097 28084 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:43.898837  1654 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:43.945045  1983 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:43.948700  1987 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:43.937959  1744 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:43.951248  1939 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:43.938799  1953 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:43.956497  1630 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:13:43.942149  1816 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.001852  1900 raft_consensus.cc:1081] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:13:43.989377  1991 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:44.038331  1996 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.045989 28081 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:44.000373  1577 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:43.997779  1983 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:44.065099 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:43.997160  1980 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:44.069145 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 92729.2us
I20260709 13:13:44.070135 28082 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:44.054073  1630 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:44.123850 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 163988us
I20260709 13:13:44.128422 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1342) took 1056 ms. Trace:
I20260709 13:13:44.130188 28081 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:44.130605 28081 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:44.063202  1654 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:44.131177 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:44.150609 28760 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:44.131441 28081 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.676622 28760 raft_consensus.cc:606] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:44.677053 28760 raft_consensus.cc:614] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:44.458369 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:44.129169 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 313345us
I20260709 13:13:44.677102 28081 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:44.569185  1577 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.683076 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:44.402251 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 104820us
I20260709 13:13:44.299050  1988 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:44.515075  1962 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:44.691303  1962 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.693674  1744 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:44.129940 28758 rpcz_store.cc:276] 0709 13:13:43.068928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.069044 (+   116us) service_pool.cc:224] Handling call
0709 13:13:43.070838 (+  1794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.095633 (+ 24795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.095664 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:43.096187 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:13:43.194833 (+ 98646us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:43.194873 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.194941 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.124550 (+929609us) raft_consensus.cc:1685] finished
0709 13:13:44.124783 (+   233us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.124939 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24362,"raft.queue_time_us":25537,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":6524,"thread_start_us":98217,"threads_started":1,"wal-append.queue_time_us":241432}
I20260709 13:13:44.695526 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:44.699785 28082 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:44.712148 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 581193us
W20260709 13:13:44.673928  1977 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
W20260709 13:13:44.675331  2000 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.373s	user 0.001s	sys 0.001s
W20260709 13:13:44.674763  1997 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.612s	user 0.000s	sys 0.001s
W20260709 13:13:44.603991  1593 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.041s	sys 0.051s
I20260709 13:13:44.713254 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:44.130471  1630 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:44.624352  1944 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.285826  2004 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:44.728143  2011 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.508829  1987 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:44.733358  2002 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.596s	user 0.000s	sys 0.001s
W20260709 13:13:44.737393  2003 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.674s	user 0.000s	sys 0.001s
I20260709 13:13:44.738027  1593 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:44.738797  1995 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.677s	user 0.000s	sys 0.001s
I20260709 13:13:44.068974  1996 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.276310  1939 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:44.725891  1778 consensus_meta.cc:287] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 1.048s	user 0.012s	sys 0.000s
I20260709 13:13:44.751019  1944 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:44.756963 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66928.5us
I20260709 13:13:44.777132  2014 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:44.781108 28081 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:13:44.781709 28081 pending_rounds.cc:85] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 10
I20260709 13:13:44.783927  1957 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:44.787072  2004 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:44.678562  1980 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:44.793673 28081 pending_rounds.cc:107] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:13:44.811116 28081 raft_consensus.cc:2889] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:44.769716  1593 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.755306  2011 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:44.794106 28125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.006s	sys 0.005s
W20260709 13:13:44.820119 28125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.006s	sys 0.005s
W20260709 13:13:44.820412 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 1.03431e+06us
I20260709 13:13:44.812367 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1302) took 1151 ms. Trace:
I20260709 13:13:44.807602 28084 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:44.812285  1957 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.779304  2019 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:44.813862  1962 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:44.831895  1962 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:44.831754  2019 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.832484  1962 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.791393  1939 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:44.816793 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:44.818245 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1301) took 1162 ms. Trace:
I20260709 13:13:44.818842 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3142) took 1300 ms. Trace:
I20260709 13:13:44.770236  1778 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.688364  1988 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:44.825660 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.838486 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1149) took 1372 ms. Trace:
W20260709 13:13:44.839048 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:44.841158 28125 connection.cc:256] RPC call timeout handler was delayed by -1.7380896359998133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:13:44.820947 28226 rpcz_store.cc:276] 0709 13:13:43.656488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.659333 (+  2845us) service_pool.cc:224] Handling call
0709 13:13:43.683279 (+ 23946us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.713264 (+ 29985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.713320 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:43.744202 (+ 30882us) log.cc:844] Serialized 26 byte log entry
0709 13:13:43.925704 (+181502us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:43.925743 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.925810 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.797004 (+871194us) raft_consensus.cc:1685] finished
0709 13:13:44.797025 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.807660 (+ 10635us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15413,"raft.queue_time_us":21997,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":17180,"thread_start_us":134990,"threads_started":1,"wal-append.queue_time_us":232200}
I20260709 13:13:44.846968  1992 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:44.847833  1969 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 1.691s	user 0.010s	sys 0.017s
I20260709 13:13:44.847733  1992 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.848325  1962 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:44.849496 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 136368us
I20260709 13:13:44.838546 28232 rpcz_store.cc:276] 0709 13:13:43.518119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.532977 (+ 14858us) service_pool.cc:224] Handling call
0709 13:13:43.533321 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.533724 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.533760 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:43.534297 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:13:43.914167 (+379870us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:43.914225 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.914320 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.818619 (+904299us) raft_consensus.cc:1685] finished
0709 13:13:44.818641 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.818783 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7338,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":8122,"thread_start_us":379415,"threads_started":1,"wal-append.queue_time_us":414204}
I20260709 13:13:44.839068 28757 rpcz_store.cc:276] 0709 13:13:43.466181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.493142 (+ 26961us) service_pool.cc:224] Handling call
0709 13:13:43.681474 (+188332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.688218 (+  6744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.838220 (+1150002us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54006db850. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:44.838406 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":318701312}
I20260709 13:13:44.850808 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:44.837340 28234 rpcz_store.cc:276] 0709 13:13:43.656104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.656250 (+   146us) service_pool.cc:224] Handling call
0709 13:13:43.656680 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.657092 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.657133 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:43.666681 (+  9548us) log.cc:844] Serialized 26 byte log entry
0709 13:13:43.773043 (+106362us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:43.773093 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.773164 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.818017 (+1044853us) raft_consensus.cc:1685] finished
0709 13:13:44.818038 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.818173 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9876,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"thread_start_us":105935,"threads_started":1,"wal-append.queue_time_us":285044}
I20260709 13:13:44.853039 28758 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:44.857306  2004 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.857996  2004 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:44.819572  2018 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.861697 28759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fab37ead976b4378b1298941cec677a8"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:44.862602 28759 raft_consensus.cc:606] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:44.785312  2014 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:44.873884  2017 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:44.878093  2017 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.878562  2017 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:44.864861  1908 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.302s	user 0.012s	sys 0.002s
I20260709 13:13:44.859740  1900 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:13:44.884335 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.885408 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:44.888526 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:44.872821  1962 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:44.890421 28226 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
W20260709 13:13:44.908974 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 92320.8us
W20260709 13:13:44.915998 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:44.909713  1999 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.757s	user 0.000s	sys 0.012s
W20260709 13:13:44.918143 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.910071  1577 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:44.925843 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 75122.1us
I20260709 13:13:44.927317 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:44.886584  1987 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.933287 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:44.889359  1908 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.874019  2018 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.894889  1900 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.942052  1944 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:44.979138  1944 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:44.980798  2029 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.981504  2029 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.970510  1655 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.983311  1655 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.983788  2029 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:44.984380  2029 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.985219  2034 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:44.985752  2034 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:44.986138  2034 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:44.986378 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:44.986532  1655 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:44.987077 28226 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:44.969048 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:44.958616 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:44.975274  2009 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:44.928757  1577 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:44.997772 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:44.969798  2013 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:45.027016  2013 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.006331  2009 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.030562  2009 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:45.031418  1593 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.032042  1856 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:45.032531  2013 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:44.992801  2026 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:45.010167  1988 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:45.036178  2016 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:13:44.996874  2027 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:44.996140  2029 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:45.032544  1856 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:45.040915  1654 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:45.041853  1778 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:45.041908  1856 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:45.043217  1856 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:44.992295  2004 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:44.988713  2020 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:44.960155  1996 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:44.993927  1992 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:45.045589 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 76614.9us
I20260709 13:13:45.035156  1630 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.043238  2014 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:45.054157  1577 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:45.054710  1957 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.055899  2033 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:13:45.058947 28764 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 1345) took 1674 ms (client timeout 1809 ms). Trace:
I20260709 13:13:44.943385  1962 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:13:45.011540  2011 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:45.017441  1991 consensus_meta.cc:287] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.946s	user 0.008s	sys 0.002s
I20260709 13:13:45.047721 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:44.938140  1987 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:45.059427 28764 rpcz_store.cc:269] 0709 13:13:43.384824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.385109 (+   285us) service_pool.cc:224] Handling call
0709 13:13:45.058645 (+1673536us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b5400512cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:45.058777 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211982080}
I20260709 13:13:45.048384  1953 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:45.075048 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 106878us
W20260709 13:13:45.083122 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 197784us
I20260709 13:13:45.069022  1835 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:45.082444  2021 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:13:45.083281  1983 consensus_meta.cc:287] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.945s	user 0.010s	sys 0.001s
I20260709 13:13:45.083133  2027 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.052495  2017 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.095471 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.095738 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.096467 28084 raft_consensus.cc:2410] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:13:45.097646 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.085873  1962 consensus_queue.cc:1243] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:13:45.090603  2014 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:13:45.106772 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 59130.5us
I20260709 13:13:45.092294  2020 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.109546 28082 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 13:13:45.114153 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:45.114941 28156 raft_consensus.cc:2410] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:13:45.090711  2004 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.108912 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.094561  1608 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.096246  1996 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:45.111325 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.116269  2028 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:45.116865  1987 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:45.113643 28161 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:45.084025  1991 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.101109  1939 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.123116  1939 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.123796 28080 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:13:45.124400  1991 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:45.119175 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.125510 28160 raft_consensus.cc:2410] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:13:45.125679 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:45.125955 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.126807 28078 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:45.126572 28079 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:45.120082 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.131461 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.112847  2011 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.108945  1654 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:45.126991 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:45.081468  2026 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.133108 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:45.118918  2014 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:45.135687 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.139660 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:45.143179 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.097059  1835 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.137598 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:45.150241 28156 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:45.151248 28084 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
W20260709 13:13:45.141759 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55691.6us
I20260709 13:13:45.152680 28160 raft_consensus.cc:2410] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 18 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 13:13:45.153337 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:45.095567  1983 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.158166 28230 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:45.159015 28199 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:45.139701  1900 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:45.161084 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.133419  1608 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:45.161509 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:45.165354 28229 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:45.166880  1608 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:45.167723  1608 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:13:45.168964 28157 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:45.161966 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.129691  1996 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:45.152146  1953 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:45.128328  2004 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.175688  2035 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.138868  1987 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:13:45.138846  2034 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.120124  2020 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.194015 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:45.163687  1630 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.194794 28077 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:45.179067  2021 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:13:45.153916  1654 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.140260  2011 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:45.180681  1953 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.200922 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:45.208659 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.212319 28155 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:45.214299 28156 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:45.218824 28160 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:45.227334 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.228133 28084 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:45.229337 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.230291 28153 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:45.186075 28229 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:45.240065 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:45.218410  1953 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:45.169996  2028 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:45.197522  1608 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:13:45.205864  1449 consensus_queue.cc:1059] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:13:45.191721  2035 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.209517  1593 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:13:45.189711  2014 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:45.230746  1685 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 13:13:45.584388 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.304217 28077 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
W20260709 13:13:45.468888 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:45.364796 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 126883us
I20260709 13:13:45.226079  1962 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:45.595592 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.278798 28155 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:45.278153 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:45.456779 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 356995us
I20260709 13:13:45.301218 28156 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:45.599805 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.598348 28153 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:45.614976 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.615047 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.615756 28155 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:45.616065 28084 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:45.619788  1449 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:45.620294  1449 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:45.594998  1996 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:45.622315 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.301357  1950 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:45.250015  2036 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:45.342099  2017 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.390746  1835 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.365456  2020 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.634426 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.603320  2004 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.243736  2021 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.383s
I20260709 13:13:45.641000 28155 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:45.642122 28153 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:45.632668  1654 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.648854 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.621253  1449 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.642347  1950 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.657886 28153 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:45.637975  1983 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.654156 28156 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:45.636132  2011 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:45.674108 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:45.680004  1593 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:45.668810  2034 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.658769  1988 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:45.683151  1988 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:45.674476  2048 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:13:45.676656  2035 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.677428  1938 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:45.650019  2028 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:45.685619 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.694586 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.695283 28156 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:45.661477  1950 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:45.695120 28153 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
W20260709 13:13:45.693516 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 58253.5us
I20260709 13:13:45.706370 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:45.709291 28758 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:13:45.694167  1938 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:45.710855 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:45.711084 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:45.712020 28757 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:45.707300 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:45.748220 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3141) took 2403 ms. Trace:
I20260709 13:13:45.748859 28233 rpcz_store.cc:276] 0709 13:13:43.346107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.346422 (+   315us) service_pool.cc:224] Handling call
0709 13:13:43.386939 (+ 40517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.900640 (+513701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.747838 (+1847198us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b5400306e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:45.748089 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":904774912}
I20260709 13:13:45.749470 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:45.749629 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:45.750628 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:45.755457 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:45.758780 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:45.660476  2036 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.780021 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.783794 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:45.784114 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.784518 28156 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:45.785297 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:45.786372 28077 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:45.787844 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:45.784610 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.792621 28078 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:45.790570 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:45.791805 28758 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:45.795766 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:45.797688 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:45.796605 28230 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:45.801558 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:45.802191 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:45.802794 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:45.804067 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:45.811658 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:45.812580 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:45.835264 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:45.839582 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:45.804697 28077 raft_consensus.cc:2376] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15. Re-sending same reply.
I20260709 13:13:45.844887 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:45.785085  2021 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:45.840220 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:45.845732 28234 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:45.849354 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:45.855445 28156 raft_consensus.cc:2368] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:13:45.803809  1449 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:45.805311  1619 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.860499 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:46.095052 28756 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:45.804028  1962 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:45.813663  1685 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:13:46.052495 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.814132  1983 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:13:45.861821 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.116008 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.904373 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.119048 28084 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:13:46.120296 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2669) took 1011 ms. Trace:
I20260709 13:13:45.911475 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:45.869712 28232 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.011s	sys 0.006s
I20260709 13:13:46.126590 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2668) took 1017 ms. Trace:
I20260709 13:13:46.116637 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:13:46.133556 28232 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.014s	sys 0.044s
W20260709 13:13:46.085427  2027 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.976s	user 0.006s	sys 0.007s
I20260709 13:13:46.131862 28159 rpcz_store.cc:276] 0709 13:13:45.109464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.109727 (+   263us) service_pool.cc:224] Handling call
0709 13:13:45.115938 (+  6211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.395479 (+279541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.395533 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.458391 (+ 62858us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.874731 (+416340us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:45.912342 (+ 37611us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.912633 (+   291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.912769 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.119925 (+207156us) raft_consensus.cc:1685] finished
0709 13:13:46.119963 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.120177 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51254,"prepare.queue_time_us":45431,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":38320,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":5892,"thread_start_us":672587,"threads_started":2,"wal-append.queue_time_us":651200}
W20260709 13:13:45.907559  2058 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:13:45.910122  2045 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:13:46.133723 28158 rpcz_store.cc:276] 0709 13:13:45.099451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.099608 (+   157us) service_pool.cc:224] Handling call
0709 13:13:45.113208 (+ 13600us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.427608 (+314400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.431454 (+  3846us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.600710 (+169256us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.732018 (+131308us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:45.732085 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.732191 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.012338 (+280147us) raft_consensus.cc:1685] finished
0709 13:13:46.012361 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.117177 (+104816us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6716,"raft.queue_time_us":40425,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":353423,"threads_started":2,"wal-append.queue_time_us":324265}
I20260709 13:13:45.913399  1988 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.109427  1962 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:46.125665  1685 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:46.175134  1962 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:46.175289  1685 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:45.869406  2049 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.175134  1988 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:46.181161  2049 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.181499  1988 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.183632  2049 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.183857  1988 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:45.918017  2021 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:46.185427  1685 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.185506  2021 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.187287  1962 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.144107 28083 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:13:45.850792  1980 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.001s	sys 0.013s
I20260709 13:13:45.919466  2063 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:46.188902  2063 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:46.191180  2047 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.845s	user 0.000s	sys 0.008s
I20260709 13:13:46.191285  2063 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:45.917546  2060 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.141166  2027 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:45.919121  2036 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:45.893059  2028 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:45.917146  2059 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:46.192744  2060 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.156050 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2664) took 1060 ms. Trace:
I20260709 13:13:46.192764  2028 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.193826 28154 rpcz_store.cc:276] 0709 13:13:45.095335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.095457 (+   122us) service_pool.cc:224] Handling call
0709 13:13:45.095765 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.132346 (+ 36581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.132369 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.140562 (+  8193us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.573317 (+432755us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:13:45.777325 (+204008us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.777656 (+   331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.777767 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.155777 (+378010us) raft_consensus.cc:1685] finished
0709 13:13:46.155802 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.155977 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21852,"prepare.queue_time_us":338170,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":34297,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":22814,"thread_start_us":587013,"threads_started":2,"wal-append.queue_time_us":574038}
W20260709 13:13:46.155865  2001 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.977s	user 0.003s	sys 0.012s
I20260709 13:13:46.156555 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.196010 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.196149 28159 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:46.107139  1908 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.197114 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:46.128489  1619 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.197921 28159 raft_consensus.cc:2410] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:13:46.197693 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.198808 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.159039 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3159) took 1003 ms. Trace:
I20260709 13:13:46.199391 28159 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
W20260709 13:13:46.200619  1953 consensus_meta.cc:287] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.004s
I20260709 13:13:46.160185 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.202013 28158 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:46.158635  2005 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.163826  2055 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:46.164950  2061 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.227073  2061 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.228418 28084 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:46.229517  2061 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:46.230060  2055 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.165210  1950 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:45.856518  1938 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.130568  1983 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:46.232636  1983 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:46.233518  1950 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.235720  2028 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:45.908910  2035 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:46.188446  1980 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.198s	user 0.005s	sys 0.029s
W20260709 13:13:46.144665 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.193025  2059 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.200278 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:46.162688 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.199563 28226 rpcz_store.cc:276] 0709 13:13:45.155807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.156593 (+   786us) service_pool.cc:224] Handling call
0709 13:13:45.178600 (+ 22007us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:45.201982 (+ 23382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.202032 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.283665 (+ 81633us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.846851 (+563186us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:45.846897 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.846984 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:46.158766 (+311782us) raft_consensus.cc:1685] finished
0709 13:13:46.158789 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.158964 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11880,"raft.queue_time_us":17344,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":8532,"thread_start_us":442076,"threads_started":1,"wal-append.queue_time_us":761876}
I20260709 13:13:46.201076  1953 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.187819 28756 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
W20260709 13:13:46.216560 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 297847us
I20260709 13:13:46.224196 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.245383 28156 raft_consensus.cc:2410] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:13:46.225899  2005 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.199410  1619 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.226336 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:46.253371 28158 raft_consensus.cc:2376] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15. Re-sending same reply.
I20260709 13:13:46.189235  1685 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.236990 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.254160 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.237789 28152 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:46.172094  2012 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:46.248047 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:13:46.241123  2068 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.121s	user 0.000s	sys 0.005s
I20260709 13:13:46.255656 28159 raft_consensus.cc:2410] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 18 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 13:13:46.257893 28086 raft_consensus.cc:2368] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15.
I20260709 13:13:46.256969 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:46.256104 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:46.261165 28080 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:46.263065 28156 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:13:46.266769 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.276859 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.278244 28156 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:13:46.266013  1685 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:46.279906 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.276693  1656 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.283529  1953 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:46.285982  2055 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:46.254086  2035 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.308363 28156 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:46.309592  2035 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.281019 28124 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:46.270036  1962 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.269316  1744 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:46.294247 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:46.311926 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:46.303037  1980 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:46.303260  2063 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:46.303145  1908 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:46.329910  1908 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.316802  2028 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:13:46.332260  2028 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:46.333491  2028 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.260257  2012 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.337349  1744 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.341303  2005 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:46.343112  2029 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:46.346238 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2666) took 1247 ms. Trace:
W20260709 13:13:46.313773 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 65858.9us
I20260709 13:13:46.314532 28081 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:46.306734  1656 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.288471  2066 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:13:46.327603  2027 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:13:46.328142  2063 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.312857  2019 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.304919 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:46.346700 28157 rpcz_store.cc:276] 0709 13:13:45.098611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.098763 (+   152us) service_pool.cc:224] Handling call
0709 13:13:46.346043 (+1247280us) spinlock_profiling.cc:243] Waited 48.5 ms on lock 0x7b540014fd54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:46.346172 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111452032}
I20260709 13:13:46.360562 28079 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:46.329284 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.362218 28086 raft_consensus.cc:2410] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:13:46.353132 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3161) took 1196 ms. Trace:
I20260709 13:13:46.362689 28232 rpcz_store.cc:276] 0709 13:13:45.156382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.156670 (+   288us) service_pool.cc:224] Handling call
0709 13:13:45.177731 (+ 21061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:45.177997 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:46.134389 (+956392us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.351981 (+217592us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.352047 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:46.352099 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.352958 (+   859us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.353074 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12805,"prepare.queue_time_us":61485,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":425,"thread_start_us":1144816,"threads_started":2}
I20260709 13:13:46.336799  2012 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:46.373180 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:46.343132  2005 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.377604 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.378428 28158 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:46.379333 28157 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:46.379949 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.364933  2027 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:46.393867 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 2966) took 1282 ms. Trace:
I20260709 13:13:46.395478 28157 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:46.398274 28158 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:46.399503 28152 rpcz_store.cc:276] 0709 13:13:45.111649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.111806 (+   157us) service_pool.cc:224] Handling call
0709 13:13:46.393675 (+1281869us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540010ac54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:46.393795 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2423705856}
I20260709 13:13:46.401221 28152 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:13:46.402649 28152 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:13:46.402987 28152 pending_rounds.cc:85] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 8
I20260709 13:13:46.403203 28152 pending_rounds.cc:107] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:13:46.403438 28152 raft_consensus.cc:2889] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:46.375166  1908 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:46.346872  1980 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.388962  1962 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:46.415244 28152 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:46.414024 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:46.423640 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 50554.8us
I20260709 13:13:46.415056 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.413763 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.431061 28157 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:46.376902  2012 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:13:46.415642  1744 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.449664 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:46.390182  2005 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.413623  2070 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:46.452382 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:46.468575 28756 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:13:46.470588 28157 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:13:46.470889  1974 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:13:46.471513 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 180582us
I20260709 13:13:46.471699 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.472105 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.472702 28230 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:46.473002 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:46.473564 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:46.473892 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.474550 28235 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:46.475319 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.475920 28235 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:46.477008  1449 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:46.477427  1449 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:46.473598 28234 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:46.476456 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.483475 28199 leader_election.cc:400] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15.
I20260709 13:13:46.483872 28199 leader_election.cc:403] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:46.487540 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:46.506214 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 49980.3us
I20260709 13:13:46.507726 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.512552 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.514094 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:46.514772 28080 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 12.
I20260709 13:13:46.518970 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.521039 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.525334 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.525012 28233 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:46.526033 28756 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:46.526404 28233 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:46.521763 28086 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:13:46.534894 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:46.470014  1956 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:46.536592 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:46.545586 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 72895.6us
I20260709 13:13:46.546887  1449 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.548367  1449 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:46.554306  2081 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:46.512768  1685 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:13:46.556444 28152 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:13:46.557757 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.513051  1908 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:46.564205 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.565078 28758 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:13:46.520627  2033 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:46.536207  1988 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:13:46.573261 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.469759  1962 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:13:46.578219 28757 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:46.589156  1656 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:46.469393  2070 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.591703  1816 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.523674  2027 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.593475  2070 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:46.594149 28760 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:13:46.598917 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.599037 28079 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:46.562228  2021 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:13:46.602770 28086 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:13:46.603277 28760 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:46.586690 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.604086 28077 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:13:46.512527  2019 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:13:46.583127  2063 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:46.560051  2023 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:46.542608  2082 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:46.611164 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:46.568672  1956 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:46.613623 28230 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:13:46.615372 28086 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:13:46.620217 28758 raft_consensus.cc:2368] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:13:46.618135 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:46.611958  1908 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:46.629248 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.629451  1908 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:46.630079 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.573882  2033 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.632984  2033 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:46.609927  1983 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.633477  1956 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.635692  1983 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.618621 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:46.637113 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:46.637377  2024 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.646s	user 0.005s	sys 0.006s
I20260709 13:13:46.621881 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.624411  2005 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:46.625579  1980 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:46.601747  2035 raft_consensus.cc:2764] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:13:46.626551 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.654713 28152 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:46.654510 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:46.627108  2082 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.614702  2023 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:46.641628 28086 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:13:46.643020 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.647068 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.655283 28758 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:46.660204 28234 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:13:46.656759 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:46.653653  1983 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:46.666947  1983 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.667820 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:46.668689 28234 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:46.669682  2074 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:46.670414  2023 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:46.642985  1608 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:46.663978 28230 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:44070
I20260709 13:13:46.710111 28230 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:46.710573 28230 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:46.711462 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55040us
I20260709 13:13:46.709095 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:46.707445 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2484) took 1360 ms. Trace:
I20260709 13:13:46.715202 28081 rpcz_store.cc:276] 0709 13:13:45.346636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.596289 (+249653us) service_pool.cc:224] Handling call
0709 13:13:46.707288 (+1110999us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b540017c354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:46.707392 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590124288}
I20260709 13:13:46.720402 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:46.726333 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:46.726755 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:46.680261 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.741525  2096 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.742259  2096 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.748159 28758 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:13:46.754606 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.761421  1908 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:46.770900  2055 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:46.773365  1908 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:46.773877  1908 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:46.780383 28234 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:46.781515 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:46.783566 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 71445.9us
I20260709 13:13:46.676862  1983 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.786574 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:46.837157 28152 raft_consensus.cc:2376] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 22. Re-sending same reply.
I20260709 13:13:46.709971  1608 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:46.719619  2028 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.701557  2082 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:46.696841  2019 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:46.800032 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.695704  2060 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.695997  2018 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:46.800787 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.872712 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.874185 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.728163  1982 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:46.875286  1987 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:46.875905  1987 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.695134  2059 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.754945  2021 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:46.877172  2021 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.877832 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.878404  1987 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.878031  2019 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.880556  2019 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:46.706161  2084 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 13:13:46.882817  2084 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:13:46.877638  2021 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:46.883831 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.885921 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.889863 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.890581 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2466) took 1091 ms. Trace:
I20260709 13:13:46.890691 28759 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:46.891600  1988 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15.
I20260709 13:13:46.680507  2005 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.129s
W20260709 13:13:46.821820 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 160698us
I20260709 13:13:46.897104 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.898437 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.898962 28153 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:13:46.899619 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.900928 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.901007 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.899657 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:46.902340 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:46.902997 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:46.855094 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:46.904134 28227 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:13:46.907284 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.863865 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2970) took 1720 ms. Trace:
I20260709 13:13:46.854064  1608 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:46.856719  2028 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:46.868320  1939 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:46.870998 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.874934 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:46.875739  1982 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.891258 28231 rpcz_store.cc:276] 0709 13:13:45.799139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.818435 (+ 19296us) service_pool.cc:224] Handling call
0709 13:13:46.890204 (+1071769us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b5400306e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:46.890485 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1599796096}
I20260709 13:13:46.786180  1990 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:13:46.910395 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:46.901093 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:46.910154 28160 rpcz_store.cc:276] 0709 13:13:45.143427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.155087 (+ 11660us) service_pool.cc:224] Handling call
0709 13:13:45.214828 (+ 59741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:45.231066 (+ 16238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.863652 (+1632586us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b5400053550. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:46.863804 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101841152}
I20260709 13:13:46.810995 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.839167  2096 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:46.951561 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.914335 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:47.074043  2093 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:13:47.075973 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.078157 28730 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:47.079058 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 150124us
I20260709 13:13:47.082631 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.083431 28153 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:47.026115 28161 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:46.918154 28155 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
W20260709 13:13:46.985596 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.090080 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.078331 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:46.861085 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.086565 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:47.087267 28152 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:47.088074 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:47.021845 28159 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:13:47.034315 28158 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:13:47.039757 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:47.040236 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.093186 28232 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:46.932052 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3163) took 1687 ms. Trace:
I20260709 13:13:46.916421 28226 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
W20260709 13:13:47.113310 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.081954  1449 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.001s
I20260709 13:13:46.951887 28230 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.111224 28228 rpcz_store.cc:276] 0709 13:13:45.244375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.244630 (+   255us) service_pool.cc:224] Handling call
0709 13:13:45.298205 (+ 53575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:45.623216 (+325011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:45.623259 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:45.623302 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:45.623337 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.931687 (+1308350us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54002dafd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:46.931757 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.931947 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746915200}
I20260709 13:13:47.120189 28234 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:47.118706 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.115708 28155 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:13:47.116745 28227 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:47.122447 28153 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:13:47.119216 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:47.129976 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:47.127693 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.133653 28049 leader_election.cc:400] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:13:47.127199 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.138090 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.139022 28161 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:47.139397 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:47.140501 28049 leader_election.cc:403] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:13:47.135499 28234 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:47.091805  1990 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.144842 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:46.969806  2074 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:47.149669 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.150391 28160 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:47.154330  1908 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:47.126899  1974 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:13:47.149735 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:46.926443  2021 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:46.929994  2023 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.143247  1656 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:47.143523  2031 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:13:47.143389  2085 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:13:47.184441 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:47.186208 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.188964 28228 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:13:47.192435 28226 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:47.038362  2026 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.194837 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:47.197845 28230 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.200142 28227 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:47.219838 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:47.145391  2081 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:47.183595  1990 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:47.227280 28232 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:47.221863 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:47.225136 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 91707.1us
W20260709 13:13:47.197134  2033 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.005s
I20260709 13:13:47.235204 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:47.238590 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:47.231781 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:47.234061 28730 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.192026  2063 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:47.246384 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.197542  2092 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:47.242609 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:47.243695 28078 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:47.205652  2031 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:47.231935 28756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.007s	sys 0.014s
W20260709 13:13:47.256441 28756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.007s	sys 0.015s
I20260709 13:13:47.241886 28153 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:13:47.249576 28083 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:47.233387  2029 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:47.256510 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.259701 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:47.240455  1816 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:13:47.250721 28235 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:47.259755 28234 raft_consensus.cc:2410] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:13:47.251852 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.239755 28233 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:13:47.202627  2027 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.070s	sys 0.098s
I20260709 13:13:47.252799 28228 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:13:47.251891 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46238.7us
I20260709 13:13:47.205652  1656 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:47.226276  1630 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:47.254444 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:47.268601 28233 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:47.269572 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:47.247373 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 410247us
I20260709 13:13:47.243583  2026 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:47.196877  2005 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:13:47.264236 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:47.304121 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.304949 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.247645  2012 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:47.305270 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.242734  1837 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 11. Current term is 12.
I20260709 13:13:47.264384 28125 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:47.283294 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:47.306582  1837 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:47.308020 28083 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:13:47.258780  2031 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:13:47.267117  2027 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.071s	sys 0.099s
W20260709 13:13:47.301405 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.269951  1630 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.298476  2026 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.325227  1630 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:47.325620 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:47.270860  1656 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:47.264926  1816 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:47.307054  1837 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:47.265965  2074 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.312242  2023 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.312961 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:47.352978 28152 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.313622  2028 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:47.314199 28161 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:13:47.314788 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:47.355684 28158 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:47.318140 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.357216 28081 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:13:47.318483 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:47.319276 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.263913  1655 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:47.321102 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.321614 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:47.322402  2005 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:47.254017  2092 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:47.325292  2026 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:47.337244 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:47.261520  2063 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:47.338165 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.338124 28156 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:13:47.262075  2029 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.242251  2033 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.265543  2097 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:47.268935  2096 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:47.298707  2102 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.343483 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.308079 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.344647 28082 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:47.345918 28233 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:13:47.401329 28233 pending_rounds.cc:85] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 11
I20260709 13:13:47.401772 28233 pending_rounds.cc:107] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:13:47.402113 28233 raft_consensus.cc:2889] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:47.402156  1982 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.403950 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:47.346683 28050 leader_election.cc:336] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.544s (SENT)
I20260709 13:13:47.364939  2005 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:47.342882 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.338009  1856 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:47.409966 28086 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:13:47.361554 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:47.374372  2092 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:47.348014  2012 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:47.411453 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.388195  2029 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:47.375523 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.382191 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.354628  1837 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:47.349375  2101 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:47.394685 28227 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:47.395146 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.404686 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.405092 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:47.415091 28155 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:47.419651 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:47.405715 28152 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
W20260709 13:13:47.288553 28728 leader_election.cc:336] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.571s (SENT)
I20260709 13:13:47.407411 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.410519 28079 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:13:47.362972 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:47.380959  1655 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:13:47.421056 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 132565us
W20260709 13:13:47.409310  1957 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:13:47.432796 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.423884 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.362926  1608 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:47.433610 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 86963.1us
I20260709 13:13:47.422901 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:47.394568  2096 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:47.258687 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:47.439126 28155 raft_consensus.cc:2368] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14.
I20260709 13:13:47.438006 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:47.412416  1856 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:47.458079 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:47.461416 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.461665 28227 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:13:47.462069 28155 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:47.444774 28158 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:47.465656 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.466876 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.419070  2082 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:47.418002  2023 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:47.470155 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.476303 28157 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 17: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:47.472293 28160 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:47.438803  2096 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:13:47.465184  2070 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.030s	sys 0.017s
I20260709 13:13:47.461158  1987 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.393864  1816 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.457418  2027 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.452344  2028 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:47.469297  1856 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:47.484586  2018 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:47.502810 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 162579us
I20260709 13:13:47.460983  2036 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:13:47.494295  2070 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.030s	sys 0.018s
I20260709 13:13:47.510893 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:47.473251  1845 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:13:47.518796 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:47.521804  2094 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:13:47.449862  2101 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.530298 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:47.487696  2081 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:13:47.552920 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:47.553445  2033 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.555540  2101 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:47.509272  2028 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.565925  2028 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:47.561157 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:47.569227  1593 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:47.569499  1608 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.505072  1987 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.570890  2100 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:47.571964  1987 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.572288  2100 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:47.529600  1685 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:13:47.573433  2063 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:47.573336  2081 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:47.573625  2100 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:47.574149  2100 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:47.574445  2081 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:47.574862  2081 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:47.575753  2081 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.485484  2005 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.577530  2005 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:47.561131 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:47.580518  2103 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:13:47.561923  2106 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:13:47.539496 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1301) took 1328 ms. Trace:
I20260709 13:13:47.518451  2012 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.581944 28078 rpcz_store.cc:276] 0709 13:13:46.210426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.590817 (+380391us) service_pool.cc:224] Handling call
0709 13:13:46.591142 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.252501 (+661359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.538934 (+286433us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b54001af4d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:47.539208 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500234496}
W20260709 13:13:47.582535 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 311345us
I20260709 13:13:47.519383  1980 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:47.510278  2027 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:47.583956 28152 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:47.532924  2023 raft_consensus.cc:2764] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:13:47.585137  2027 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:47.564709  2104 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:13:47.581133 28153 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.525509  1816 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:47.587571 28761 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 16 LEARNER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:13:47.587502  1990 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.571066 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.514619  2036 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:47.588829 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:47.581094  2103 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:47.590801  2036 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:47.511488  1655 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.590329  2103 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:47.598882 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.613139 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:47.617027 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:47.613351 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 125242us
I20260709 13:13:47.619480 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:47.633168 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:47.636247 28153 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:47.637499 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:47.638090 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 107919us
I20260709 13:13:47.532685  2050 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:47.640920  2050 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:47.641420  2050 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.642818  1908 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:47.645145 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.645826 28159 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:47.648288  1845 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:47.648814  1845 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:47.649176  1845 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:47.649870  1630 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:47.650167  1656 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.652305  1630 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:47.583391  2026 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:47.652940 28160 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:47.653388 28158 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:13:47.655267  2026 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:47.569559  2109 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 13:13:47.636897  2070 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:47.659036  2070 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:47.659597  2070 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:47.665082  1950 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:47.666666  1950 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.671537  2031 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.672689 28083 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:13:47.673884 28764 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:47.674245 28764 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:47.673727 28152 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:13:47.676055  1990 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:47.675810 28764 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.695538 28764 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:47.676956 28152 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:13:47.658823 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:47.679059  1974 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:47.681396  1449 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:47.685457 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:47.686184 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:47.686864 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:47.688122  1950 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.688271 28226 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:13:47.690824 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:47.639034 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:47.692627 28160 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:13:47.693152 28159 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:47.666821  2021 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:47.696012 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.698751 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 2988) took 2017 ms. Trace:
I20260709 13:13:47.734650 28156 rpcz_store.cc:276] 0709 13:13:45.681680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.353607 (+671927us) service_pool.cc:224] Handling call
0709 13:13:46.372358 (+ 18751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.388477 (+1016119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.698526 (+310049us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54000a3cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:47.698696 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2220549376}
I20260709 13:13:47.652459  1656 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.701646 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:47.721067 28234 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:47.737423 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 279423us
I20260709 13:13:47.741374 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.742210  2114 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:47.742684 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.743537  2114 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:47.743961 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:47.744931  2114 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:47.639511  2081 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:47.701932 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:47.727443 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.729679 28229 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:47.625818  1938 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:47.753357 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:47.747879 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:47.730909 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 97809.8us
I20260709 13:13:47.731499 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:47.570609  2101 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.128s
W20260709 13:13:47.763768 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 124792us
I20260709 13:13:47.768663 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:47.699348  2031 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.764313 28234 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:47.765265 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.766069 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:47.745306  2114 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:47.780262 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:47.781025 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:47.733702  1980 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:47.782225 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:47.783535 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.753098  2102 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:47.746395  2103 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:47.781791 28161 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:47.745357 28127 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.728967  1990 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:13:47.810379 28760 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:13:47.767185  2021 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:47.802275 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:47.746224  2018 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.057s
I20260709 13:13:47.818383 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:47.785243  2059 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:13:47.807271 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61980us
I20260709 13:13:47.775789  1655 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:13:47.760607  2084 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:13:47.802197 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:47.820502 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:47.784716  2114 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:47.788720  2075 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:47.824466 28127 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.753839  2092 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.772650  1856 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:47.826924 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:47.801481  1449 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:13:47.753211  1837 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.831856 28728 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.801819  2115 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:47.833735 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:47.829932  2123 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:13:47.777099  2074 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:13:47.729887  2100 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.837837 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:47.855904 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:47.856921 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:47.814945  1957 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:47.823568  2026 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:47.841921  2092 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:47.846423 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:47.851612 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 83219.3us
I20260709 13:13:47.826222  1630 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:47.858492 28199 leader_election.cc:400] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14.
I20260709 13:13:47.868238 28199 leader_election.cc:403] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:13:47.868613 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.847792  2055 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:47.851915 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:47.869248 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:47.868345 28050 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:47.835804  2034 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:47.873107 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:47.821955  2021 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.882761 28756 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:47.873107 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:47.898691 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.900091  2027 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:47.900579 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.900763  2027 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:47.874982  2075 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14.
I20260709 13:13:47.901739  1608 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:47.902477  1856 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:47.871745  2055 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:47.903396  2055 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:47.878516  1950 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:47.904312  1950 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:47.879735  2070 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.853596  2102 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:47.854993  2036 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:47.912972  2102 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:47.915067  2102 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:47.848342  2115 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:47.915618  2115 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:47.889865  1837 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.890079 28228 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:13:47.919697  1938 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:47.920291  1938 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:47.890492 27982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 193) took 16089 ms (client timeout 15000 ms). Trace:
I20260709 13:13:47.886935 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:47.885591  1980 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:47.892567  2019 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:47.896706 28154 raft_consensus.cc:2410] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 13:13:47.876958  2118 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:13:47.905347 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:47.907667 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:47.867043  1982 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:47.867076  2059 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:47.913115  1656 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:47.911795  2028 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:47.914247  2096 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.881834  1630 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:13:47.925467 28755 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:47.929661 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:47.900426  1845 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:47.870297  1891 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:13:48.007838 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 120382us
I20260709 13:13:48.107362  2070 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:48.107599  1988 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:48.081276  2014 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:48.108122  1988 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:48.108291  2014 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.109112  1987 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.109258  1957 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:48.072613 28154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3005) took 1882 ms (client timeout 1832 ms). Trace:
W20260709 13:13:48.125752 28154 rpcz_store.cc:269] 0709 13:13:46.190575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.197462 (+  6887us) service_pool.cc:224] Handling call
0709 13:13:48.072413 (+1874951us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b540033e854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:48.072543 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3905020160}
I20260709 13:13:48.070006 28157 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
W20260709 13:13:48.069337 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.076912  2019 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:48.078073 28198 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:48.067706 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:47.933184  1938 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:48.080091  2034 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:13:47.928153  1980 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:48.011237 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 145414us
I20260709 13:13:47.934119  2075 raft_consensus.cc:2764] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 13:13:48.001793  2029 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.568s	user 0.010s	sys 0.000s
I20260709 13:13:48.016776  1950 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:47.880718  2050 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.076400 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35896 (request call id 1163) took 1177 ms. Trace:
W20260709 13:13:47.922217 27982 rpcz_store.cc:269] 0709 13:13:31.800766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.801078 (+   312us) service_pool.cc:224] Handling call
0709 13:13:47.890254 (+16089176us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:41.228918 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:41.300088 (+ 71170us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:41.300134 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:13:41.300153 (+    19us) tablet.cc:662] Decoding operations
0709 13:13:41.302494 (+  2341us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:41.302571 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:41.302607 (+    36us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:13:41.373359 (+ 70752us) tablet.cc:701] Row locks acquired
0709 13:13:41.375975 (+  2616us) write_op.cc:260] PREPARE: finished
0709 13:13:41.381839 (+  5864us) write_op.cc:270] Start()
0709 13:13:41.381968 (+   129us) write_op.cc:276] Timestamp: P: 1783602821381785 usec, L: 0
0709 13:13:41.381996 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:13:41.391963 (+  9967us) log.cc:844] Serialized 9729 byte log entry
0709 13:13:41.623629 (+231666us) op_driver.cc:464] REPLICATION: finished
0709 13:13:41.739735 (+116106us) write_op.cc:301] APPLY: starting
0709 13:13:41.739951 (+   216us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:42.367868 (+627917us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:42.368089 (+   221us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:42.904015 (+535926us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:42.904674 (+   659us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:13:42.904719 (+    45us) write_op.cc:312] APPLY: finished
0709 13:13:42.905396 (+   677us) log.cc:844] Serialized 220 byte log entry
0709 13:13:43.163378 (+257982us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:43.357563 (+194185us) write_op.cc:454] Released schema lock
0709 13:13:43.395794 (+ 38231us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:43.572680 (+176886us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:43.649329 (+ 76649us) write_op.cc:454] Released schema lock
0709 13:13:44.808203 (+1158874us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
Metrics: {"mutex_wait_us":3004796,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":115624,"apply.run_cpu_time_us":71476,"apply.run_wall_time_us":1812495,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108977,"num_ops":25,"prepare.queue_time_us":295184,"prepare.run_cpu_time_us":25115,"prepare.run_wall_time_us":290050,"raft.queue_time_us":983,"raft.run_cpu_time_us":1987,"raft.run_wall_time_us":120857,"replication_time_us":241474,"spinlock_wait_cycles":2896256,"thread_start_us":622232,"threads_started":5,"wal-append.queue_time_us":435959,"wal-append.run_cpu_time_us":33395,"wal-append.run_wall_time_us":3763015}]]}
I20260709 13:13:48.071219  2028 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:48.096040 28764 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:48.070272  2021 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:47.931403  1939 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:48.100343 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 170746us
I20260709 13:13:48.184787 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:48.106356  2027 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:48.074677  2026 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:48.141986 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 63991.6us
W20260709 13:13:48.216249 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.149618 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:48.151120 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:48.225121 28755 rpcz_store.cc:276] 0709 13:13:46.898919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.903228 (+  4309us) service_pool.cc:224] Handling call
0709 13:13:47.254208 (+350980us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:13:48.076224 (+822016us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54008da254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:48.076325 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1253779584}
I20260709 13:13:48.154505 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:48.002111  2036 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:48.155347 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:48.155014  1744 raft_consensus.cc:1064] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:13:48.156234 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:48.178043 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:48.127144  1856 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:48.179375 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:48.180282 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:48.153724  2131 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:13:48.125209  1987 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.073594  2055 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.186483 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:48.213563 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:48.144603  1608 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:48.142947 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 208232us
I20260709 13:13:48.268603 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:48.269126 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:48.248267 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.253566 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:48.270831 28077 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:48.262055  1837 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:48.273828  1837 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:48.274250  1987 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:48.253286 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 98001.8us
I20260709 13:13:48.274777  1837 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.144809  1891 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:48.276247  2050 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:48.277137  2019 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:48.259657  1744 raft_consensus.cc:1068] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:13:48.280077  1449 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.262259  2129 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:48.281216 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:48.282001 28758 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 22: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:48.145058  2102 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.282881  2055 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:48.283780  2102 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:48.255486 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.226305 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:48.285193 28759 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: []
W20260709 13:13:48.285537 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59325.6us
I20260709 13:13:48.269052 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:48.286494 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:48.125365  1957 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:48.294087 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:48.215791  2029 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.295157  1957 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:48.256661 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 78765us
I20260709 13:13:48.179242  1980 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:48.262640 28085 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:48.264839  1856 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.265738 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:48.251972  2127 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:13:48.280589  1449 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.287523 28154 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:48.257889 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:48.306775 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:13:48.307541 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49735.9us
I20260709 13:13:48.288797 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:48.288820 28077 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:48.284382  2014 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:48.293159 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:48.256490  2109 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:13:48.109083  1988 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.230437 28198 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:48.227900  1845 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:48.311664 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:48.330181 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:48.330824 28083 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:48.330880 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:48.312261 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:48.331825 28080 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 22: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:48.181945  1816 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 13:13:48.321463  1856 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:48.322109 28155 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 21: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:48.324769  1982 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:48.170888  2082 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.126s
I20260709 13:13:48.334728 28728 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:48.337248 28154 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:48.341836 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:48.347965 28159 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:48.344221 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:48.345047 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:48.348629 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:48.349481 28763 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:48.324455  1449 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:48.351935 28158 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:13:48.329540  2096 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:48.355778 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:48.356904  1845 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:48.357728 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:48.357879  1990 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:48.358376 28764 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:48.359076  1845 raft_consensus.cc:1064] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:13:48.359505  1845 raft_consensus.cc:1068] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:13:48.359575 28758 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:48.360023 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:48.361330 28761 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:13:48.364194 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:48.256784  1939 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.380914 28085 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:48.372124  2014 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:13:48.382030  2082 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:13:48.382540 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 152180us
I20260709 13:13:48.394299 28198 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:48.347880  2126 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:13:48.383881  2034 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:48.343200  1982 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:48.330516  1655 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:48.344565  1608 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:48.389213 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 80972.7us
I20260709 13:13:48.341594  1891 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:13:48.391146  2059 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:48.396700 28229 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:48.382421  2012 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:13:48.368301  2028 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:13:48.409276  1982 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:48.385046 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50403.3us
I20260709 13:13:48.408049  1988 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.406930 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:48.412810  2034 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:48.411007 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:48.391208  2096 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:48.433473  2034 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.433990  2096 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.441602  1988 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.444294  1988 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:48.413208 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 134786us
I20260709 13:13:48.413563 28228 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:48.391289  2005 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:48.341248  2075 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:48.391289  2082 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:48.264678  2027 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.407843  1655 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.424641 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:48.415074  2029 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:48.401557 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:48.408773  2055 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:13:48.408461 28125 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:48.406147  1744 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:13:48.417084  2050 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:48.400658  2063 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 13:13:48.432179  2081 consensus_meta.cc:287] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.000s
I20260709 13:13:48.455518 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:48.424978 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:48.436443  2096 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:48.426903 28158 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:13:48.430508  2021 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:48.458485 28155 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:48.461526 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 53156.7us
W20260709 13:13:48.469483 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 80770.5us
I20260709 13:13:48.472985 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:48.420996  1845 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:13:48.457182  1655 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:48.421181  2035 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:13:48.428541  1449 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:13:48.422909  2127 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:13:48.487634 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:48.456619  2075 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:48.463841  2084 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:48.457636  2059 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:48.428391  1947 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:13:48.471674  2028 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:13:48.497704 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 149127us
I20260709 13:13:48.509522 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:48.497479  2081 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:48.513379 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:48.486492  2055 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:48.498562  2021 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:48.527973 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:48.459671  2126 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:13:48.538329 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:48.579505 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:48.588094 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:48.589393 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:48.591158 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:48.591966 28227 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:48.539244 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:48.540639 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:48.594751 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:48.542084 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:48.596019 28233 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
W20260709 13:13:48.596334 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 68422.2us
I20260709 13:13:48.597234 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:48.536963 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:48.483058  2096 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:48.570914 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 124837us
I20260709 13:13:48.560945 28155 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:48.576534 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:48.612866 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:48.613492 28158 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:48.545018 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:48.509708  2075 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:48.509871  2084 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:48.598889 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:48.630275 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:48.600003 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:48.579593  2114 consensus_meta.cc:287] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.745s	user 0.009s	sys 0.003s
I20260709 13:13:48.600792 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:48.601886 28760 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:13:48.529234  2130 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:13:48.605041  2027 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:48.522104  2101 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:48.608863 28083 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:48.609717 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:48.611317 28232 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:48.617245 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:48.600126 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:48.631076 28160 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:48.575973  2019 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:48.649216 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:48.573885  2028 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:48.660509 28230 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:48.528965  2109 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.106s
I20260709 13:13:48.608017  2029 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:48.630872  1608 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.646394  2084 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:48.675067 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.675508 28158 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:48.658979  2019 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:48.655262  2101 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:48.675675  2059 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.579233  2141 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.186s
I20260709 13:13:48.730903 28758 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [19.15-21.16]
W20260709 13:13:48.692616 28199 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.633s (SENT) [suppressed 27 similar messages]
I20260709 13:13:48.621205  2096 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:48.718133  2140 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
W20260709 13:13:48.724200  2115 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.808s	user 0.009s	sys 0.001s
W20260709 13:13:48.712905  2143 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:13:48.677402  1608 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.717003  1938 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:13:48.723217  2028 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.765236  1938 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:48.717063  2019 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:48.660172  2114 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.771049  2114 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:48.714202  2100 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:48.731329  2141 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:48.674998  2027 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:48.716372  1987 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:13:48.695662  2084 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.748894  2029 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:13:48.762348  2115 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.721372  2021 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.764794  1608 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:48.717362  2101 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:48.677130  2144 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:13:48.782337  2103 consensus_meta.cc:287] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.957s	user 0.012s	sys 0.000s
I20260709 13:13:48.831331  1939 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:48.831715  2115 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:48.836025  2019 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.818311  2027 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.842307  2084 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:48.858299  1980 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.001s
I20260709 13:13:48.859658 28757 raft_consensus.cc:2410] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:13:48.859604 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:48.872560 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 179995us
I20260709 13:13:48.872915 28761 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:48.879355 28154 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:13:48.882920 28154 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:13:48.884742 28152 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:48.811687  2141 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:48.829237  1982 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.874670 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:48.893474 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 280788us
I20260709 13:13:48.894368 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:48.894155  2141 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:48.895124 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 440475us
I20260709 13:13:48.525794  2082 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.904172 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:48.910355 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:48.911300 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:48.911355 28762 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:13:48.913707 28159 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:13:48.915472 28762 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:48.916152 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:48.893564  2096 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:48.895372  2141 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:48.930902 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:48.913678  2144 raft_consensus.cc:2764] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:13:48.928751  2147 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:48.890827  2050 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:48.928222  2102 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:13:48.893931  1980 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:48.953902 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:48.914366  1939 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:48.929728  2155 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:48.949488  2096 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:48.941808  2063 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:13:48.928169  2115 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:13:48.929641  2103 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:13:48.986400 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:48.961553  2082 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:48.977885  1939 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.931237  2021 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:13:49.011111 28235 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:48.977671  2059 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:49.012567  2096 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:48.958899  1655 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:48.987090  2155 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:48.991202  2144 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:49.007727  2063 consensus_queue.cc:1243] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 17.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:13:49.025274  1980 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.018646  2155 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:49.017591 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:49.008181  1939 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:49.033617 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:49.036697 28235 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:48.978201  2050 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.041908  2028 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:49.046877 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:49.041107  2005 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.584s	user 0.006s	sys 0.005s
I20260709 13:13:49.040086  2144 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:49.030648  1982 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:49.053231 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:49.053897 28758 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
W20260709 13:13:49.050379 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 119559us
I20260709 13:13:49.042030  2055 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:13:48.973034  2147 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.056046 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:49.055368 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:49.056262 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:49.057260 28198 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:49.054517  2150 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 13:13:49.047883  2167 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:49.061020 28762 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:49.061336 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:49.064879  2153 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:13:49.065207  2019 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:49.066350  2152 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:13:49.058504  2055 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.067906  2055 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.068899  1655 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:49.069221  2055 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:13:49.069337  2166 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:13:49.069563  1982 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:49.070000  1982 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:49.070359  1982 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:49.053669  2005 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:49.053491  2027 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.062211  2028 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:49.070848  1990 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:13:49.071420  2028 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:49.071841  2028 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:49.071919  1990 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:49.072400  1990 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:49.073271  1990 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.074466  1990 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:49.078070 28760 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:13:49.076936 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:49.076336  2103 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:13:49.062662  2167 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.079437 28761 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:49.078552  2070 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:13:49.079092  1988 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:49.080644  2147 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:49.099211  2063 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 13:13:49.083945  1939 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.106804 28755 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:49.085726  2029 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:13:49.116521 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:49.117298 28762 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:49.099526  2019 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:49.115423  1939 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.080811  2014 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 13:13:49.101130  2070 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:49.105985  1988 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.081360  2101 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.096850  2075 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.138358 28759 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [19.15-21.16]
I20260709 13:13:49.085574  2167 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.125784  1988 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:49.152908 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:49.154682 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:49.155331 28161 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:49.157054 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:49.157688 28154 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:49.157804 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:49.158509 28158 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 12.
I20260709 13:13:49.159828 28152 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:13:49.127092  2059 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:13:49.170362  2050 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.170181  1982 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:49.173077  2075 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:49.160714  2070 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.174016  2070 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:49.174113 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:49.177728  2101 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:49.152768  2175 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:13:49.181330 28083 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:49.182555 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.183138 28086 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:49.152662  1939 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:13:49.165151  2167 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:49.185175 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 675356us
I20260709 13:13:49.186102  1980 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:49.186509  1980 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.190023 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:49.192332 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.194487 28154 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:49.151355  2019 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.200380 28159 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:13:49.172612  1982 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:49.206216 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:49.210243 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:49.199896 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 550740us
I20260709 13:13:49.219039 28154 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:49.219460 28050 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:49.212677 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:49.220358 28156 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:49.209622  1744 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.221668  2149 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:49.222078  1744 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.224316  1744 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:49.207142  1845 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.225065  1845 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.226171  2035 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.226681  2035 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.227257  1845 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:49.230669  2158 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.344s	user 0.002s	sys 0.000s
I20260709 13:13:49.197009  1947 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:49.234452 28759 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:13:49.211174  2034 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.002s	sys 0.008s
I20260709 13:13:49.237356 28154 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:49.250144 28758 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:13:49.248997 28156 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:49.235127  2035 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.239471  2019 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.238970  1744 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:49.235940  2100 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:13:49.210896  2092 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:13:49.236235  1980 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.268338 28762 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
W20260709 13:13:49.256585  2163 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.369s	user 0.000s	sys 0.002s
W20260709 13:13:49.240692  2165 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.352s	user 0.002s	sys 0.000s
W20260709 13:13:49.249857 28757 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35896 (request call id 1180) took 1548 ms (client timeout 1954 ms). Trace:
W20260709 13:13:49.294210 28757 rpcz_store.cc:269] 0709 13:13:47.701023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.701337 (+   314us) service_pool.cc:224] Handling call
0709 13:13:49.249619 (+1548282us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400563954. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:49.249779 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2579044096}
I20260709 13:13:49.250082  2102 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:49.313753  2102 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:49.314780  2102 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.315846  2102 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:49.253722 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3209) took 1111 ms. Trace:
I20260709 13:13:49.316920 28234 rpcz_store.cc:276] 0709 13:13:48.142015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.142825 (+   810us) service_pool.cc:224] Handling call
0709 13:13:48.143219 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:48.314022 (+170803us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.314070 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:48.314589 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:13:48.751645 (+437056us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:48.752353 (+   708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.253347 (+500994us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400355cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:49.253399 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.253442 (+    43us) raft_consensus.cc:1685] finished
0709 13:13:49.253456 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.253661 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47576,"raft.queue_time_us":18583,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":17587,"spinlock_wait_cycles":351262464,"thread_start_us":367009,"threads_started":1,"wal-append.queue_time_us":509055}
W20260709 13:13:49.269203  2034 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.005s	sys 0.008s
I20260709 13:13:49.268997 28758 pending_rounds.cc:85] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 11
I20260709 13:13:49.319833 28758 pending_rounds.cc:107] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:13:49.320166 28758 raft_consensus.cc:2889] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:49.322000  2172 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:49.253522  1908 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:49.322647  2172 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.322892  1908 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.323498 28153 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:49.323565  2172 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.323839  1908 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.247437  1957 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:49.324927  2172 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:49.325084  1957 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.326042  1957 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.327371  1957 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:49.237780 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:13:49.329552 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 91817.9us
W20260709 13:13:49.329731  2161 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.441s	user 0.002s	sys 0.000s
I20260709 13:13:49.330121 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:49.332427 28086 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:49.333357  2147 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:49.300841  1982 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:13:49.289152 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:49.335402  2147 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:13:49.335652 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 46601.5us
I20260709 13:13:49.336346  1856 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:49.295270  2115 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:49.253146  2155 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.247107 27981 catalog_manager.cc:5697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 9 to 12, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 12 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:49.336956  1856 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:49.337736 28234 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:13:49.338191 28234 pending_rounds.cc:85] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 7
I20260709 13:13:49.338469 28234 pending_rounds.cc:107] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 18.8
I20260709 13:13:49.338040  1856 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.338692 28234 raft_consensus.cc:2889] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:49.336920  2115 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:49.307861 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:49.308502 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:49.307401 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:49.341177 28226 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:49.244663  2028 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.343305  2028 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.343890 28760 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:49.306648 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:49.301494  1938 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.310274 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:49.347858  1837 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:49.348603  2084 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:13:49.336874  2141 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:49.349406 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:49.349567  2141 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.351110  2084 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:49.351464  1950 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:49.351643  2084 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:49.352491 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:49.352852  2141 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:49.353919  2096 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:49.354362  2096 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:49.354733  2096 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:49.356137 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:49.356937 28152 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:49.358130 28154 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:13:49.322013  2034 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:49.360476 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:49.361085 28152 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:49.339902  2115 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.337850 27981 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:49.365638 28229 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:49.366562 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:49.367189 28229 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:49.352905  2084 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.368665 28229 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:49.351505 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:49.371541 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:49.372385 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:49.326180  1988 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.379926 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:49.384068 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:49.401276 28083 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:13:49.401818 28755 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:13:49.391424 28159 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:13:49.392645 28156 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: [19.12-19.12]
W20260709 13:13:49.393503 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.401839 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:49.393446 28085 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:49.379798 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:49.404542 28155 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:49.393724  2101 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:49.406536  2101 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:49.407948 28152 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:49.409026 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.394127 28161 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:49.410367 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:49.411670 28159 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:13:49.412425 28230 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:49.324919  1908 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:49.413116 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:49.426781 28231 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:13:49.427125 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.427608 28155 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:13:49.418947 28083 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:49.428190  1845 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:13:49.428961 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:49.430481  2155 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:49.430641 28155 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:13:49.432288 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:49.432142  2127 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:13:49.433285  2127 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:49.423993 28152 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:13:49.429100  2100 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:13:49.434063 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:49.434634  2100 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:49.435135  2100 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:49.435981  2100 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.408540  2141 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:49.410050  2021 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:13:49.438174 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:49.379025  2147 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.433712  2127 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:49.423041 28080 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:49.441972 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:49.420547 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:49.441983 28152 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:13:49.442502 28155 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:13:49.443940 28083 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:13:49.454031 28755 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:49.458662 28157 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:49.422883  1982 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:49.450506  2082 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.455546 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:49.463806 28083 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: []
I20260709 13:13:49.413424  2023 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.474030 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.454284 28155 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:49.471369 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.441458  2154 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.477828  2154 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.399878  2172 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:13:49.400261  2028 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:13:49.487953 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:49.488336 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.469681  2175 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:49.360471  2034 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.484220 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.411151  1990 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:13:49.417847  2146 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:13:49.483347 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:49.423108  2167 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
W20260709 13:13:49.492962 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.444854  2059 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:13:49.468978  1982 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:49.455685  2147 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.493940 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.495800  2063 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:13:49.496419  2101 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.481730  2155 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:49.481498  2154 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.453099  2141 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:13:49.500553 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.493428 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:49.498679 27981 catalog_manager.cc:5697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 6 to 7, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 7 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:49.503791 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:49.508126  2179 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:13:49.542160  2141 consensus_queue.cc:1243] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:13:49.542781  2178 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:49.543188  2178 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.543999  2178 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.503783  2127 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.521544 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.534384 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.482013  2082 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.484104  2023 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.567729  2115 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:49.539881 28084 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:49.513322  1990 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:13:49.545010 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 51702.8us
I20260709 13:13:49.574720 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:49.575866 28758 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:49.546355  2034 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.547749 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:49.579980 28161 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:49.547879  2096 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.549587 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.550689 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:49.552537  1837 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:13:49.553377 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.501744  2175 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.520320  2035 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:49.589298  2035 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:49.589769  2175 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.590515  2035 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.592223  2035 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:49.526168  1982 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:49.560235 27981 catalog_manager.cc:5697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:49.594471 27981 catalog_manager.cc:5697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:49.576562 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:49.545653  2021 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:49.581315 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:49.597815 28161 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
W20260709 13:13:49.586735 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.597088 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:49.597982  2096 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.602437 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.603274 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.604374 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.612697 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.614634 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.624644 28077 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:49.625422 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.627398 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.628980 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.604269 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:49.630147 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 79543.7us
I20260709 13:13:49.630429 28077 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:13:49.631215 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:49.632194  1449 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:49.553819  2155 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:49.636656 28084 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:13:49.583930  1990 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:49.625526  1891 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:13:49.632922  2159 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.743s	user 0.002s	sys 0.004s
I20260709 13:13:49.647053 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.652076 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:49.639979  1656 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:49.636826  2178 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:49.660132 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:49.649910  2188 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:13:49.625123  2005 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:13:49.664888  1449 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:49.667158 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:49.640799  2127 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:49.650691  2019 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:49.692289 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:49.640164  2149 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:13:49.693811 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.684588  2195 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:49.697412 28084 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:49.665416  1891 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.662241  1988 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.682829  1656 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:49.730618 28084 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:13:49.734849  2147 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:49.740940  2134 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:49.698937  2019 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:49.682433  2031 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:13:49.744529  2019 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.745455  1449 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:13:49.746544 27981 catalog_manager.cc:5697] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 8 to 11, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 11 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:49.746232  1449 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:49.757934  1449 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:49.700484  2195 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.759959 28227 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:13:49.760035 28228 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
W20260709 13:13:49.762883 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 158706us
I20260709 13:13:49.703234  1655 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:49.775496  1449 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.776713  1449 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:49.795223 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:49.799978 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:49.766399  2183 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:13:49.746343  2031 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:49.779292  1656 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.832718  2031 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:49.833689  2031 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.835027  2031 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:49.753504  1837 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:49.766899  2192 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:13:49.813608  2196 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:13:49.849875 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 273381us
W20260709 13:13:49.814535  2184 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 13:13:49.815443  2181 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:13:49.818668  2186 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 13:13:49.862015 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:49.820221  2102 consensus_meta.cc:287] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.504s	user 0.005s	sys 0.006s
I20260709 13:13:49.781653  1947 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:13:49.829067 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.877151 28086 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:49.829741  1938 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:49.829073  1891 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:49.831542  2176 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:49.768852  1856 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.793087  1845 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:13:49.852079 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:49.852983 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:49.851065 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:49.777820  2195 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:49.853806 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:49.863154 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.860532 28158 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:49.794337  2175 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:49.867540 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.890125 28081 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:13:49.812482  2187 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:13:49.845853  2092 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:49.747435  2155 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:49.792870  1744 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:13:49.914626 28228 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:49.912146 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:50.180130 28077 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:13:50.015462 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:50.129552 28078 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:13:50.182588 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:50.117118  2196 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:13:49.959185 28085 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:49.876487  2021 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:50.183277 28229 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:50.027107 28086 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:13:49.912885 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:49.891436  2034 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:50.205246 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:49.959504 28760 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
W20260709 13:13:50.208168 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.205674 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3218) took 1015 ms. Trace:
I20260709 13:13:50.209671 28232 rpcz_store.cc:276] 0709 13:13:49.190234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.190847 (+   613us) service_pool.cc:224] Handling call
0709 13:13:49.191133 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:49.210797 (+ 19664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.210827 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:49.211778 (+   951us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.443355 (+231577us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:49.443410 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.443475 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.205473 (+761998us) raft_consensus.cc:1685] finished
0709 13:13:50.205493 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.205618 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19290,"raft.queue_time_us":20519,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":16710,"thread_start_us":185400,"threads_started":1,"wal-append.queue_time_us":367207}
I20260709 13:13:49.875269  2028 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:50.086009  2191 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 13:13:50.044651  2185 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.472s	user 0.000s	sys 0.003s
W20260709 13:13:50.044312  2194 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:13:50.186693  1950 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.220645 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.221771 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:50.222501 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.223376 28758 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:50.184149  1908 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.228581 28078 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:13:50.206991  2115 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.002s
I20260709 13:13:50.002749  2154 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:50.227144  1982 consensus_meta.cc:287] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.001s
I20260709 13:13:50.247249 28759 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:13:50.247785 28759 pending_rounds.cc:85] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 14
I20260709 13:13:50.248088 28759 pending_rounds.cc:107] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:13:50.248425 28759 raft_consensus.cc:2889] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:50.183259  2175 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:49.950096  1856 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.271256  2201 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:50.246610  2178 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.001s
I20260709 13:13:50.276983  2145 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.216874  2034 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.276358 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.208992  2092 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.310294  2034 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.310850  2034 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:13:50.219564  2028 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.312487 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:50.258189  2075 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.316924 28764 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:50.282783 28049 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:50.246968  1908 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:50.311348  2092 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:50.312851 28763 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:50.364270 28764 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:13:50.397953 28230 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:13:50.398456 28230 pending_rounds.cc:85] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
I20260709 13:13:50.398741 28230 pending_rounds.cc:107] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:13:50.398964 28230 raft_consensus.cc:2889] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:50.399176 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 118136us
I20260709 13:13:50.217952  2176 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:50.407887 28156 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:13:50.244906  1950 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.413889  1908 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:50.251184  2154 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.260720  1982 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:50.456604 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1205) took 1131 ms. Trace:
I20260709 13:13:50.406885  2146 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:50.406518  2145 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:50.469507  2212 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:13:50.470430  2175 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:50.471468  2204 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 13:13:50.474037  2176 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:50.475590  2028 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.475699  2082 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:50.452276  1908 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.477358  2134 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:50.480782  2100 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.353677  2021 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:50.481693  1816 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:50.482590 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.569495 28764 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:50.483649 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.570428 28761 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
W20260709 13:13:50.574708  2214 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:13:50.457283 28153 rpcz_store.cc:276] 0709 13:13:49.325825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.325953 (+   128us) service_pool.cc:224] Handling call
0709 13:13:49.326267 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:49.326628 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.326656 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:49.327131 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.471089 (+143958us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:49.471132 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.471197 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.835314 (+364117us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:50.417666 (+582352us) raft_consensus.cc:1685] finished
0709 13:13:50.417692 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.456508 (+ 38816us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1101,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":40164736,"thread_start_us":142804,"threads_started":1,"wal-append.queue_time_us":181294}
I20260709 13:13:50.487975 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:50.504137  1837 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:50.505772 28760 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:13:50.490274  2127 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.801s	user 0.011s	sys 0.000s
I20260709 13:13:50.526706  1856 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.398082  2075 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.409410  1744 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:13:50.485913  2092 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.416447  1656 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 13:13:50.486269  2146 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.610215  2146 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:50.610559  1938 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.719s	user 0.011s	sys 0.001s
I20260709 13:13:50.571764  2100 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.611444  1990 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.362455  2201 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:50.614261  2031 consensus_meta.cc:287] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.763s	user 0.008s	sys 0.004s
I20260709 13:13:50.614530  2201 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.615442 28160 raft_consensus.cc:2410] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:13:50.617630  2145 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:50.617755 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3023) took 1236 ms. Trace:
I20260709 13:13:50.618381  2100 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:50.618233 28160 rpcz_store.cc:276] 0709 13:13:49.381831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.381958 (+   127us) service_pool.cc:224] Handling call
0709 13:13:50.617532 (+1235574us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400238554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:50.617688 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2768574720}
I20260709 13:13:50.619287  1449 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:50.599330  2028 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.619711  1449 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:50.620127  2028 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.623098 28083 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:50.624995  2195 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.626266 28758 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:13:50.622218  2028 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.628618 28234 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:50.624486 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2704) took 1137 ms. Trace:
I20260709 13:13:50.629686 28083 rpcz_store.cc:276] 0709 13:13:49.487552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.487681 (+   129us) service_pool.cc:224] Handling call
0709 13:13:50.624328 (+1136647us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54000e1d54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:50.624425 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2558378240}
I20260709 13:13:50.630656  2195 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:50.631999 28229 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:50.632098  2023 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:50.634692 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:50.636294  2092 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.638681  2092 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:50.640429  2154 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.645530  1982 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:50.646318  1982 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:50.646476  2209 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.193s	user 0.001s	sys 0.002s
I20260709 13:13:50.646929  1982 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:50.563427 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 341724us
W20260709 13:13:50.564368  2035 consensus_meta.cc:287] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.972s	user 0.009s	sys 0.006s
I20260709 13:13:50.500173  1947 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:13:50.565040 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 163808us
I20260709 13:13:50.652719 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.567073  1908 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:50.596200 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 108343us
I20260709 13:13:50.658221 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:50.572372  1816 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:50.600219  2034 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:50.651811 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:50.565179  2134 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:50.661526 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:50.666038 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.656126 28758 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:50.671487  2219 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:13:50.671769  2134 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:50.668953  1856 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:50.672930 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.673617  1856 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:50.673764 28761 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:50.674851 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:50.674615  1856 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.676178 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:50.659641 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:50.677522 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.677800 28124 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:50.679001  2219 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:50.679457  2219 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:50.679832  2219 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:50.680339  1938 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:50.680814  1938 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:50.669806 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:50.681191  1938 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:50.682127 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:50.686082  2075 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.667754  2055 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.663645  2154 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:50.601984  1837 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:50.669440  2034 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.688763  1974 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.701072 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.715715 28760 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
W20260709 13:13:50.713634 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:50.730386  2220 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:13:50.714462  2154 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.706013  2033 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:13:50.710843  2103 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:50.734164  2103 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.735005  1908 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.669816  1947 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:50.741410 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.748307 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.718487 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2711) took 1169 ms. Trace:
I20260709 13:13:50.706254  2127 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:13:50.778992  2096 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:50.754622 28199 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.030s	sys 0.040s
I20260709 13:13:50.699177  2034 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:50.762714 28079 rpcz_store.cc:276] 0709 13:13:49.549528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.551605 (+  2077us) service_pool.cc:224] Handling call
0709 13:13:49.551944 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:49.572993 (+ 21049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.573056 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:49.573565 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.662927 (+ 89362us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:49.662969 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.710246 (+1047277us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b540017e154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:50.710292 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.710338 (+    46us) raft_consensus.cc:1685] finished
0709 13:13:50.710356 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.718382 (+  8026us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":796,"raft.queue_time_us":1039,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":41886,"spinlock_wait_cycles":47284608,"thread_start_us":89014,"threads_started":1,"wal-append.queue_time_us":184078}
W20260709 13:13:50.782523 28199 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.030s	sys 0.042s
I20260709 13:13:50.761236 28760 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
W20260709 13:13:50.783034 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 843989us
I20260709 13:13:50.700220  2028 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:50.783921 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.785955 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.792129 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:50.730245  1837 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:50.732662  2154 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.776166  1982 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.795683 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:50.702967  1980 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:50.794445 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 112389us
I20260709 13:13:50.825289  2172 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:50.825946  2172 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:50.828348  2172 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:50.736370  1974 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.830341  1974 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.713027  2055 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:50.707410  2126 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:13:50.833914 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:50.783890  2103 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:50.787551  2059 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.835966 28125 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:50.838044 28125 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:50.839735 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:13:50.833101  2224 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:13:50.819171  2176 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:50.850050 28125 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:50.790640  1908 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:13:50.836306  1982 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:50.797379  2075 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:13:50.735911  1816 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:13:50.862838 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 160291us
I20260709 13:13:50.842602  2207 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:50.842060  2028 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:50.869907 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:50.870630 28079 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:50.870651 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:50.843986  2228 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:50.873181 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.836892  2154 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:13:50.873692 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:13:50.878219 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.843811  2178 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:13:50.872596  2149 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:13:50.883244 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.844154  2059 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.845820  1938 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.877326  2035 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:50.884487 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:50.903548 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:50.880965  2075 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:50.879413  2028 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:50.884100  2228 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.879626  2031 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 13:13:50.917667  2213 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.445s	user 0.002s	sys 0.000s
I20260709 13:13:50.920126 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3042) took 1136 ms. Trace:
I20260709 13:13:50.920516 28159 rpcz_store.cc:276] 0709 13:13:49.783732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.783939 (+   207us) service_pool.cc:224] Handling call
0709 13:13:49.784425 (+   486us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:49.784991 (+   566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.785039 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:49.785826 (+   787us) log.cc:844] Serialized 52 byte log entry
0709 13:13:50.186531 (+400705us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:50.639140 (+452609us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.639497 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.640468 (+   971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.640560 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.917063 (+276503us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:50.919852 (+  2789us) raft_consensus.cc:1685] finished
0709 13:13:50.919876 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.920062 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":452352,"prepare.queue_time_us":452731,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":10329,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":31528,"spinlock_wait_cycles":43006208,"thread_start_us":129037,"threads_started":1,"wal-append.queue_time_us":635040}
I20260709 13:13:50.923457  2176 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:50.923910  2176 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:50.914078  2075 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.888932  1816 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:50.881515  1856 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:50.884454  2149 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:50.935292 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2734) took 1074 ms. Trace:
I20260709 13:13:50.924711  2176 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.938372  2055 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:50.938937  2176 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:50.939299  2126 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:13:50.940055  2126 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:50.941444  2031 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:13:50.941846  2031 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:50.942093  2023 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:13:50.942988  2227 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:50.942778  2031 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:50.943564  2227 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.944200  2031 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:50.935930 28154 rpcz_store.cc:276] 0709 13:13:49.860645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.860866 (+   221us) service_pool.cc:224] Handling call
0709 13:13:49.861165 (+   299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:50.317424 (+456259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:50.317454 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:50.403936 (+ 86482us) log.cc:844] Serialized 52 byte log entry
0709 13:13:50.600059 (+196123us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:50.600122 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.707849 (+107727us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b5400317754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:50.707903 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.927514 (+219611us) raft_consensus.cc:1685] finished
0709 13:13:50.927534 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.935197 (+  7663us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":764,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":48915,"spinlock_wait_cycles":1047291136,"thread_start_us":134426,"threads_started":1,"wal-append.queue_time_us":240424}
I20260709 13:13:50.950325  2219 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:50.926460  2203 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.169s	user 0.001s	sys 0.002s
I20260709 13:13:50.913656  2155 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:50.936296 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 258556us
I20260709 13:13:50.951151  2059 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.926965  2019 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:50.948446  1947 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:50.974781 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:50.990926 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 140965us
I20260709 13:13:50.979650  2084 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.979528  2155 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.917696  2228 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.985388  2075 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:50.946036  2227 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:50.965180 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 122161us
I20260709 13:13:50.992267 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:51.000753 28760 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 19: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:50.903630  2146 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:50.968614  1856 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:51.000123 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 118622us
W20260709 13:13:51.004556 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 101115us
I20260709 13:13:51.004673 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:51.006131 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:50.989586  2019 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:51.009933 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 136311us
I20260709 13:13:51.018512 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.018884 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.005741  2149 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.021694  1990 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:51.022008  2149 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:51.022868  2230 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.023319  2225 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.023384  2230 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.023985  2225 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.026013  2230 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:51.026170  2225 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:51.028115 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.029346  2023 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.029511  2019 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.029899  2023 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.030818  2019 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:51.032042  2023 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.033166  2241 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.033774  2241 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:51.034931 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.033324  2134 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:51.036906 28085 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:13:51.030827  1449 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.038050  1449 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:51.039323 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:51.042017 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.042675 28079 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:13:51.042225 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.047978 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.048652 28078 raft_consensus.cc:2376] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19. Re-sending same reply.
I20260709 13:13:50.989120  2154 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:13:51.050262 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.051743 28083 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:13:51.052258 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.054136 28080 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:51.052371 28086 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:13:51.003485  2028 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:51.057904 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.079579 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.058126 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.080195 28232 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:51.061157 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:51.053772  1856 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.061700 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:51.088222 28761 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:51.063824  1950 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.064352 28083 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:13:51.065348 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:51.065790  2229 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:13:51.068006 28082 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:13:51.069016 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.078274  2221 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:13:51.018232  2075 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
W20260709 13:13:51.074947 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.035892  2241 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:51.110996  2155 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.078358  2034 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.111321  2219 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.111819  1990 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:13:51.103823  2221 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:51.113121  2034 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.006410  2227 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:13:51.113436  2221 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.079371  2028 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.077348  1938 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.114642  2028 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:51.115540 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.116882  2201 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:51.117578 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:51.053134  2096 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:13:51.053387  2237 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:13:50.997638  2084 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.122182  2228 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.122474  2084 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:51.099087 28081 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:13:51.073701  2018 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:13:51.125738 28198 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:51.126464  2207 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:51.127163 28228 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:51.128198 28198 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.129328  1891 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:13:51.129505  2175 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:51.129946  1891 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:51.130096  2178 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.130635  2178 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.130861  2175 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:51.131393  2175 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:51.130949  1891 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.132232  2175 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.132671  2178 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.133554  2101 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:51.142669  1856 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:51.142616  2101 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.144922  2101 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:51.097419 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.112082  2236 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:51.157042 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:51.157346 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:51.158196 28758 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:13:51.158993 28757 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:13:51.113827  2146 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.117398  2201 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.098641  2182 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
W20260709 13:13:51.085690 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 66150us
I20260709 13:13:51.171403 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.146245 28235 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:13:51.172183 28227 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:13:51.151871 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:51.153630 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.154532 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.106149 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:51.100335 28085 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:13:51.174893 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:51.168865 28760 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:51.145577 28759 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:13:51.088886  2059 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:51.110466  2226 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:13:51.173264 28764 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:51.093006  2021 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.173135 28081 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
W20260709 13:13:51.203621 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 75501.7us
I20260709 13:13:51.205272 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.206496 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.206774 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.212791 28234 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:51.206354 28755 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:13:51.208357  2021 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.214653 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.215386 28227 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:51.133695  2175 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:51.211580 28198 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.217752 28159 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:13:51.217974 28235 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:51.219065 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 214462us
W20260709 13:13:51.219636 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 47333.4us
I20260709 13:13:51.221004 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:51.223444 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.219004 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.217397 28198 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.230490 28159 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:51.234073 28078 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:13:51.234962 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:51.236124 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.239193 28158 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:51.236996 28155 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:51.240212 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.232246 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.242029 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:51.242400 28085 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:51.242730 28755 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:13:51.243422 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.244329 28227 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:51.246867 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.247514 28155 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:51.244926 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.250000 28155 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:51.254112 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.255182 28084 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:51.258037 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.261343 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:51.262379 28154 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 15: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:51.222084  1982 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.171468  2236 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.209792  2059 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.230834  2193 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:13:51.241071  2146 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 13:13:51.267160 28156 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:51.297782 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1206) took 1016 ms. Trace:
I20260709 13:13:51.448904 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:51.451933 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 234581us
I20260709 13:13:51.415802  2201 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:51.453604 28198 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.267613  2055 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:51.455462  2207 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:51.455895  2207 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.370826  2096 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:13:51.269536  2084 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:51.456909  2075 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:51.453550  2201 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:13:51.457688  2201 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.458093  2201 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:13:51.454794 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 222903us
I20260709 13:13:51.268371  2228 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:51.460167 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:51.249307  1856 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.215s
I20260709 13:13:51.461591 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:51.462648 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:51.229497  2063 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:51.465780 28230 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:51.466032  2019 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:51.465965  1980 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:51.324836  2021 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.459350 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.467067  2145 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:13:51.466826  2028 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.465114  2167 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:51.468765 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.469153  2178 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:13:51.456499  2207 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:51.242663  2005 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:13:51.471487  1982 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.474635 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.475381 28755 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:51.579838 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:51.274724 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.265527  2238 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.211s
I20260709 13:13:51.269793  2182 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:13:51.477216 28155 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:51.363106 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 105136us
I20260709 13:13:51.552554  1957 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:51.488245  2134 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.489171  2149 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.497244 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.497406 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:51.618218 28152 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:51.619388 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.619658 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.497141 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.503537 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:51.271236 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:51.512076 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:51.621927 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.460064  2228 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:51.440099  2176 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.002s
I20260709 13:13:51.449604 28756 rpcz_store.cc:276] 0709 13:13:50.281405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.281927 (+   522us) service_pool.cc:224] Handling call
0709 13:13:50.355458 (+ 73531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:50.360150 (+  4692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:50.360184 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:50.360662 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:13:50.949827 (+589165us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b1800366fa0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:13:51.002686 (+ 52859us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:51.002745 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:51.003044 (+   299us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:51.276576 (+273532us) raft_consensus.cc:1685] finished
0709 13:13:51.276610 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:51.297686 (+ 21076us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4245,"raft.queue_time_us":5055,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":11037,"spinlock_wait_cycles":30475136,"thread_start_us":52647,"threads_started":1,"wal-append.queue_time_us":664827}
I20260709 13:13:51.550683 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.551981  2227 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:51.552352 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:51.626140 28227 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
W20260709 13:13:51.626752 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 74508.4us
I20260709 13:13:51.626888 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.627658 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:51.370399  2237 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:51.456579  2096 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.553529 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:51.546852 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.567842  2005 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.628496  2249 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:13:51.638547 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:51.638818 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.634016 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.559235  1891 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.544073  2247 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:51.560050  2263 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:51.454646  2055 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.611644  2034 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.561730 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.564312 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.567837 28156 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:13:51.596685  2219 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:51.569765  2178 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:51.494244  2063 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.561941  2167 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.574290  2059 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:51.494442  2084 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.583369 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:51.583794 28755 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:51.585239 28080 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:13:51.249459  2154 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.218s
I20260709 13:13:51.596401 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.671026 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.672073 28155 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:51.597388 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.560832  2018 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:51.610873 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.623172  1938 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:13:51.631229 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:51.697227 28158 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:51.591696  2182 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:51.644433 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.566174  1845 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
W20260709 13:13:51.562806 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 100091us
W20260709 13:13:51.549587  2223 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.004s	sys 0.013s
I20260709 13:13:51.661034 28756 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 16 LEARNER]: Advancing to term 17
I20260709 13:13:51.633550  2096 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:13:51.640540  2005 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:51.668823 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.731585 28157 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:51.669221 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:51.556298 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:51.655938  2055 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:13:51.697912 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 114263us
I20260709 13:13:51.660668  2249 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:51.630134  2134 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.742956  2134 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.746465  2221 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:51.747805  2249 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:51.748482  2249 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:51.749050  2249 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:51.751590  2178 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:51.753361  2259 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:13:51.755127  2028 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.755661  1656 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:51.756131  2028 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:51.757180 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.632376  2149 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.709043  2182 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:13:51.670078  2253 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.194s	user 0.000s	sys 0.002s
I20260709 13:13:51.663376  2167 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:51.755194  1957 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:51.729888 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.668459  2084 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:51.638036  2237 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.756150  1656 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:51.790484  2237 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:51.633662  2227 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.791339  1656 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.792616  2227 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:51.741163  2264 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:13:51.774431  2028 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:51.793745 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.794124 28159 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:51.794615 28161 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:51.794518  2028 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.795548 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.758697 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:51.723193  2223 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.004s	sys 0.013s
I20260709 13:13:51.649865  2247 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.739182  2238 consensus_queue.cc:1059] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:13:51.758443 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.736315  2201 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.659013  2063 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:51.737141 28198 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.765770 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:51.653033  2263 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.664269  2034 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.666000  2219 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:51.662062  1891 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.778044 28756 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 17 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:51.727012 28158 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:51.781523 28155 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:51.796705 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.796685 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.799597 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:51.702057  2018 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.760012 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:51.803303 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:51.807133  2055 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.807623  2247 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:51.810070 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:51.834661  2018 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:51.835202  2055 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.835402 28229 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:51.835842 28762 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:51.812325 28157 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:51.763970 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:51.813505 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 216189us
I20260709 13:13:51.785038  1957 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:51.838250 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:13:51.839110 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 102072us
I20260709 13:13:51.842546 28124 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:51.847601 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:51.848415 28155 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:51.855990 28152 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:13:51.858361 28161 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:13:51.861950 28227 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:51.847431  2055 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:51.864033 28161 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:13:51.830986  2236 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.828202  1990 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:51.866170 28155 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:13:51.849612  2247 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:13:51.874042 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 172517us
I20260709 13:13:51.850729  2154 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:51.847628  2059 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:51.867408  2055 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.889003  2055 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:51.889704  2266 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:13:51.892942  2263 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:13:51.893944  2154 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:13:51.886966  2261 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:13:51.882655  2247 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:51.869832  2175 consensus_meta.cc:287] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.002s
I20260709 13:13:51.884487  2228 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:51.865125  1982 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.924350  1891 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.884899  1990 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.920003  2247 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:51.895329  2154 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.941076  2149 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:51.933797  2055 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.944475  2247 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.960672  2286 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:51.961421  2286 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:51.973168  2286 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:52.005879 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 378037us
I20260709 13:13:52.008266 28155 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:52.015424 28161 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:13:52.023167 28226 raft_consensus.cc:2410] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:13:52.025637 28155 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:13:52.033651 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 449826us
I20260709 13:13:52.039376 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:52.043953 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 490313us
I20260709 13:13:52.045081 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:52.046561 28758 raft_consensus.cc:2410] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:13:52.046736 28759 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:52.073361  1816 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:52.074563  1845 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:52.078097 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:52.079241 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:52.080117 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.083015  1947 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:52.082688  1957 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.083567  1947 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.084008  1947 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:52.085453  1856 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.086326  1856 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:52.088176 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.043980 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:52.075811  1845 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:52.096715 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 60471.1us
I20260709 13:13:52.100209 28760 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: [18.14-20.15]
W20260709 13:13:52.100543 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 181761us
I20260709 13:13:52.103538  1990 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.104852  2228 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:13:52.107545 28160 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:13:52.108052  2055 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.109977  2287 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:52.110834  1856 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.114655  1957 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:51.944475  2167 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.121124  2219 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.122247  2201 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.127506  1845 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.130198  2247 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:52.131953 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 91157.8us
I20260709 13:13:52.133891 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:52.139395 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 100720us
I20260709 13:13:52.120396  2167 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:52.149458 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 59188.6us
I20260709 13:13:52.140722 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:52.163833  2149 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:52.167335 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:52.167559  2149 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:52.169256 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.170403 28086 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:52.168304  2149 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:52.150188  1990 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:52.189903 28052 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:52.168267 28760 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:52.180537  2149 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.208168  2193 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:52.209244  2193 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.211278  2063 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:52.214567 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.221050  2273 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:52.232717  1449 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:52.233330  1449 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:52.233728  1449 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:52.234763 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:52.205485  1845 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:52.243727  1816 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:13:52.250476 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 110017us
I20260709 13:13:52.250778  1816 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.251389  1816 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:52.255995  2146 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.242645  1845 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:52.257256  1845 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:52.256986  2146 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.260216  2146 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.261288  2055 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:52.261547 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.262074  2055 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.264051 28079 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:52.264576  2055 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:52.271524  2167 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.252350 28730 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:52.272001  1957 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:13:52.272276  2167 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.280385  2167 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.282976  1656 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:52.291229 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:52.301292  2274 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:52.305588  2282 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.093s	user 0.003s	sys 0.001s
W20260709 13:13:52.307610 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 55486.8us
I20260709 13:13:52.310997 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.313570 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:52.313890  2176 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:13:52.314473  2176 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:52.318442  1449 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.323441  2149 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:52.317394 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.200397  2249 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:52.338282  2005 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.013s
I20260709 13:13:52.338883  2005 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.340205  2249 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.332485  2034 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.343194 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:52.344143 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69167.9us
I20260709 13:13:52.346621 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:52.346971 28080 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:13:52.347514 28080 pending_rounds.cc:85] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 13
I20260709 13:13:52.348384 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:52.351346  2145 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:52.352375 28080 pending_rounds.cc:107] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:13:52.352898 28080 raft_consensus.cc:2889] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:52.353312 28080 pending_rounds.cc:107] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:13:52.316319  2176 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.353583  1950 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:52.354107  1950 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:52.354569  1950 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:52.354769  2176 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:52.355288 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:52.353597 28080 raft_consensus.cc:2889] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:52.359938  2267 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:13:52.350217  2274 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:52.366789 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.401597  1449 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.362469  2034 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:52.422933  2292 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:13:52.422995 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:52.424860 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 69723.9us
I20260709 13:13:52.425293 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2527) took 1171 ms. Trace:
I20260709 13:13:52.425946 28082 rpcz_store.cc:276] 0709 13:13:51.253571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.598677 (+345106us) service_pool.cc:224] Handling call
0709 13:13:51.633612 (+ 34935us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.045465 (+411853us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.045522 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.046321 (+   799us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.092451 (+ 46130us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:52.092512 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.142769 (+ 50257us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:52.424838 (+282069us) raft_consensus.cc:1685] finished
0709 13:13:52.424869 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:52.425088 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":411047,"raft.queue_time_us":449846,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":160,"thread_start_us":45507,"threads_started":1,"wal-append.queue_time_us":220571}
I20260709 13:13:52.429582 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:52.430135  1947 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.430859 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.431186 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.431711 28233 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:52.431965 28086 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:13:52.432639 28083 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:13:52.433034 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.433003 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:52.433939 28082 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:52.404418  1744 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:52.365865  1845 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.402724  2146 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.441392 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:52.435935  1947 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:52.446868 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 80633us
I20260709 13:13:52.451414  2146 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:52.454099 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 105967us
I20260709 13:13:52.454357  2146 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:52.450358 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 227273us
I20260709 13:13:52.437350  1744 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:52.465847 28078 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
W20260709 13:13:52.468394  2084 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.681s	user 0.005s	sys 0.015s
I20260709 13:13:52.457746  2167 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:13:52.469539  1656 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.474545 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:52.475267 28760 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:52.475111  1656 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.476475 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:52.477669 28759 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:13:52.479166 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:52.479722 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.473690  2084 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.482535  2084 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:52.483474 28763 raft_consensus.cc:2368] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 20. Current term is 21.
W20260709 13:13:52.485078  2207 consensus_meta.cc:287] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.899s	user 0.011s	sys 0.011s
I20260709 13:13:52.485574  2207 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.487020 28231 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:52.487957  2207 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.489815 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.490051 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.491217 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.493993  1744 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:13:52.498632  2096 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.741s	user 0.005s	sys 0.015s
I20260709 13:13:52.477660  1656 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.499181  2096 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.446024  2005 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:52.476547 28755 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:13:52.506949  2096 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.509047 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:52.510460 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48930.8us
I20260709 13:13:52.513538 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.514240 28226 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:52.513528 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:52.528683 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.530109 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.531872  2145 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.532042 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.532595  2145 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:52.533200  2127 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.533768  2127 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.535874 28082 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:13:52.539075 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.539475  2127 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.542263 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:52.542980 28763 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
W20260709 13:13:52.543964  2294 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.139s	user 0.000s	sys 0.003s
I20260709 13:13:52.535552  1950 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.547506  1845 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:52.548255  2253 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.065s	user 0.000s	sys 0.003s
W20260709 13:13:52.549810  2295 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.003s
I20260709 13:13:52.551175 28198 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:52.552937  2262 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.883s	user 0.000s	sys 0.009s
I20260709 13:13:52.555428  2193 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.556828 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.557824 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.558635 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:52.559746 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:52.560647 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.561743  2219 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:13:52.561923 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48482.9us
I20260709 13:13:52.563009 28760 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:13:52.571066 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:52.594492 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 43600.6us
I20260709 13:13:52.596938 28230 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:52.597849 28235 raft_consensus.cc:2368] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21.
I20260709 13:13:52.598430  2021 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:52.598912  2021 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.599346  2021 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:52.598912 28233 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:13:52.597685 28234 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:52.585592  2219 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:52.585790  2207 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:52.595988  2149 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:13:52.609980  2167 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:52.627384 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.628051  1974 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:52.628717  2075 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.629233  2167 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.631809  1908 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:52.632561  1908 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.635375  1908 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:52.617017  2219 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:52.621484  2149 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:52.660161 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.621981  1947 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:52.662029  2293 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:13:52.659518  2149 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:13:52.667411 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48983.4us
I20260709 13:13:52.651499  1974 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.649854  2096 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:13:52.661109  2268 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.689627 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:52.696986  1974 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.698453 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:52.699124 28756 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:52.699345 28124 leader_election.cc:400] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 20. Current term is 21.
I20260709 13:13:52.699752 28124 leader_election.cc:403] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:13:52.700300 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:52.701093 28764 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:13:52.629266  2075 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.704159  2167 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:52.705955  2075 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.713668 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:52.716138 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 59572.2us
I20260709 13:13:52.721741  2268 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.726432 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:52.726832  2145 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.730536 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:52.730850  1947 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.731834 28082 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:52.733191  2249 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:52.733639  1947 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.734792 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:52.738081 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 127758us
I20260709 13:13:52.739189 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:52.740335  2154 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:52.724387  2268 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:52.716342  2034 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:52.752596  2034 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.753439 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:52.754834  2155 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:52.755393  2034 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.749049  2075 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:52.757055  2155 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:52.757611  2155 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:52.758940  2155 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.756722 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1350) took 1202 ms. Trace:
I20260709 13:13:52.761250 28084 rpcz_store.cc:276] 0709 13:13:51.510948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.581640 (+ 70692us) service_pool.cc:224] Handling call
0709 13:13:51.582093 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.042847 (+460754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.042921 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.044180 (+  1259us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.152602 (+108422us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:13:52.152650 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.159369 (+  6719us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:52.712715 (+553346us) raft_consensus.cc:1685] finished
0709 13:13:52.712758 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:52.712913 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":459606,"raft.queue_time_us":500669,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":410,"thread_start_us":107743,"threads_started":1,"wal-append.queue_time_us":316268}
I20260709 13:13:52.762562 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:52.763542 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.671922  2207 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.754168 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:52.765169 28756 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:52.755621 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:52.743126 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:52.766878 28764 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 23.
W20260709 13:13:52.758790  2306 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
I20260709 13:13:52.769737 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.773715 28078 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:52.774479 28084 raft_consensus.cc:2393] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 21: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:52.751837  2305 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:13:52.786057 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:52.790760 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:52.784358 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:52.745682  1950 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:52.804760 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.792043 28763 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:13:52.807682 28082 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:52.742767  2167 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:52.810750 28756 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:52.811965 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:52.815968 28764 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:52.819620 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 81362.2us
I20260709 13:13:52.814757 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:52.820663 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:52.820961 28083 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:52.826984 28755 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:13:52.830250  1449 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:52.830749  1449 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.831125  1449 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:52.799896  2154 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.834821 28052 leader_election.cc:400] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21.
I20260709 13:13:52.835186 28052 leader_election.cc:403] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:13:52.811149  2034 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:52.822690  2304 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:13:52.843767  2019 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:52.844308  2019 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:52.844719  2019 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:52.845541  2182 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.066s	user 0.018s	sys 0.005s
I20260709 13:13:52.846053  2182 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.847354 28085 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 22: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:52.847517 28227 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:52.847702  2230 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:52.848603  2230 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:52.853366 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 112680us
I20260709 13:13:52.855406 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:52.856189  2019 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.861608  2019 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:52.868867  2167 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.873224  2207 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:52.818269  2096 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:13:52.879575 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 140107us
I20260709 13:13:52.877707 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:52.847875 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 1094) took 1604 ms. Trace:
W20260709 13:13:52.886302 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 132987us
I20260709 13:13:52.889912 28233 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:52.894850 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.895891 28760 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:13:52.872443  2167 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:52.886962 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:52.905359 28078 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:13:52.852495  2230 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:52.885380  1816 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.633s	user 0.007s	sys 0.009s
I20260709 13:13:52.909958 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:52.916618 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:52.895188  2145 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:52.917817 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.913758 28084 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:13:52.916769 28079 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:52.885607  1947 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21.
I20260709 13:13:52.919008 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:52.922868 28756 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:13:52.930425 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:52.886291 28761 rpcz_store.cc:276] 0709 13:13:51.193204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.193439 (+   235us) service_pool.cc:224] Handling call
0709 13:13:51.222793 (+ 29354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:51.223938 (+  1145us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:13:52.339268 (+1115330us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b18003ec5c0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:13:52.442688 (+103420us) log.cc:844] Serialized 14 byte log entry
0709 13:13:52.688374 (+245686us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.715452 (+ 27078us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.743813 (+ 28361us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:13:52.743850 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.770522 (+ 26672us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:52.770585 (+    63us) raft_consensus.cc:1685] finished
0709 13:13:52.785163 (+ 14578us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:52.797748 (+ 12585us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1416,"prepare.queue_time_us":515221,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":15692,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":26412,"spinlock_wait_cycles":25463040,"thread_start_us":167188,"threads_started":2,"wal-append.queue_time_us":254499}
I20260709 13:13:52.932986 28760 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
W20260709 13:13:52.935074 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 136777us
I20260709 13:13:52.929466 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:52.940627 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:52.943189 28755 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:13:52.950382  2315 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:52.951088  2315 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.928658  2145 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.918190  2034 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:52.892946  1744 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.957572 28235 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:52.908900  2021 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.970255 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:52.977923 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:52.979914 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:52.981745  2221 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:52.982167  1950 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:13:52.982654  1950 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:52.927992  2096 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:52.983050  2127 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:13:52.983408  2096 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.952145  2075 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:52.988071 28764 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:52.975849  1449 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:52.989328 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 97392.1us
I20260709 13:13:52.924401  1816 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:52.996891 28764 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:13:52.909691  2154 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:52.961241  2145 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:52.984153  2127 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:52.983076  1950 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:53.018898 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:53.010289 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:53.011936 28762 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:13:52.997990 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.013427 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:52.981758  2063 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:52.982239  2221 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:53.026547 28231 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:52.981758  2315 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.030225 28077 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:13:53.034618 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 272254us
W20260709 13:13:53.041550 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 80439.9us
I20260709 13:13:53.042651 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:53.048581 28235 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:13:53.030066  2221 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:53.041342  2134 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:53.051360 28761 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 13.
I20260709 13:13:53.060913 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:53.045152  1908 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:53.050297  2315 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:13:53.070561 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:53.052948  2075 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:53.075335 28763 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:53.074416 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.058344  2134 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:53.062641  2249 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 20. Current term is 21.
I20260709 13:13:53.081370 28755 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:13:53.080801 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 99530.5us
I20260709 13:13:53.070302  1908 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:53.075028  2075 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.092849  1974 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:53.106634 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:53.119865  2134 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.121846  2134 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:53.129694 28083 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:13:53.132232  2063 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:53.131726  1974 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.132717  2063 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:53.133204  2063 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.134908  1974 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.089927  1908 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.085717  2182 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.108212  2075 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:53.149521 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 99828.1us
I20260709 13:13:53.107256  2149 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:53.163200 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 84245.3us
W20260709 13:13:53.173727 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 86291.4us
I20260709 13:13:53.151968  2315 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:53.164240  2096 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.183980 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.186461 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:53.188704 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.171293  1744 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.201889 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:53.202109  2096 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:53.202968 28229 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 22: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:53.202819  2096 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.193936 28198 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.188299 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:53.207312 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:53.210327 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.210578 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:53.211251 28232 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:53.208587 28124 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.211344 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:53.213604 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:53.214478 28227 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.216503 28234 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:53.218571 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:53.219753 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.224154 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:13:53.225519 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 49774.8us
W20260709 13:13:53.205922  2219 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.003s
I20260709 13:13:53.231434  2155 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:53.221814  2084 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:53.236161  2084 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.236542  2084 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:53.236660  2096 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.216841  2182 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:53.257094 28228 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:53.261754  2063 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.275317  2063 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:53.245082  2178 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:13:53.246974  2019 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:53.294518  2019 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:53.295619  2329 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:53.295413  2019 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.296131  2329 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:53.296689  2329 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:53.248162  2207 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:53.298076  2207 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:53.237795  2319 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.299120  2319 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.299209  2207 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.300220 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.215567  1656 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:53.301091 28755 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 22: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:53.301190  1656 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:53.301445  2319 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.213287  2034 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:13:53.302635  2034 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:53.302701 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.303221  2034 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:53.303478 28762 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:13:53.299206 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.297636  1816 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.225055  2288 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:13:53.305086  2288 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:53.302330 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.306216  2019 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.306078  2288 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.257200  2182 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:53.309875  2182 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.314191  2154 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.266058  2145 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.318567  2149 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.319000  2145 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.319556  2021 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.218173  2273 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:53.325050  2273 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.325455  2273 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:53.233101  2167 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 13:13:53.325489 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 101222us
I20260709 13:13:53.203671  2320 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:13:53.328706  2320 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:53.329217  2320 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:53.330179  2320 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.331497  2320 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:53.302306  1656 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.232371 28198 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:53.333739 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 107428us
I20260709 13:13:53.334723 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.233485  2219 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.234310  2193 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:53.336274  2193 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.316454 28762 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:53.338587  2193 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:53.335724  1982 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:53.339421  2021 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:13:53.340042  1982 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.340658  1982 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:53.288372  1449 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.282922  2176 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:53.345155  2315 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.377127 28227 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:53.378275 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:53.382015 28232 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:53.335570 28198 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.381330 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:53.390575 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 142916us
I20260709 13:13:53.392673 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.393314 28761 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:53.393189 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:53.397897 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 62413.9us
I20260709 13:13:53.239365  2155 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:53.398741 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.292218  2084 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.380719  2021 raft_consensus.cc:2764] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:13:53.412533 28234 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:53.413484 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:53.413355 28198 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.420713 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:53.421648 28228 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:13:53.423713 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:53.406172  2167 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:13:53.404641  2127 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:13:53.304309  2034 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.408587  2155 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:53.406940  1656 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:53.409076  1947 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:53.441814 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:53.442757  2084 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:53.443177  2310 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:53.443723  2310 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:53.449774 28229 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:53.454111 28761 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:13:53.451318 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.455698 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.468379 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:53.436372  1947 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:53.444929  2063 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:53.425277  2221 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.456547  1974 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.456358  2167 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:53.470656  2034 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:53.471230  1656 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:13:53.494079 28159 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:53.476787  1947 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:53.457439  1950 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.503073 28756 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:13:53.484431  2334 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:53.508909 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.504390 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:53.518347  1656 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:53.518594 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:53.519487 28158 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:13:53.519593 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:53.522143  2326 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:13:53.488677  1974 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.525485 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:53.526157 28160 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:53.527393  1974 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.532912 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:53.533712 28161 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:53.444659  2310 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:53.539649 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 70924.4us
I20260709 13:13:53.540974  2219 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.551499 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:53.552559  2167 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.552553 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:53.566190  2221 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:53.568415 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.564316  1950 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:53.553498  2335 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:13:53.587956 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:53.589316 28084 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 13: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:53.512094  2334 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:53.595618 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.596868  2219 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.597610 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.597473  2219 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.598284 28762 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:53.600865  2249 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:53.601351 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.601496  2249 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.602128 28758 raft_consensus.cc:2368] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14.
I20260709 13:13:53.575346  1982 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.603080 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:53.604031  1982 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.604213  2249 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.576682  2329 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.519488  1656 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.606873 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:53.609144 28077 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:53.619572  2273 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.612112 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:53.641348 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:53.604785  2334 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:53.633006 28156 raft_consensus.cc:2410] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:13:53.634692 28049 leader_election.cc:400] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14.
I20260709 13:13:53.656652 28049 leader_election.cc:403] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:13:53.600019  2219 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.645195 28762 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:13:53.645503 28228 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
W20260709 13:13:53.649852 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 248120us
I20260709 13:13:53.671090 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:53.670045 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.679278 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.680186 28755 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:53.682901 28082 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:13:53.688050  1449 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:13:53.690454 28761 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 20: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:53.688185 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:53.685467  2075 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.001s
W20260709 13:13:53.691442  1908 consensus_meta.cc:287] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.534s	user 0.002s	sys 0.010s
I20260709 13:13:53.661844  2323 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.673640  2221 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:53.678416  2182 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:13:53.697018 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 156554us
I20260709 13:13:53.704313 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:53.684676  2127 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:13:53.709141 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 497568us
W20260709 13:13:53.711369 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.668370  2019 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:13:53.717769 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.718664 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:53.694784  2219 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:53.684592  2315 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:53.699072  2149 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:53.721791 28232 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:53.707703  2145 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:53.713172  2310 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.735183 28152 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:53.692893  2134 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:13:53.737257  2019 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:13:53.738323  2315 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.739131  2219 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.739921  2154 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.740087  1982 raft_consensus.cc:1064] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:13:53.796751 28080 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:13:53.796844  1982 raft_consensus.cc:1068] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:13:53.797211 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:53.740942  2288 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:13:53.741683  2268 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.798626  2268 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:53.805667  2338 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:13:53.806779 28078 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:53.808028 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:53.813635  2329 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.744410 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:53.745112 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:53.748960  2321 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:53.816387  2321 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.709092  2075 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.818562 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:53.818934  2075 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:53.715274  1908 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.727385  2127 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:53.820958  2127 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.821601  1908 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:53.729200  2182 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:53.726397 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:53.726651  2325 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.778594 28082 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 23.
I20260709 13:13:53.834731 28226 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:53.721948  2063 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:53.795588 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.839521 28758 raft_consensus.cc:2368] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20.
I20260709 13:13:53.796205  2316 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:53.754380 28077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:53.818715  2321 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.713799  2327 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.763706 28161 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:13:53.862390 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:53.708436  2096 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.777805  2019 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:53.725162  2149 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:53.823451  2182 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.840550 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:53.835171  2288 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:53.842028 28762 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:13:53.844419 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:53.845140 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:53.846210  2287 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:13:53.715862  2323 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.848783  2005 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:13:53.903308  2005 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:53.903724  2005 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:53.904831  2327 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.849229 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:53.861343  2154 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.907202  2327 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.908375  2154 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.858309  2315 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.926379  2334 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.849793 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:53.927798 28158 raft_consensus.cc:2368] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21.
I20260709 13:13:53.928841 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:53.929296  2334 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.929488 28158 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:53.873606 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:53.861766 28050 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.683s (SENT) [suppressed 10 similar messages]
I20260709 13:13:53.876564 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:53.800278  2155 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.877524 28156 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:53.880637 28160 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:13:53.934203 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:53.934804 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:53.935746 28160 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:53.935848 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:53.889369  2019 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.893340  2288 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.894392 28228 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:53.889410  1947 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.952965  1947 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.897329  2273 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.898366  2063 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.953843 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:53.954844 28231 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:53.957041 28086 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20. Ops: []
W20260709 13:13:53.901226 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 180164us
I20260709 13:13:53.957621  2221 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14.
I20260709 13:13:53.901818  2219 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:53.861634  2325 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:53.960332  2325 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.911414 28157 raft_consensus.cc:2410] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:13:53.904706  2005 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.788169  2310 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.900000  1816 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:53.962087  2268 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.913491  2323 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:53.913978 28153 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:53.877907  2096 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:53.857961 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:53.965256 28077 raft_consensus.cc:606] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:53.965853 28077 raft_consensus.cc:614] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:53.966667 28082 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:53.992268 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:53.996315 28152 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:53.991331  2005 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:53.997666 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:53.998412 28157 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:53.999364 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.003674  2096 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:54.000491 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.009346 28157 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:53.870291 28755 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:13:53.905620  1845 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:53.934741 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.013970 28153 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:53.880652 28124 leader_election.cc:400] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20.
I20260709 13:13:53.936657 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.014622 28124 leader_election.cc:403] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:13:53.936842  2155 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:54.014236 28156 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:54.018594 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.019196 28156 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:53.948565 28161 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:53.948817 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:53.949694 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:53.956434  2149 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:53.958086  2019 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:53.850200  2341 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:54.024950 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:54.033969 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 80021.1us
W20260709 13:13:54.034638 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 330387us
I20260709 13:13:54.036423 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:54.040620 28085 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:54.041651 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.043273 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:53.962550  1816 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:53.964013 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:53.882685 28756 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:13:54.020886 28762 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:54.028848 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 166530us
W20260709 13:13:54.032399 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 170638us
W20260709 13:13:54.033327 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 169922us
I20260709 13:13:54.049522 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:54.050240 28235 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:53.993796  2021 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:13:54.052945 28152 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:54.044020 28156 raft_consensus.cc:2376] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15. Re-sending same reply.
W20260709 13:13:54.053223 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 61070.4us
W20260709 13:13:54.053750 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 173267us
I20260709 13:13:54.035233 28155 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
W20260709 13:13:54.054854 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 90934.5us
W20260709 13:13:54.072221 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.066063 28198 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:54.087505 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.088498 28228 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:54.091621 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.094031 28762 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
W20260709 13:13:54.095758 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.095894 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:54.111121 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.111604  2075 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:13:54.097127 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:53.991662  1908 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:13:54.098584 28231 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
W20260709 13:13:54.098502 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 62140.3us
I20260709 13:13:54.065207  2348 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:54.102350  2127 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:54.102499  2349 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:13:54.013867  1845 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:54.105038 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.133435 28152 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:54.088193 28156 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:13:54.110221 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:54.127338 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.105751  2193 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:54.146735 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.105188  2149 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:54.113103 28229 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:54.118861  1449 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:54.053411  1816 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:54.130781 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:54.135239 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:54.112440  2063 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:13:54.150529 28078 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:54.150882 28085 raft_consensus.cc:2393] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:54.117584 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.135085  2287 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:54.151768  2287 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:54.152362 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.152451  1845 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.153301 28154 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:54.152799  2287 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.146458  1947 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:54.153926  1845 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:54.154172 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.155226  2134 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:54.155297 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.156118 28154 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:54.155859  2134 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.157389 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.158144 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.158443 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.159085 28156 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 12.
I20260709 13:13:54.159518 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.150029  2193 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.160964 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.161190  2193 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:54.161552 28152 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:54.149288  2034 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:13:54.144086 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.163769 28153 raft_consensus.cc:2393] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:54.164568  2334 raft_consensus.cc:1064] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:13:54.165056  2334 raft_consensus.cc:1068] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:13:54.112228  2145 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:54.167753 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.145656 28235 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:13:54.173849 28154 raft_consensus.cc:2368] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21.
W20260709 13:13:54.175756 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45168us
I20260709 13:13:54.183825 28231 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:13:54.189625 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.190295 28160 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:54.197618 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:54.204946 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:54.207144 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:54.147229  2075 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:54.149622  2310 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:54.132444  2127 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.242465 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:54.221474 28154 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:13:54.243420 28085 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:54.244060 28154 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:54.244882 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:54.221196 27983 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 11 to 15, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 15 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.225011 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:54.222558 28235 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:13:54.224080 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:54.224983 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:54.173669  2315 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:54.266173 28153 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:13:54.230551  2134 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:54.272207 28079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:54.272842 28079 raft_consensus.cc:606] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:54.273154 28079 raft_consensus.cc:614] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:54.273306 28158 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:13:54.231526 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:54.274148 28082 raft_consensus.cc:2410] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:13:54.232257 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:54.125994  2323 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:13:54.154222  1947 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.125548  2096 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 13:13:54.242686  2127 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:54.225809  2167 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:54.285542 28764 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:54.116164  1656 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:13:54.268190 28160 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
W20260709 13:13:54.268622 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44633.5us
I20260709 13:13:54.292879 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.274744 28078 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:13:54.271564 28728 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:54.280625 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.295092 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.213466 28161 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:13:54.287750 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:54.200616  2063 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:54.298715  2063 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.293794 28160 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:13:54.300103  2075 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.302165 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.302855 28154 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:54.304178  1947 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:54.276451  2334 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:54.309795  2193 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.312314  2193 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:54.291798  2323 consensus_queue.cc:1243] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:13:54.313081 28756 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 19: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:54.313839 27983 catalog_manager.cc:5697] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.295769 28158 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:54.315805 27983 catalog_manager.cc:5697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 10 to 14, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 14 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:54.328641 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 54321.3us
W20260709 13:13:54.332134 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 60632.2us
W20260709 13:13:54.332926 27981 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 277) took 63904 ms (client timeout 15000 ms). Trace:
I20260709 13:13:54.332947  1656 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:54.333714  1656 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:54.333775 28728 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:54.335532  2325 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:54.336141  2325 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:54.334954  1656 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.340942 28153 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:13:54.341625 28153 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:54.338903 27983 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:54.343952 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 59309.2us
I20260709 13:13:54.349330 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:54.350281  2349 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:54.350863  2349 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:54.351847 28152 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:54.351912  2349 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.323192  2155 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:13:54.358490 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:54.360383 28762 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:13:54.342480 28153 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.365298 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:54.367986 28079 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:13:54.363020 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:54.372061 28153 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:54.372799 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.286058  2315 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.345378  2193 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:54.394999 28152 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:13:54.391589 28157 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:54.381960 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:54.380056 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46361.2us
I20260709 13:13:54.399893 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:54.398293 28158 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:54.374074  2274 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:54.367328  2155 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:54.413779 28228 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:13:54.429452 28228 pending_rounds.cc:85] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 13
I20260709 13:13:54.429737 28228 pending_rounds.cc:107] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:13:54.430028 28228 raft_consensus.cc:2889] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:54.414599  2268 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:13:54.414613 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.398890  2096 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:13:54.422273 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.424913 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.433329 27983 catalog_manager.cc:5697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:54.436861  2315 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:54.437358  1982 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:54.377315  2319 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:54.441259 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:54.455503 28229 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:13:54.441816  2325 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.469429 28084 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:13:54.469969  2325 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:54.409288  2288 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:13:54.474440 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 180256us
I20260709 13:13:54.450318  2352 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 13:13:54.480527  2149 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.455204  2273 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:13:54.481261  2149 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.482339  2350 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:54.484282  2149 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:54.485385 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.487468 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.376089  2182 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:54.400902  1656 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:54.489351 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.489141 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:54.367102  2221 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:13:54.378958  1974 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:13:54.458178  2155 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.460322  2310 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:13:54.460950  2019 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:54.454910  1982 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:54.465045  1947 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:54.465179 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:54.510532 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.465895 28082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "847d15cbd80f44b18e547291eed25617"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:54.511963 28756 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:13:54.512504 28082 raft_consensus.cc:606] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:54.512995 28082 raft_consensus.cc:614] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:54.403774  2075 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:13:54.467099  2355 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:54.491612 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:54.492404 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.493988 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.494606 28159 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:54.495800 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.506387 28763 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:13:54.513890 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.367522  2145 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.516355 28153 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:13:54.522209 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.524048 28157 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:54.510886 27983 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 7 to 12, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 12 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.461004  2319 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:54.489959  2182 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.536203 28161 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:54.536521 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.495276  2352 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:54.592404 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:54.593036 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.592703 28158 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:13:54.592609 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:54.565791 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.604110 28229 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:54.607067 28152 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:54.543146  2034 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
W20260709 13:13:54.607677 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 51917.4us
I20260709 13:13:54.561398 27983 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 11 to 14, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 14 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.535531  2182 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:13:54.620035 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 102460us
W20260709 13:13:54.619019 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 169133us
W20260709 13:13:54.625867 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 175942us
W20260709 13:13:54.333469 27981 rpcz_store.cc:269] 0709 13:12:50.428333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.428779 (+   446us) service_pool.cc:224] Handling call
0709 13:13:54.332657 (+63903878us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:50.781966 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:50.916018 (+134052us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:50.916089 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:13:50.916114 (+    25us) tablet.cc:662] Decoding operations
0709 13:13:50.925197 (+  9083us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:50.925291 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:50.925328 (+    37us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:13:50.926175 (+   847us) tablet.cc:701] Row locks acquired
0709 13:13:50.926232 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:13:50.926527 (+   295us) write_op.cc:270] Start()
0709 13:13:50.926686 (+   159us) write_op.cc:276] Timestamp: P: 1783602830926452 usec, L: 0
0709 13:13:50.926708 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:13:50.995829 (+ 69121us) log.cc:844] Serialized 2482 byte log entry
0709 13:13:51.136254 (+140425us) spinlock_profiling.cc:243] Waited 7.86 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59cb56d0ac 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b
0709 13:13:51.956453 (+820199us) op_driver.cc:464] REPLICATION: finished
0709 13:13:52.221129 (+264676us) write_op.cc:301] APPLY: starting
0709 13:13:52.221314 (+   185us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:52.228022 (+  6708us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:52.228093 (+    71us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:52.447449 (+219356us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:52.454095 (+  6646us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:13:52.454158 (+    63us) write_op.cc:312] APPLY: finished
0709 13:13:52.454563 (+   405us) log.cc:844] Serialized 65 byte log entry
0709 13:13:52.454834 (+   271us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:52.455051 (+   217us) write_op.cc:454] Released schema lock
0709 13:13:52.455168 (+   117us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:52.455761 (+   593us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:52.455792 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":575262,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":264277,"apply.run_cpu_time_us":15397,"apply.run_wall_time_us":234678,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29291,"num_ops":6,"prepare.queue_time_us":370370,"prepare.run_cpu_time_us":24235,"prepare.run_wall_time_us":1035272,"raft.queue_time_us":161743,"raft.run_cpu_time_us":23431,"raft.run_wall_time_us":293483,"replication_time_us":1029580,"spinlock_wait_cycles":18081536,"thread_start_us":1330346,"threads_started":4,"wal-append.queue_time_us":477891,"wal-append.run_cpu_time_us":25422,"wal-append.run_wall_time_us":2072873}]]}
I20260709 13:13:54.627681 27983 catalog_manager.cc:5697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.629043 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:54.630589 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:54.547750  2075 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:54.529232  1947 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.634687 27983 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 8 to 12, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 12 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.559530  2005 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:54.635205 28756 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:13:54.608673  2155 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:54.598412  2096 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:13:54.626978  2350 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:13:54.644091  2084 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:54.548521  1816 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.665593 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.559954  2355 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:54.672441 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50683.5us
I20260709 13:13:54.679775 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:54.648810  2134 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:54.684361 28083 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:13:54.632894  2221 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
W20260709 13:13:54.662729  2348 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
I20260709 13:13:54.675294 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:54.657356  2149 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:13:54.682874 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2526277130000381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 13:13:54.693889 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0510277129999395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
W20260709 13:13:54.683179 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 53761.3us
I20260709 13:13:54.694182 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.694814 28152 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:54.687564 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:54.648181  2323 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:13:54.635893  2288 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:13:54.695513 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:54.701426 28078 raft_consensus.cc:606] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:54.705044 28077 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:54.705035 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:54.680964  2096 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:54.661805  2005 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.706084 28229 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:54.710986 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.692209  2134 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.702600 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:54.690115  2310 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:54.713019 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:54.701267  2364 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:13:54.693642  2149 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:54.699527  2348 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20.
I20260709 13:13:54.691663  2084 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:54.644632  1982 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.725690 27983 catalog_manager.cc:5697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:13:54.767154 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.779548 28152 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:54.790156 28229 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:54.770735 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 68206.2us
I20260709 13:13:54.700559  2352 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:54.782245 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.708287  2096 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:54.726372  2154 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:54.792821 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66044us
I20260709 13:13:54.729355  2134 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:54.766598  2310 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:54.834681  1845 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.844875  2310 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:54.845045  1845 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.847290  2193 raft_consensus.cc:3055] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:13:54.847432 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:54.847738  1845 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:54.848170 28152 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:54.834190  2352 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:54.849637  2352 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.847870  2193 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:54.836963 28233 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:54.851379  2193 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.852764  2193 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:54.853190 28233 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:13:54.853749  2378 raft_consensus.cc:1064] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:13:54.835290 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.859930  2369 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.834764 28157 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:13:54.861178 28157 pending_rounds.cc:85] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 13
W20260709 13:13:54.861289  2362 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:13:54.861521 28157 pending_rounds.cc:107] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:13:54.861866 28157 raft_consensus.cc:2889] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:54.837093 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.837347 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.837641 28234 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:54.843750 27983 catalog_manager.cc:5697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.836261 28229 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:13:54.869530 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.838121 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.672510  2319 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.876554 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:54.878800 28155 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:13:54.877570 28756 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:54.882552 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:54.883828 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:54.889925 28152 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:54.782748  2315 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:54.891422  1449 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:13:54.897219  1449 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.897637  1449 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:54.899001 28159 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:13:54.903532 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.940927 28758 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 20. Current term is 21. Ops: []
W20260709 13:13:54.941507 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 314901us
I20260709 13:13:54.945930 28760 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:13:54.835912  1816 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:54.948100 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55412.5us
I20260709 13:13:54.880168  2373 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.860518  2369 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:54.951475 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 354498us
I20260709 13:13:54.956424 28763 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:54.957473 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:54.958206 28156 raft_consensus.cc:2393] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 14: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:54.959375 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.892432  2350 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:13:54.968645 28156 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:13:54.968820 28760 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:13:54.969703 28760 raft_consensus.cc:606] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:54.970084 28760 raft_consensus.cc:614] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:54.945647  2021 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.954953 27983 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 9 to 10, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 10 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.837039  2149 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.834923  2084 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.975198 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:54.989172 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.944103  2154 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.998749  1449 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.995685 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:54.726985  2075 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:54.856624  2378 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 12, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.019973 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:55.021114 28079 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
W20260709 13:13:55.021667 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.033558  1449 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.052227 28158 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:13:55.053220 28158 pending_rounds.cc:85] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 11
I20260709 13:13:55.053510 28158 pending_rounds.cc:107] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:13:55.053771 28158 raft_consensus.cc:2889] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:55.014261  2288 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:54.960428  2373 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.069182  2005 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:55.070047  2384 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:13:55.072067  2355 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:55.075968  2370 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:13:54.979851  2021 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.019516  2149 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:55.081099 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.081283 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:55.081673 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.082046 28235 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:55.082159 28232 raft_consensus.cc:2368] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20.
I20260709 13:13:55.082831 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.083104 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:55.057544  2287 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.083520 28235 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:55.083870  2287 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.084323 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.080305 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.089726 28229 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:55.082885 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.009227  1947 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:55.081169 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.090684 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.086659  2096 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:54.998759  2323 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:55.100553  2323 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.102350  2373 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:55.019516  2371 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.084s
I20260709 13:13:55.102922  2323 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:55.008704  2334 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:13:55.105710  2369 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.107765  2134 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:55.066316 28230 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:13:55.105733  2096 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:13:55.088091 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.117756  2134 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:55.119503  2325 consensus_meta.cc:287] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.000s
I20260709 13:13:55.086169  2287 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.141431  2288 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:55.157017  2182 consensus_meta.cc:287] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 13:13:55.159833  2021 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:55.092545 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 301224us
W20260709 13:13:55.072476  2274 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.622s	user 0.007s	sys 0.001s
I20260709 13:13:55.097136 28232 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:55.098721 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.099551 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:55.099920  2154 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:55.166798 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.188503 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:55.160209  1656 consensus_meta.cc:287] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.661s	user 0.013s	sys 0.000s
I20260709 13:13:55.215415  2096 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:13:55.213143 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 179558us
W20260709 13:13:55.217746 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 260762us
I20260709 13:13:55.230145 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.229416 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.230823 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.227133 28226 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:13:55.230283 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:55.231331 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.241155 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.243958 28079 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:13:55.209649  2310 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:55.186934  2274 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:55.252295  2310 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.271173 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:55.272181 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:55.273118 28760 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:55.158150  2182 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:55.276973 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.271878 28758 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:55.264840  2084 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:55.286194 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 61755.9us
I20260709 13:13:55.288781  2274 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.289283  2334 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.290812 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.291600 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:55.363315 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:55.300832 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.368014 28232 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
W20260709 13:13:55.404189 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:55.292394 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 62257.3us
I20260709 13:13:55.410895 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:55.411741 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:55.413667 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2556) took 1214 ms. Trace:
I20260709 13:13:55.414073 28077 rpcz_store.cc:276] 0709 13:13:54.199512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.199643 (+   131us) service_pool.cc:224] Handling call
0709 13:13:55.413458 (+1213815us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54000e0454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:55.413587 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043633920}
I20260709 13:13:55.419178  2182 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:55.420536  2382 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.444s	user 0.001s	sys 0.003s
I20260709 13:13:55.424743 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1238) took 1278 ms. Trace:
I20260709 13:13:55.425289 28155 rpcz_store.cc:276] 0709 13:13:54.146836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.159191 (+ 12355us) service_pool.cc:224] Handling call
0709 13:13:55.424486 (+1265295us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b5400376ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:55.424621 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554581376}
I20260709 13:13:55.432821  2334 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:55.435380  2334 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:55.363354  2274 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.438164  1982 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:55.439703  2274 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.442242 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.453653 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:55.454479 28226 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:55.461123 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 95096.9us
I20260709 13:13:55.461880  2134 raft_consensus.cc:993] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:13:55.462425  2372 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:55.462604  2134 raft_consensus.cc:1081] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:13:55.463129  2372 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.465404 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:55.467105  2145 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:55.467584  2145 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.467964  2145 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:55.469125 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:55.477216 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.222724  2349 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 13:13:55.405061 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:55.406119 28085 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 19: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:55.495846 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:55.501997 28050 leader_election.cc:400] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21.
W20260709 13:13:55.479282 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 498837us
I20260709 13:13:55.505106 28228 raft_consensus.cc:2393] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 19: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:55.269496  2075 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:13:55.513082 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:55.513868 28755 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
W20260709 13:13:55.514672 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 77531.1us
I20260709 13:13:55.516196  2377 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:13:55.517076 28759 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 17 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:13:55.519513  2155 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:55.520249 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:55.521196  2377 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:55.521653  2377 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:55.522040  2377 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:13:55.522289 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 111478us
I20260709 13:13:55.523355 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:55.524431  2154 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:55.524820  2154 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.524955  2323 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.525180  2154 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:55.525586  2323 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.529011  2325 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:55.529485  2325 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.529827  2325 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:55.530179 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:55.530716  2377 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.532305 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:55.533283  2349 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:13:55.533689  2349 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:55.533807  2339 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:55.534941 28159 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:13:55.534708  2349 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.535923  2349 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:13:55.538497 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:55.539196 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.539897 28728 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:55.542439  2327 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.540990 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:55.540802  2377 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.542510  2005 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:13:55.543826  2005 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:55.544953  2327 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:55.544653  2005 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.547444  2327 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.549063 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:55.549953 28759 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:13:55.520161  2155 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:55.552461  2155 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.557829 28760 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
W20260709 13:13:55.557890 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 43531.2us
I20260709 13:13:55.560349 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:55.561424 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:55.563720 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:55.569206 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:55.509450 28235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39712
I20260709 13:13:55.573926 28235 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:55.574376 28235 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:55.574481 28757 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:13:55.592044 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:55.593415 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.565066  2323 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:55.607832 28227 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:55.557514  2134 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.502741  2084 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.622210  2397 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:13:55.608846  2154 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.628129  2134 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.629053 28158 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 15: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:55.629618 28082 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:13:55.630789  2134 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:55.593495  2178 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:13:55.652240 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 91972.6us
I20260709 13:13:55.658480 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.659543 28230 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:55.654109 28235 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:55.663882 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.666991 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:55.669384 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 77439.4us
I20260709 13:13:55.670110 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:55.670956 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:55.625772  2084 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.687434 28235 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:55.689571 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:55.693796  2348 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.694785  2348 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:55.698357 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.700541 28085 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:13:55.649269  2323 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.705042 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:55.706720 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:13:55.707408 28078 raft_consensus.cc:606] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:55.707762 28078 raft_consensus.cc:614] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:55.716635 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:55.717724 28758 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:13:55.719236 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:55.722328  2327 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:55.723357  2327 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:55.701529  1974 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:55.705724 28086 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:55.730871  2373 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:55.731806 28233 raft_consensus.cc:2410] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:13:55.733376  2373 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.734743 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.735543 28234 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:13:55.736023 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:55.657657  2178 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:55.736904 28232 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:55.736963  2178 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:55.734063  2373 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:55.725039  2154 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:55.738035 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:55.662317  2319 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:55.735028  2402 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.293s	user 0.000s	sys 0.003s
I20260709 13:13:55.741039  2319 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:55.741585 28229 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:13:55.742538 28085 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:55.743280  2319 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.733425  2323 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.713955  2325 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:55.763803 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:55.750335 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 80317.6us
I20260709 13:13:55.766731 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:55.753469 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.756394 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:55.767733 28232 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
W20260709 13:13:55.768987 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 65071.4us
I20260709 13:13:55.749476  2373 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:55.704443  2149 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:55.784303 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:55.788122 28049 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:55.783666 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:55.794059 28155 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:13:55.749346  2127 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:13:55.810127 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.811105 28226 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:55.783562  2084 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:13:55.786073  2355 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.781980  2273 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:55.789295  2149 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.765659  2063 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:13:55.793613  2348 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:55.787587  1656 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.768710  2075 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
W20260709 13:13:55.827492 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.766465  1982 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
W20260709 13:13:55.862912 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.799407  1974 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.874385 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:55.823220  2084 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.877715 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:55.807484  2127 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:55.815508  2373 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:55.879029 28728 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:55.885339 28760 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:55.731868  2372 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:55.793243  2377 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 13:13:55.856040  2149 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:55.854293  1950 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:13:55.879988  2385 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.924613  2127 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:55.931725  2373 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:55.936065  2393 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.367s	user 0.003s	sys 0.003s
I20260709 13:13:55.936690  2323 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:55.955304 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 296709us
W20260709 13:13:55.956805 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 77924.8us
I20260709 13:13:55.956939 28760 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
W20260709 13:13:55.940194 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 173233us
I20260709 13:13:55.998915 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:55.958647 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 174951us
W20260709 13:13:55.971839  2395 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
W20260709 13:13:55.982402  2406 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.281s	user 0.000s	sys 0.006s
W20260709 13:13:55.989526  2404 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.373s	user 0.002s	sys 0.000s
I20260709 13:13:55.922354  2084 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:55.873241  2134 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:55.856702  2355 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:55.873584  2273 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:56.008674  2399 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.515s	user 0.003s	sys 0.000s
I20260709 13:13:55.887336  1656 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:56.018107 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.018028 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 229966us
I20260709 13:13:55.936506  2394 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.233s
I20260709 13:13:55.967883  2096 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.918885  1974 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:56.210880 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.172778  2380 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:13:56.217273  2410 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:13:56.160313  2178 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:56.136943  2414 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 13:13:56.087899  2400 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.474s	user 0.001s	sys 0.001s
I20260709 13:13:56.046974 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1431) took 1169 ms. Trace:
I20260709 13:13:56.094821  2273 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:56.049983  2408 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 13:13:56.035430  2409 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
W20260709 13:13:56.135744 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.093007  2407 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.467s	user 0.003s	sys 0.000s
W20260709 13:13:56.051270  2349 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.000s
I20260709 13:13:55.993878  2385 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.025420  2134 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.072921 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1246) took 1111 ms. Trace:
W20260709 13:13:56.079357  2395 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:13:56.005354  2287 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:56.222498 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.222867 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:13:56.223565 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:56.226547 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.116870  2355 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:56.150722  2288 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:56.235761 28764 rpcz_store.cc:276] 0709 13:13:54.877416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.877574 (+   158us) service_pool.cc:224] Handling call
0709 13:13:54.877902 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:54.885550 (+  7648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.885583 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:54.945510 (+ 59927us) log.cc:844] Serialized 26 byte log entry
0709 13:13:55.218138 (+272628us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:55.218181 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.218250 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:55.743773 (+525523us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:56.046608 (+302835us) raft_consensus.cc:1685] finished
0709 13:13:56.046645 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.046898 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7857,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":51026,"spinlock_wait_cycles":85448576,"thread_start_us":244982,"threads_started":1,"wal-append.queue_time_us":456417}
W20260709 13:13:56.246094 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.255520 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.260097 28228 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:56.256454 28235 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:56.247395  2377 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:56.266036 28233 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:13:56.267289  2178 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:56.269285 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1244) took 1311 ms. Trace:
I20260709 13:13:56.229617  2084 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.229535  2096 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:56.251511 28156 rpcz_store.cc:276] 0709 13:13:54.961757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.973905 (+ 12148us) service_pool.cc:224] Handling call
0709 13:13:54.974231 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:54.980309 (+  6078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.980339 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:54.980997 (+   658us) log.cc:844] Serialized 26 byte log entry
0709 13:13:55.163081 (+182084us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b180037cc00. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:13:55.377120 (+214039us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:13:55.396692 (+ 19572us) log.cc:844] Serialized 14 byte log entry
0709 13:13:55.504574 (+107882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.505410 (+   836us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:55.883492 (+378082us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b180037cc00. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:13:55.988659 (+105167us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:56.072571 (+ 83912us) raft_consensus.cc:1685] finished
0709 13:13:56.072602 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.072809 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16235,"prepare.queue_time_us":771,"prepare.run_cpu_time_us":9447,"prepare.run_wall_time_us":26724,"raft.queue_time_us":113598,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":44237,"spinlock_wait_cycles":303971200,"thread_start_us":213162,"threads_started":1,"wal-append.queue_time_us":572224}
I20260709 13:13:56.273557 28762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 1121) took 1009 ms. Trace:
W20260709 13:13:56.243599  2352 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.003s	sys 0.017s
I20260709 13:13:56.274272 28160 rpcz_store.cc:276] 0709 13:13:54.957689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.957809 (+   120us) service_pool.cc:224] Handling call
0709 13:13:54.958100 (+   291us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:54.960433 (+  2333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.960467 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:54.966458 (+  5991us) log.cc:844] Serialized 52 byte log entry
0709 13:13:55.182334 (+215876us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:13:55.226181 (+ 43847us) log.cc:844] Serialized 14 byte log entry
0709 13:13:55.226538 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 13:13:55.226724 (+   186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.226801 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:55.971172 (+744371us) spinlock_profiling.cc:243] Waited 67.6 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:56.269034 (+297862us) raft_consensus.cc:1685] finished
0709 13:13:56.269060 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.269220 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":66928,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":342,"raft.queue_time_us":20089,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":227,"spinlock_wait_cycles":155469056,"thread_start_us":183797,"threads_started":1,"wal-append.queue_time_us":585394}
I20260709 13:13:56.275502 28762 rpcz_store.cc:276] 0709 13:13:55.264285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.264445 (+   160us) service_pool.cc:224] Handling call
0709 13:13:55.264867 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:55.271649 (+  6782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:55.271686 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:55.280076 (+  8390us) log.cc:844] Serialized 26 byte log entry
0709 13:13:55.507221 (+227145us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:55.507267 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.507340 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:56.092289 (+584949us) spinlock_profiling.cc:243] Waited 89.6 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:56.257222 (+164933us) raft_consensus.cc:1685] finished
0709 13:13:56.257243 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.273485 (+ 16242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4218,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":6671,"spinlock_wait_cycles":206162688,"thread_start_us":226176,"threads_started":1,"wal-append.queue_time_us":345116}
I20260709 13:13:56.254221  2134 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:56.266040  2145 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.798s	user 0.011s	sys 0.002s
I20260709 13:13:56.270705  2063 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:13:56.278402  2352 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.003s	sys 0.018s
W20260709 13:13:56.288702 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.293603 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3334) took 1004 ms. Trace:
I20260709 13:13:56.285529  2373 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:56.294723 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:56.294077 28231 rpcz_store.cc:276] 0709 13:13:55.289189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.293563 (+  4374us) service_pool.cc:224] Handling call
0709 13:13:55.365001 (+ 71438us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:55.405682 (+ 40681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:55.405716 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:55.412757 (+  7041us) log.cc:844] Serialized 52 byte log entry
0709 13:13:55.648993 (+236236us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:13:55.649046 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.649145 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:56.293322 (+644177us) raft_consensus.cc:1685] finished
0709 13:13:56.293348 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.293520 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40364,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"thread_start_us":235735,"threads_started":1,"wal-append.queue_time_us":443596}
I20260709 13:13:56.273567  2288 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.301725  1974 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:56.284471  1982 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:13:56.275463  2372 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:13:56.288793  2063 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:56.306955 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:56.277174  2377 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.289163  2145 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.321225  2394 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:56.321759  2394 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.322068 28077 raft_consensus.cc:2368] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21.
I20260709 13:13:56.322119  2394 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:56.324119  2145 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:56.325533  2385 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:56.315478  2075 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:56.329962  2288 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:56.332818 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.337949 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:56.341168  2341 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:56.344101  2373 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.313647  2063 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:56.345234 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 346409us
W20260709 13:13:56.348122 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 124660us
I20260709 13:13:56.348580 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:56.350627  2221 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:56.351287  2355 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:56.351806  2355 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:56.352262  2355 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:56.353979 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.345876 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:56.339942  2394 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:56.346614  2373 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.363734  2373 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:56.356009 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.365242 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:56.366130 28756 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:13:56.366263 28083 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:13:56.367658  2096 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:56.358453  2167 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:13:56.368280  2167 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:56.369494  2412 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.339s	user 0.005s	sys 0.005s
I20260709 13:13:56.369218  2167 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.370288  2167 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:56.371012 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.353624  2319 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:56.371812  2319 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:56.372219  2319 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:56.359611 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.374221 28086 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:13:56.374357  2325 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:56.379448 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:13:56.381589  2394 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:56.359416 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:56.384120 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:56.381613 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:13:56.388657 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44818.4us
I20260709 13:13:56.390991 28152 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:13:56.389808 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:56.392830 28756 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:56.393533 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.393793 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.394150 28152 raft_consensus.cc:2376] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15. Re-sending same reply.
I20260709 13:13:56.394340 28159 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:13:56.395064 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.395393 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:56.396485 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:56.397148 28159 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:56.390631 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:56.399044 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.399310 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:56.399760 28160 raft_consensus.cc:2376] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20. Re-sending same reply.
I20260709 13:13:56.400074 28153 raft_consensus.cc:2368] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19.
I20260709 13:13:56.398787 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:56.400960 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.401204 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:56.401348 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:56.401683 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:56.402055 28154 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:56.402359 28160 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:13:56.396134 28763 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:13:56.404044 28755 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: []
I20260709 13:13:56.404074 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:56.405055 28152 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:56.406546 28156 raft_consensus.cc:2368] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21.
I20260709 13:13:56.409257 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.410038 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.414606 28157 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:13:56.415902 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.417363 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.418304 28155 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:56.421044 28157 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:56.395399  2394 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:56.394392  2325 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:56.428162 28084 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:13:56.427836 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.387526  1656 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:56.435292 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.454432 28083 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 19: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:56.457525 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.405650  2349 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:56.440282 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.447585 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:13:56.469900 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55247us
I20260709 13:13:56.414430  2034 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:56.424114  2315 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.458340 28084 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:13:56.468921 28156 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:13:56.471493 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:56.423605  2154 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:13:56.438454 28159 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:56.490145 28160 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:56.496837 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.506603 28161 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:56.395087  1845 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:13:56.497041 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.512177 28157 raft_consensus.cc:2410] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:13:56.493129 28756 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:13:56.492026  2325 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:56.453439  2221 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:56.514505 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:56.518123 28228 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:13:56.492424  2034 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.467782  2349 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:56.521534  2427 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.242s	user 0.001s	sys 0.002s
I20260709 13:13:56.525923  2034 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:56.525779  2221 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.428541  2273 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:13:56.512784  2394 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:56.520282 28728 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:56.491070  2422 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:56.460081  2385 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 13:13:56.514998 28152 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
W20260709 13:13:56.533677 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 120279us
I20260709 13:13:56.552999  2394 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:56.554402  2394 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:56.531625  2349 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:56.558347  2221 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:56.533332  2127 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.535s	user 0.009s	sys 0.004s
I20260709 13:13:56.576697 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.508056  2315 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.591500 28228 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:13:56.522118  2075 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:13:56.600672 28154 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:56.552032  2063 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:56.609458 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:56.621997 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.615226 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.623819 28083 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:56.622299 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:13:56.604907  2393 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:13:56.627719 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 118110us
I20260709 13:13:56.579447  2377 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:56.630846 28156 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:13:56.633708 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.566053  2422 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.636559 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:56.637955 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 117795us
I20260709 13:13:56.622628  2005 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:13:56.651347  2005 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.651876  2005 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:13:56.579216  2127 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:56.655015 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:13:56.643779 28127 leader_election.cc:336] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.517s (SENT)
I20260709 13:13:56.657160  2182 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:56.657538 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:56.657724  2182 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:56.613600  2325 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.658613  2182 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:56.648101  2178 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:56.650651  2422 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:56.663787 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:56.666198 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.674667 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:56.636843  2429 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:13:56.678189 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.634745  2341 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
W20260709 13:13:56.705955 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 62202.9us
W20260709 13:13:56.695192 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 50583.7us
I20260709 13:13:56.696821 28226 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:13:56.707597 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:56.677135 28232 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:56.711804 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:56.717190 28226 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:13:56.672685 28078 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:56.702458 28229 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:56.656308  2127 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:56.697611 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.720822 28228 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:13:56.721625 28728 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:56.725509  2429 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:56.725708  2443 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:56.726317  2443 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:56.726765  2443 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:56.726866  2377 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:56.721920 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:56.692305  2178 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:13:56.741004 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 108101us
W20260709 13:13:56.744889 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.767386 28127 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:56.703197  2373 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:56.703569  2422 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:56.757076 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.774883 28154 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:13:56.778373 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.779004 28161 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:56.779031 28152 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:56.775893 28228 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:56.784125 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:56.784899 28160 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:13:56.703944  2428 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
W20260709 13:13:56.791357 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.787007 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:56.791566 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:56.792258 28152 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:56.789260 28084 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:13:56.789054 28228 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:13:56.772253 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:56.796067 28154 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:13:56.797237 28154 pending_rounds.cc:85] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 14
I20260709 13:13:56.797572 28154 pending_rounds.cc:107] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:13:56.797842 28154 raft_consensus.cc:2889] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:56.801409 28160 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:13:56.802052 28161 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: [17.10-18.11]
I20260709 13:13:56.803360 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:56.804174 28159 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:13:56.806306 28084 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:13:56.813388 28152 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:56.773684  2437 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:13:56.816022 28228 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:13:56.823323 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 93344.6us
I20260709 13:13:56.826228 28159 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:56.828516 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:56.775293  2341 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:13:56.781261  2422 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:56.845925 28084 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
W20260709 13:13:56.851994  2438 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:13:56.855628  2034 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:56.858143 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:56.862402 28231 pending_rounds.cc:85] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
I20260709 13:13:56.870750 28231 pending_rounds.cc:107] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:13:56.871107 28231 raft_consensus.cc:2889] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:56.857478 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.726543  2429 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:56.878249 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 106073us
I20260709 13:13:56.880929 28078 raft_consensus.cc:2410] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:13:56.881413 28154 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:13:56.883983 28127 leader_election.cc:336] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.668s (SENT)
I20260709 13:13:56.879153 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:56.890580 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:56.822160  2325 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.885821 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:56.898756 28083 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:56.868804  2315 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:56.902177  2355 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.543s	user 0.006s	sys 0.006s
I20260709 13:13:56.869549  2437 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:56.905459 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:56.947197 28083 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:13:56.895100  2350 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:13:56.954340 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48943.1us
I20260709 13:13:56.929693  2310 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:56.974874 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.976166 28229 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:56.990046 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.900756  2323 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:13:56.988695 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.997988 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:56.948144  2456 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:13:56.970436  2341 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.003747 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.003517 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.009038 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.012293 28229 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:13:56.988474  2428 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.015053 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:56.987195  2310 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.971859  2350 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:56.953858  2355 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:56.900956  2178 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:57.038291  2178 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:56.950531  2325 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.017908 28227 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:57.022670 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:13:57.025408  2447 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:13:57.029066  1845 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:57.002478  2341 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:57.003046  2145 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:13:57.031225  2349 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:57.067327  2145 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:57.067787  2145 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:57.069089 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:57.068804  2145 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:57.070518  2145 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:57.023375  2182 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:13:57.072045  2182 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:13:57.072427  2182 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:57.073107 28153 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
W20260709 13:13:57.036340  2450 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:13:57.073402  2182 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:57.037256  2442 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
W20260709 13:13:57.032939  2134 consensus_meta.cc:287] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.745s	user 0.010s	sys 0.001s
I20260709 13:13:57.051036  2323 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:57.021049  2310 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:57.075139  2134 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:57.075740  2323 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.033771  2350 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.077608 28081 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:13:57.078219  2323 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:57.057518 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:57.079721 28158 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:57.079986 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.068912  2421 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:13:57.081053 28082 raft_consensus.cc:2368] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17.
I20260709 13:13:57.064296  1845 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:57.079054 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.066301  2341 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:57.070465 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:57.099879 28154 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:57.083740 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.080626 28227 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
W20260709 13:13:57.068480  2319 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.696s	user 0.007s	sys 0.003s
I20260709 13:13:57.074868  2182 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:57.088634 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:57.105880 28235 raft_consensus.cc:2393] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:57.101382 28228 raft_consensus.cc:2368] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18.
W20260709 13:13:57.103775 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 277804us
W20260709 13:13:57.061462  2149 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.094s	user 0.012s	sys 0.002s
I20260709 13:13:57.107107  2149 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.072516  2355 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.041554  2422 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.078244  2325 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:57.123172 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.115459 28083 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:57.108476 28050 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:57.120417 28158 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:13:57.121575 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.122063 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.068495  2349 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.125721 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.127012 28235 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:57.128310 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:13:57.129385 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 88972.7us
W20260709 13:13:57.130636 28127 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.828s (SENT)
I20260709 13:13:57.024204  2428 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.134518 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.136518 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:57.109128  2319 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:57.139092  2319 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:57.140677 28127 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.839s (SENT)
I20260709 13:13:57.141193 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:57.142102  2349 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:57.143117 28050 leader_election.cc:400] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19.
I20260709 13:13:57.144397  2034 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:57.136603  2434 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:13:57.144685 28050 leader_election.cc:400] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21.
I20260709 13:13:57.147933 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:57.150038  2149 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:57.106020  1845 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.152473  1845 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:57.153941  2327 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.154882 28080 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:57.155686  2221 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:13:57.155972 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:57.161695 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:57.162406 28081 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:57.137434 28085 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:13:57.214643 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:57.146131  2134 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.219369 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:57.211443  2458 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.111s	user 0.000s	sys 0.003s
W20260709 13:13:57.220561 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46961.3us
I20260709 13:13:57.220566 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:57.221271 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 85316.9us
I20260709 13:13:57.222877 28085 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:57.226783 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:57.130733  2422 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:13:57.230331 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:57.231081 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:57.232146 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:57.232087 28232 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:57.234159 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:57.236323 28234 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 21: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:57.231318 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.145951  2352 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:13:57.219722  2377 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:57.251340 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:57.254453 28228 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:13:57.237432  2443 consensus_meta.cc:287] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.510s	user 0.013s	sys 0.000s
I20260709 13:13:57.257750 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:57.269331 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.180164  2005 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.260231 28159 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 13:13:57.295585 28127 leader_election.cc:336] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.535s (SENT)
W20260709 13:13:57.309237 28127 leader_election.cc:336] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.992s (SENT)
I20260709 13:13:57.309971 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:57.362548 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:57.375051 28757 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:57.396154 28232 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:57.311196  2434 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:13:57.402119 28761 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 22. Current term is 23. Ops: []
I20260709 13:13:57.406759 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.407956 28228 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 19: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:13:57.416515 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 107327us
I20260709 13:13:57.417994 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:57.418913 28161 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:57.420298 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:57.381393  2327 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:57.454047 28160 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:13:57.457870 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.451264 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:57.460495 28159 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:13:57.461759 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.458412 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:57.453143  2467 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:57.419915  2434 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:57.364463  2443 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.506657 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1393) took 1079 ms. Trace:
I20260709 13:13:57.500690  2472 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:57.456161  2005 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.509263 28086 rpcz_store.cc:276] 0709 13:13:56.427213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.431027 (+  3814us) service_pool.cc:224] Handling call
0709 13:13:56.467698 (+ 36671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.662250 (+194552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:56.662283 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:56.662326 (+    43us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:13:56.662357 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.506410 (+844053us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400297cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:57.506456 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.506596 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446942080}
I20260709 13:13:57.500846  1974 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:13:57.541033 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 425922us
I20260709 13:13:57.539225  2472 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.560941  2434 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:57.562090  2443 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.562701  2428 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.564210 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:57.564968 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:57.625823 28127 leader_election.cc:336] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.617s (SENT)
W20260709 13:13:57.540309  2471 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:13:57.743345 28229 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:13:57.745007 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1258) took 1342 ms. Trace:
I20260709 13:13:57.565969 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.725047  2341 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.021s
W20260709 13:13:57.567384  2464 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:13:57.745450 28764 rpcz_store.cc:276] 0709 13:13:56.402233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.402411 (+   178us) service_pool.cc:224] Handling call
0709 13:13:56.402947 (+   536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:56.403328 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:56.403355 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:57.037770 (+634415us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b4c0004c040. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cc943a20 00007f59cc954a29 00007f59cc91a644 00007f59cc941f2d 00007f59cc941b93 00007f59ce67218b 00007f59ce672cb5 00007f59ce674004 00007f59ce673729 00007f59ce678d96 00007f59ce66dadb
0709 13:13:57.038645 (+   875us) log.cc:844] Serialized 564 byte log entry
0709 13:13:57.102584 (+ 63939us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:13:57.102638 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.102734 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:57.744655 (+641921us) raft_consensus.cc:1685] finished
0709 13:13:57.744685 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.744948 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59582720,"thread_start_us":63550,"threads_started":1,"wal-append.queue_time_us":196878}
I20260709 13:13:57.566617  2149 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:57.574836 28085 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
W20260709 13:13:57.503016  2439 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:13:57.507316  2467 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:57.506713  2473 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:13:57.750883  2467 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:57.601706 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:57.580462 28728 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:57.753399 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:57.687880 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.566419 28759 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:57.743420 28127 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:57.593127 28161 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:13:57.575917 28232 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:13:57.759948 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:57.762516 28161 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:57.765129 28161 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:13:57.769295 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:57.789047 28161 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:13:57.565564  2178 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.003s
I20260709 13:13:57.799373 28084 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
W20260709 13:13:57.594974  2475 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:13:57.813055 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:57.814429 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:57.815677 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:57.816845  2428 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:13:57.817323  2428 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:57.817703  2428 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:57.819329 28084 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:57.504462  2221 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.791368 28161 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:57.812463 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:57.541254  2325 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:57.804556 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:57.821578  2325 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.822050 28082 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:13:57.824021  2325 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:57.809389  2467 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:57.825098  2467 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:57.809134 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 228828us
I20260709 13:13:57.827163  2467 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:57.809850  2005 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:57.827909  2005 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.810262  2429 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:57.829056  2429 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.830117  2005 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.539690  1974 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:57.831053  2429 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:57.831207  1974 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:57.824600  2325 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:57.565218  2470 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.363s
I20260709 13:13:57.833149  2325 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:57.833711  2325 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.513437  2469 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:13:57.820719  2221 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:57.741441  2327 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:57.840852 28081 consensus_meta.cc:287] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.666s	user 0.011s	sys 0.002s
I20260709 13:13:57.841159 28230 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:13:57.842007 28154 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:13:57.759389  2443 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:57.863205  2325 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:57.864602  2472 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:57.845688  2469 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.865386  2469 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:57.867921 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:57.871662 28727 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.019s	sys 0.062s
W20260709 13:13:57.872123 28727 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.019s	sys 0.062s
I20260709 13:13:57.856864 28081 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:13:57.840562  2429 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.893049 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:57.894944 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:57.895905 28763 raft_consensus.cc:2410] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:13:57.898283  2385 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:57.898974  2385 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.901211 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:57.902798  2385 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.902974 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.903882 28232 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:57.905273 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:57.868835  2327 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.874346  2443 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.912292  2443 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.923590  2084 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:57.924767 28078 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2764) took 1204 ms (client timeout 1596 ms). Trace:
W20260709 13:13:57.925370 28078 rpcz_store.cc:269] 0709 13:13:56.720326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.721667 (+  1341us) service_pool.cc:224] Handling call
0709 13:13:57.924486 (+1202819us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54003aa7d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:57.924674 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270131072}
I20260709 13:13:57.925437  2429 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:57.935654 28082 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:57.933411  2429 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:57.950452 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:13:57.966404 28077 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 1387) took 2391 ms (client timeout 1550 ms). Trace:
W20260709 13:13:57.967007 28077 rpcz_store.cc:269] 0709 13:13:55.540203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.540674 (+   471us) service_pool.cc:224] Handling call
0709 13:13:57.931774 (+2391100us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b540017c854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:57.931906 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790141312}
I20260709 13:13:57.967519 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:57.968824 28758 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:57.967869 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:57.974053 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:57.975243 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:57.995455 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.993327 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:57.982749  2325 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:58.019335  2325 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.022035  2325 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:58.033458  2149 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:58.034114  2149 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:58.041724 28758 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:13:58.045468 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2772) took 1084 ms. Trace:
I20260709 13:13:58.045658  2149 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:58.045933 28083 rpcz_store.cc:276] 0709 13:13:56.961384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.961592 (+   208us) service_pool.cc:224] Handling call
0709 13:13:57.115262 (+153670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.123987 (+  8725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.045197 (+921210us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b540017e150. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:58.045403 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352455040}
I20260709 13:13:58.047413  2428 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.048885 28762 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
W20260709 13:13:58.050549  2310 consensus_meta.cc:287] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.975s	user 0.012s	sys 0.000s
I20260709 13:13:58.050994  2310 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:58.051680  2149 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:58.052213 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:13:58.001099 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 375095us
I20260709 13:13:58.017372 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1391) took 1612 ms. Trace:
I20260709 13:13:58.054488 28079 rpcz_store.cc:276] 0709 13:13:56.405219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.424385 (+ 19166us) service_pool.cc:224] Handling call
0709 13:13:56.459042 (+ 34657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.559430 (+100388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:56.559461 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:56.559499 (+    38us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:13:56.559526 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.017099 (+1457573us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b5400209254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:58.017149 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.017303 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230248192}
W20260709 13:13:58.055495  2182 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.946s	user 0.012s	sys 0.000s
W20260709 13:13:58.053942  2145 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.967s	user 0.006s	sys 0.006s
W20260709 13:13:58.056622 28127 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.621s (SENT)
I20260709 13:13:58.057034 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.057739  2325 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:58.058929 28127 leader_election.cc:336] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.690s (SENT)
I20260709 13:13:58.059443 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.060501  2456 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:58.064495  2221 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:58.065030 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:58.066716  2221 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:58.067456  1816 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.068075  1816 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.066727  2434 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:58.069361  2428 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:58.069327  2434 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.071862  2434 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:58.068477 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:58.072952 28079 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 21: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:13:58.074724  2310 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:58.077198  2469 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:58.078835  1816 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:58.079403  2452 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.228s	user 0.003s	sys 0.007s
I20260709 13:13:58.080602 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1257) took 1684 ms. Trace:
I20260709 13:13:58.083506 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:58.083611 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:58.084227 28078 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:13:58.086470 28153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3123) took 1682 ms (client timeout 1736 ms). Trace:
I20260709 13:13:58.086759 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:58.086915 28153 rpcz_store.cc:269] 0709 13:13:56.403929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.406137 (+  2208us) service_pool.cc:224] Handling call
0709 13:13:58.086312 (+1680175us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b54001afed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:58.086412 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1431067392}
I20260709 13:13:58.087582 28081 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 12.
I20260709 13:13:58.091634 28080 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:13:58.093132 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:58.095458 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 105701us
I20260709 13:13:58.080977 28760 rpcz_store.cc:276] 0709 13:13:56.396632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.396822 (+   190us) service_pool.cc:224] Handling call
0709 13:13:56.405627 (+  8805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:56.415810 (+ 10183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:56.415841 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:56.427067 (+ 11226us) log.cc:844] Serialized 26 byte log entry
0709 13:13:56.789640 (+362573us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:13:56.789693 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.790101 (+   408us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:58.078823 (+1288722us) spinlock_profiling.cc:243] Waited 8.28 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:13:58.080310 (+  1487us) raft_consensus.cc:1685] finished
0709 13:13:58.080334 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.080540 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33557,"raft.queue_time_us":12865,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":11718,"spinlock_wait_cycles":19035008,"thread_start_us":289327,"threads_started":1,"wal-append.queue_time_us":376892}
I20260709 13:13:58.096691 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.097697 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.099934 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:58.101087 28762 raft_consensus.cc:2410] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:13:58.101686 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:58.103504 28078 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
W20260709 13:13:58.060518 28127 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.695s (SENT)
I20260709 13:13:58.120042 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:58.121243 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 60742.4us
I20260709 13:13:58.121058  2432 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:58.067397  2221 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:13:58.125880 28127 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.714s (SENT)
I20260709 13:13:58.140977 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.100512  2096 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:58.135485  2178 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:13:58.135673  2436 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:13:58.137322 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.151683  2288 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:58.152208  2288 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:58.143869 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:58.155386 28078 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:13:58.105209  2034 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:58.131326 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:58.109323  2428 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.137689  2339 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:13:58.146761 28127 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.183s (SENT)
I20260709 13:13:58.158689 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.106513  2469 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:13:58.149053  2377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.016s	sys 0.024s
I20260709 13:13:58.162752 28078 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:13:58.148654  2096 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:58.149647  2456 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:58.152575  2288 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:58.157348  2034 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.179778 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:58.180490 28232 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:13:58.181325 28127 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.827s (SENT)
I20260709 13:13:58.181800 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.174610 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:58.170717  2377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.016s	sys 0.024s
I20260709 13:13:58.185961 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:58.160871  2339 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:58.187618 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.148890  2221 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.192977 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.196756 28159 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:58.199599 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.199960 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.180013  2373 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:58.183611  2034 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:58.184675  2325 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.212812 28232 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:58.198340 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.218658 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.208506 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:58.235824 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.195680  2432 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:58.167433  2428 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.216492 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:58.264227  2221 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:58.263782  2096 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.273226 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:58.274487 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.317065 28232 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:58.319267 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.321568 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.321955 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:58.322482  2415 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:58.323367 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:58.323787  2415 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:58.325294  2415 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:13:58.276074  1982 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:58.275403  2178 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.326752  1982 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:58.327032  2178 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.327824  1982 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.328974  1982 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:58.329550  2178 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:58.329789 28232 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:58.330240  2178 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:58.330931  2178 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.333384 28760 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:58.334429  2415 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:58.335060  2415 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:58.220482  2325 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.335137  2327 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:58.337538  2325 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:58.338950  2506 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.162s	user 0.002s	sys 0.000s
W20260709 13:13:58.310221 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 93801.1us
I20260709 13:13:58.326228 28756 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:58.289143  2339 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.335381  2063 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:58.284547 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1261) took 1826 ms. Trace:
I20260709 13:13:58.355284 28156 rpcz_store.cc:276] 0709 13:13:56.458599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.494944 (+ 36345us) service_pool.cc:224] Handling call
0709 13:13:56.495335 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.631347 (+136012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.284248 (+1652901us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400376ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:58.284479 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307976576}
I20260709 13:13:58.305548 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:58.343470 28728 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.359313 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.365800 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:58.354527 28235 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:13:58.368558  2178 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:58.371969 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.373538  2034 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.379422  2477 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:13:58.381825 28156 raft_consensus.cc:2368] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19.
I20260709 13:13:58.354403  2420 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:58.385216 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.354645  2075 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:58.365718  2429 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:13:58.393368 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.385411  2273 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:13:58.394320 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:58.390518  2477 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:13:58.394688  2273 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:58.403401 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:58.392696  2420 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:58.398221  2429 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:58.392472  2075 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.420672  2429 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.422061  2075 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:58.422793  2428 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:58.423590  2420 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:58.418272  2325 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:13:58.407203  2273 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:13:58.435992 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:58.440768 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 50408.6us
I20260709 13:13:58.441718 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.440495 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:58.449352  2096 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:13:58.460561  2096 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:58.461130  2096 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:13:58.443472  2127 raft_consensus.cc:2764] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:13:58.461645 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:58.461849 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:58.459834 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:13:58.448486 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 93150.6us
I20260709 13:13:58.449730 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2779) took 1399 ms. Trace:
I20260709 13:13:58.462548 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:58.458782 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2555) took 1489 ms. Trace:
I20260709 13:13:58.465939 28230 rpcz_store.cc:276] 0709 13:13:56.969364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.970102 (+   738us) service_pool.cc:224] Handling call
0709 13:13:56.987781 (+ 17679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:57.856252 (+868471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.458439 (+602187us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b5400306e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:13:58.458679 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866899712}
I20260709 13:13:58.467334 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:58.469141 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:58.465795 28080 rpcz_store.cc:276] 0709 13:13:57.050626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.164516 (+113890us) service_pool.cc:224] Handling call
0709 13:13:58.449567 (+1285051us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b54001aead4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:58.449674 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1933768064}
I20260709 13:13:58.465332 28229 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:13:58.472448 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.473789 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.476466 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:58.478444 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:58.479144 28230 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:58.472528 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:58.483296 28760 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:13:58.483294 28231 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:13:58.484571 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:13:58.454421  2505 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:13:58.485874 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.486230 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:58.486891 28760 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:58.490597 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:58.491356 28762 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:13:58.446946  2325 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:13:58.490880 28730 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.500716 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:58.490811 28198 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.501533 28079 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:58.496369 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:58.498080 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:58.504946 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:58.502142 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.500245 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:58.445009  2415 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.498930 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:58.483096  2327 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:13:58.489643  2034 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:58.454948  2288 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.673650 28756 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:13:58.510545  2063 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:13:58.508252  2492 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:13:58.704792 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:58.705458 28231 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:58.512641  2512 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:13:58.566736  2509 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:13:58.512446  2274 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
W20260709 13:13:58.644181 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 137740us
I20260709 13:13:58.708271 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:58.490696  2323 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:58.510046  2460 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:58.489552  2373 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:13:58.482827  2134 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:58.496225  2486 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:58.471570  1982 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:13:58.671816 28760 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:13:58.677275 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.688105 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:13:58.702476 28127 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.961s (SENT)
I20260709 13:13:58.507858  2456 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:58.713949 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.714502 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:58.675556  2288 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:58.721874 28160 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:58.675912  2415 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:13:58.718916 28156 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:13:58.703363  2063 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:58.737314 28728 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:58.738592  1816 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:58.752044 28728 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:58.711608  2460 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.711282  2512 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:58.711728  2075 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
W20260709 13:13:58.737937 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49975.6us
I20260709 13:13:58.857301 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.740078  2443 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:13:58.737234 28127 leader_election.cc:336] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.017s (SENT)
I20260709 13:13:58.859191 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:13:58.836882 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.792481 28231 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
W20260709 13:13:58.784068 28086 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2581) took 1639 ms (client timeout 2167 ms). Trace:
W20260709 13:13:58.879601 28086 rpcz_store.cc:269] 0709 13:13:57.145138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.720495 (+575357us) service_pool.cc:224] Handling call
0709 13:13:58.783870 (+1063375us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54000e29d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:58.784010 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504619904}
I20260709 13:13:58.748980  2325 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:13:58.779903  2508 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:13:58.759308  2415 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:13:58.873772  2513 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.091s	user 0.000s	sys 0.002s
I20260709 13:13:58.855799  2460 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:58.876257 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 139050us
I20260709 13:13:58.843672  2323 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:58.759622  2510 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:13:58.879016 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 170823us
I20260709 13:13:58.792279  2149 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:13:58.758419  2063 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:13:58.757732  2288 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:58.894735 28127 leader_election.cc:336] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.064s (SENT)
I20260709 13:13:58.880038  2420 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:58.775111 28728 leader_election.cc:400] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19.
I20260709 13:13:58.879931  2492 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:58.899698 28728 leader_election.cc:403] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:13:58.879908  2512 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:58.777762  1816 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:58.867640  2096 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.897275 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:13:58.898155 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:58.777235  2084 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.117s
W20260709 13:13:58.900214 28199 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.015s	user 0.007s	sys 0.013s
I20260709 13:13:58.896864  2510 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:58.912111 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.900882 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:58.902998  2420 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:58.908717 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.909960 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:58.910663 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:58.911748 28127 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.229s (SENT) [suppressed 42 similar messages]
I20260709 13:13:58.912214 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:58.916088 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 154469us
I20260709 13:13:58.915757 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:58.917105 28199 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.007s	sys 0.015s
W20260709 13:13:58.902524  2517 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:13:58.913496 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:13:58.915416 28152 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:13:58.917269 28157 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:13:58.948392 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47651.4us
I20260709 13:13:58.950415 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.917553 28728 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:13:58.917472 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 1.11492e+06us
I20260709 13:13:58.918963  2420 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:58.919471 28158 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:58.921157  1816 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:13:58.925004  2005 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19.
I20260709 13:13:58.952697 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.926693  2466 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:13:58.915494  2034 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:13:58.914528  2523 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:13:58.931834 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.955808 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:58.965721 28157 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:13:58.977767 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.979277 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:58.972388 28152 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:58.982204 28161 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:13:58.986944 28050 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:13:58.987135 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:58.900630  2323 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:58.991665 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 79992.2us
I20260709 13:13:58.993469 28158 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:13:58.954928  2145 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:13:58.974920  2415 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:13:58.981985  2484 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:13:58.962051  2466 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:58.986434  2403 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:58.989427  2274 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:59.009832 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:58.945578  2273 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:58.988070  2325 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.989324  2149 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:13:58.986091  2134 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:13:59.013652  2145 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:13:59.005455  2415 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:59.017650 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 85913.6us
W20260709 13:13:59.003278  2436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.019s	sys 0.017s
I20260709 13:13:58.989177  2352 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:13:59.010634  2484 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:59.021986 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:59.023105  2134 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.009672  2492 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.013062 28155 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 17
W20260709 13:13:59.011206 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 59541.6us
I20260709 13:13:59.008921  2523 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:59.076269  2523 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:59.077365  2523 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.023690  2145 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:59.021399  2403 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:59.032881  2005 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.079608  2145 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:59.079908  2005 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.080235  2403 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:59.034875  2096 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:59.042346  2323 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:59.010007  2510 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.013458  2429 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:13:59.050144  2273 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:59.063424 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56737.2us
I20260709 13:13:59.091130 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:59.012389  2084 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:59.094478 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:59.094811 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:13:59.033082  2436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.020s	sys 0.018s
I20260709 13:13:59.097009 28157 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:59.033416  2415 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:13:59.099742 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.064119  2484 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:59.103662 28761 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 23. Current term is 24. Ops: []
W20260709 13:13:59.100766 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 113874us
I20260709 13:13:59.020431  2274 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.073706  2134 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:13:59.105901 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:13:59.016134  2524 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:13:59.024483  2325 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:59.132272 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.147538 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 52148.7us
I20260709 13:13:59.134411 28155 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:13:59.052626  2460 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.176465 28229 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:13:59.182253 28230 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:13:59.118711  2510 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:13:59.213212 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 82992.4us
I20260709 13:13:59.118021  2429 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:59.219645 28234 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:13:59.216065 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 148092us
W20260709 13:13:59.226871 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 70004.8us
I20260709 13:13:59.088871  2492 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:13:59.231999 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 112363us
I20260709 13:13:59.140618  2075 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.092s
I20260709 13:13:59.133846  1982 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:13:59.172435  2532 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:13:59.169480  2436 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:13:59.239778 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:13:59.134032  2511 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:13:59.238351 28728 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:59.253955  2274 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:59.099782  2466 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.257630  2466 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:13:59.262589  2352 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:13:59.264397  2096 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:59.273766 28050 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:13:59.265062  2096 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:59.282549 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48967.3us
I20260709 13:13:59.288398 28761 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:13:59.293814 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:59.301505 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:59.304267 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:59.307770  2034 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:13:59.264415  2532 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:59.309516  2532 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:59.301260  2524 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:13:59.310740  2524 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:59.311740  2524 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:59.313079  2325 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:59.315441 28762 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:13:59.316429 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:59.318964 28157 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:13:59.320261 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:59.321892 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:59.327308 28762 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:13:59.338420 28085 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:59.345072 28756 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 13: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:13:59.303210  2273 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:13:59.354456  2063 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.355151  2544 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:59.355702  2544 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.357842  2544 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:59.358417  2539 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.358300  2145 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.358974  2539 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.303025  2154 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 13:13:59.361858  2154 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:13:59.362211  2154 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:13:59.365262 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:59.360550  2145 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:59.378139 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.349879  2273 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.313766  2403 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
W20260709 13:13:59.410413 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 182981us
I20260709 13:13:59.319430  2542 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:13:59.372376  2063 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:59.373745  2084 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:59.347551  2486 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:13:59.370553  2540 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:13:59.337742  2429 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.438057 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3389) took 1353 ms. Trace:
I20260709 13:13:59.356576  2512 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.441366  2460 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:59.429149  2273 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:59.368181  2352 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:13:59.446676  2484 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.452323 28232 rpcz_store.cc:276] 0709 13:13:58.102899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.330703 (+227804us) service_pool.cc:224] Handling call
0709 13:13:58.331081 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.331501 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.331528 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.014399 (+682871us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b180003c820. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a
0709 13:13:59.041299 (+ 26900us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b2c0004c898. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:13:59.041820 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.180315 (+138495us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:59.180368 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.180479 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.437824 (+257345us) raft_consensus.cc:1685] finished
0709 13:13:59.437847 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.437993 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10617,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":14883,"spinlock_wait_cycles":57828992,"thread_start_us":77179,"threads_started":1,"wal-append.queue_time_us":230560}
I20260709 13:13:59.438274  2154 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.433537  2511 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:13:59.458173 28156 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:13:59.428851  2492 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:59.470410  2511 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:59.471076  1656 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:59.471480  2511 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:59.472553  2096 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:59.472785  2511 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:13:59.473325  2154 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:13:59.479333 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.489837 28233 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:13:59.493108  2415 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.497164  2512 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:59.498121 28229 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:59.499588  2394 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:13:59.500162  2394 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:13:59.503633 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:13:59.507097 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:59.508386 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:59.509029 28230 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:13:59.509616 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:59.510231 28764 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:13:59.517530 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:59.507797 28759 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
W20260709 13:13:59.467010  2535 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:13:59.523885 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:59.524760 28756 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:13:59.461622  2539 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:59.530583 28758 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: []
I20260709 13:13:59.532578 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:13:59.507630  2310 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:13:59.562463 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:59.572320 28230 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:59.507119  2005 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:59.452142  2084 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.593340 28232 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:59.599869 28764 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:13:59.623023  2339 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:59.624477 28756 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:13:59.632330 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3377) took 2400 ms. Trace:
I20260709 13:13:59.590844  2310 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:13:59.636907 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:59.632983 28227 rpcz_store.cc:276] 0709 13:13:57.229310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.229490 (+   180us) service_pool.cc:224] Handling call
0709 13:13:57.229967 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:57.243558 (+ 13591us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:57.243611 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.014624 (+1771013us) spinlock_profiling.cc:243] Waited 8.15 ms on lock 0x7b180028b040. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a
0709 13:13:59.053215 (+ 38591us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b2c0004c898. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:13:59.053731 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.128446 (+ 74715us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:59.128502 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.128604 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.629025 (+500421us) raft_consensus.cc:1685] finished
0709 13:13:59.629050 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.629168 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2607,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10830,"spinlock_wait_cycles":25845376,"thread_start_us":74261,"threads_started":1,"wal-append.queue_time_us":130501}
I20260709 13:13:59.604096  2096 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:59.631000 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 59268.1us
I20260709 13:13:59.591041  2134 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:59.646785 28160 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:13:59.671362 28085 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:13:59.672830 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:59.581729  2394 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.673516 28077 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:13:59.678900 28757 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:13:59.679702 28755 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:13:59.679874 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:13:59.684705 28049 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:13:59.686141 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:59.692303 28083 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
I20260709 13:13:59.694813 28049 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:59.724180 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:59.727521 28049 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:59.733014 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:13:59.734521 28763 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:13:59.653528  2096 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:59.744135 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:13:59.689595  2221 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:13:59.690353  2532 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:13:59.730265  2325 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.826033 28160 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:59.745127  2394 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:13:59.668650  2310 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:13:59.832101 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.834388 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:13:59.835073 28160 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:59.835942 28085 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:13:59.833683 28159 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.016s	sys 0.033s
W20260709 13:13:59.842976 28159 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.017s	sys 0.034s
I20260709 13:13:59.857764 28160 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:13:59.863627 28760 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:13:59.868175 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:13:59.823099  2511 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:13:59.729897  2486 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:13:59.869658 28080 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:13:59.871160 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:13:59.833741  2523 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.020s	sys 0.030s
W20260709 13:13:59.874025 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 340085us
I20260709 13:13:59.874708 28160 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:13:59.831143  2484 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:59.886384 28230 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:13:59.888351 28730 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:59.821882  2325 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:59.864918  2492 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:59.875998  2523 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.020s	sys 0.031s
I20260709 13:13:59.823500  2221 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.896212 28160 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:13:59.899922 28080 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
W20260709 13:13:59.902077 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.823726  2532 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.881148  2403 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:13:59.893781  2429 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:13:59.905030  2323 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.910341  1656 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:13:59.932260 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46440.2us
I20260709 13:13:59.915500  2432 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:13:59.921500 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.913920  2084 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:13:59.913224 28199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.016s	sys 0.016s
W20260709 13:13:59.901106 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 173665us
W20260709 13:13:59.985246 28199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.017s	sys 0.017s
W20260709 13:13:59.985749 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 747655us
I20260709 13:13:59.986306 28049 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:13:59.927984  2323 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:59.901959 28158 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:13:59.928856  2403 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:59.882966  2511 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:59.944094  2432 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:13:59.947283  2532 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:13:59.944125  2557 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 13:13:59.957302  2556 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 13:13:59.957178  2562 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:13:59.958341  2550 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:14:00.064179  2273 consensus_meta.cc:287] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.002s
W20260709 13:14:00.064819 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 78622.8us
I20260709 13:14:00.064955  2273 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.960662  2063 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:00.066252 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:14:00.036628  2527 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.927s	user 0.004s	sys 0.000s
I20260709 13:14:00.058236  2542 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.067977  2542 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.069375 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1409) took 1137 ms. Trace:
I20260709 13:14:00.070106  2542 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:00.070737  2542 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:00.071106  2578 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:14:00.071717 27983 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 278) took 43862 ms (client timeout 15000 ms). Trace:
W20260709 13:13:59.965135  2548 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
W20260709 13:13:59.966101  2555 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:13:59.928340  2429 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:13:59.969415  2325 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:59.975118  2221 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:13:59.979909  2145 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:00.087221 28077 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 13.
I20260709 13:13:59.981808  2096 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:13:59.981702  2574 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:13:59.986577 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:00.092036 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 105573us
I20260709 13:13:59.989830 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:13:59.881213  2486 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:13:59.994529 28158 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 22. Current term is 23. Ops: []
I20260709 13:13:59.998438 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3150) took 1088 ms. Trace:
I20260709 13:14:00.007330  2341 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:00.015822  2545 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:00.020874 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:13:59.912806  2492 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.023557  2134 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:00.064141  2415 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:00.034461  2510 consensus_meta.cc:287] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.808s	user 0.006s	sys 0.005s
W20260709 13:14:00.025431  2466 consensus_meta.cc:287] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.765s	user 0.004s	sys 0.010s
I20260709 13:14:00.039263  2484 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:00.040627 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:00.089200  2572 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:00.051944 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.144094 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.144702 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.118867  2545 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:00.069756 28231 rpcz_store.cc:276] 0709 13:13:58.931835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.931963 (+   128us) service_pool.cc:224] Handling call
0709 13:13:58.932313 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.932645 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.932673 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:58.933119 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 13:13:58.988025 (+ 54906us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:13:58.988067 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.988137 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.035985 (+1047848us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:00.069105 (+ 33120us) raft_consensus.cc:1685] finished
0709 13:14:00.069126 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.069316 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1958,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":79830400,"thread_start_us":54600,"threads_started":1,"wal-append.queue_time_us":121170}
I20260709 13:14:00.064098  2323 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:00.078962 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:00.088078 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.091456  2325 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:14:00.121131  2540 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:00.095976 28234 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 13:14:00.164757 28234 pending_rounds.cc:85] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 15
I20260709 13:14:00.032636  2511 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:00.165109 28234 pending_rounds.cc:107] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 22.16
W20260709 13:14:00.126770 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 105967us
I20260709 13:14:00.165431 28234 raft_consensus.cc:2889] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:00.165995 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:00.134927 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 52992.8us
I20260709 13:14:00.124744 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:00.167279 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1282) took 1074 ms. Trace:
W20260709 13:14:00.043363  2154 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.003s
W20260709 13:14:00.141562 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 101047us
I20260709 13:14:00.168700 28083 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:00.127830  2578 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:00.058171  2432 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:00.117744  2341 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:00.145614  2544 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:00.147606 28229 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:00.151391 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:00.072474  2542 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:14:00.154929 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:00.187431 28235 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:00.067185  2063 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:00.131233  2127 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
W20260709 13:14:00.200052 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 133960us
I20260709 13:14:00.164666  2540 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.130901  2394 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:00.092918  1816 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:14:00.096815 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:00.086766  2524 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:00.170361 28080 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:00.189085  2436 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:14:00.142349 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.180425  2273 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:00.198536  2531 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:00.167912 28154 rpcz_store.cc:276] 0709 13:13:59.093377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.093571 (+   194us) service_pool.cc:224] Handling call
0709 13:13:59.094103 (+   532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.104439 (+ 10336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.104506 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.119024 (+ 14518us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.377167 (+258143us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:13:59.650548 (+273381us) log.cc:844] Serialized 14 byte log entry
0709 13:13:59.665596 (+ 15048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.665698 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.166900 (+501202us) raft_consensus.cc:1685] finished
0709 13:14:00.166935 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.167192 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8438,"prepare.queue_time_us":459319,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":2247,"raft.queue_time_us":6045,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":11775,"thread_start_us":428094,"threads_started":2,"wal-append.queue_time_us":414722}
I20260709 13:14:00.223831  2403 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:14:00.228027  2564 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:00.209033  2429 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.238065 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:00.239250 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:00.239879 28080 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:00.241111 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:00.243533  2432 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:00.244042  2486 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:00.232811  2325 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.245373  2325 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:00.247912  2325 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:00.248957 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:00.249801 28161 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:14:00.182927 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:00.161022 28233 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:00.205962  2154 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:00.212154  2540 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:00.166046  2530 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.207860 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:00.152608  2134 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:14:00.212473 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:00.166932 28198 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:00.292989 28153 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:00.293138 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 126255us
W20260709 13:14:00.214296 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 117615us
I20260709 13:14:00.119194 28155 rpcz_store.cc:276] 0709 13:13:58.910313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.910453 (+   140us) service_pool.cc:224] Handling call
0709 13:13:59.998149 (+1087696us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b540010ac54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:13:59.998323 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222791936}
I20260709 13:14:00.176811 28228 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:00.297246 28228 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:00.298375 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:00.207778  2149 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.013s	sys 0.022s
I20260709 13:14:00.169525 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:00.231812 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:00.303136 28085 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:00.237315 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:00.072109 27983 rpcz_store.cc:269] 0709 13:13:16.217011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.370682 (+153671us) service_pool.cc:224] Handling call
0709 13:14:00.071636 (+43700954us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:55.721622 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:55.722306 (+   684us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:55.722364 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:13:55.722384 (+    20us) tablet.cc:662] Decoding operations
0709 13:13:55.723781 (+  1397us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:55.723866 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:55.723892 (+    26us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:13:55.963681 (+239789us) tablet.cc:701] Row locks acquired
0709 13:13:55.997267 (+ 33586us) write_op.cc:260] PREPARE: finished
0709 13:13:56.008329 (+ 11062us) write_op.cc:270] Start()
0709 13:13:56.008721 (+   392us) write_op.cc:276] Timestamp: P: 1783602836008229 usec, L: 0
0709 13:13:56.010434 (+  1713us) op_driver.cc:348] REPLICATION: starting
0709 13:13:56.168091 (+157657us) log.cc:844] Serialized 4868 byte log entry
0709 13:13:57.018318 (+850227us) op_driver.cc:464] REPLICATION: finished
0709 13:13:57.234727 (+216409us) write_op.cc:301] APPLY: starting
0709 13:13:57.240220 (+  5493us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:57.845326 (+605106us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce672e28 00007f59d20ad581 00007f59d20acd58 00007f59d20b19f1 00007f59d20b19a9 00007f59d20b1939 00007f59d20b1901 00007f59d20b0b2d 000000000038f354 000000000038f199
0709 13:13:57.970484 (+125158us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:57.972045 (+  1561us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:58.040027 (+ 67982us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:58.040324 (+   297us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:13:58.040367 (+    43us) write_op.cc:312] APPLY: finished
0709 13:13:58.040678 (+   311us) log.cc:844] Serialized 113 byte log entry
0709 13:13:58.159640 (+118962us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:58.168275 (+  8635us) write_op.cc:454] Released schema lock
0709 13:13:58.175713 (+  7438us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:58.283471 (+107758us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67432a 00007f59ce5f9f14 00007f59ce5f9ec3 00007f59ce5f9e38 00007f59ce5f9df3 00007f59ce5f90dc 00007f59ce622e16 00007f59ce60acbb 00007f59ce5f539d 00007f59ce5f8d75
0709 13:13:58.398562 (+115091us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b180035dac0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a522 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:13:58.398734 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:58.398770 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1425694,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":188314,"apply.run_cpu_time_us":31991,"apply.run_wall_time_us":976467,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54455,"num_ops":12,"prepare.queue_time_us":549341,"prepare.run_cpu_time_us":46348,"prepare.run_wall_time_us":2146809,"raft.queue_time_us":422065,"raft.run_cpu_time_us":20254,"raft.run_wall_time_us":1328510,"replication_time_us":976630,"spinlock_wait_cycles":42329472,"thread_start_us":806294,"threads_started":5,"wal-append.queue_time_us":247991,"wal-append.run_cpu_time_us":29059,"wal-append.run_wall_time_us":2712735}]]}
I20260709 13:14:00.303521 28077 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:00.253623 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:00.249890 28081 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:00.255405 28161 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:00.306586 28079 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:14:00.120236 28761 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.001s	sys 0.035s
W20260709 13:14:00.307233 28761 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.001s	sys 0.035s
I20260709 13:14:00.256285 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:00.250908 28233 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:00.252882 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:00.195750 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:00.258812 28764 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:00.210788  2394 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.291215 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:00.188855  2578 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.291558 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:00.299082 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 91291.8us
I20260709 13:14:00.300482 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:00.296526 28125 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:00.232997  2531 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:00.309439 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:00.324862 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:00.325446 28231 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:00.310386 28229 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:00.312809 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:00.315757 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.318041 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:00.304951 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:00.298151 28228 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.330327 28228 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:00.324002 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 154552us
I20260709 13:14:00.326165 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:00.332577 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:00.306316  2149 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.042s	user 0.014s	sys 0.022s
I20260709 13:14:00.252311  2436 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:00.294780  2273 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.362577 28153 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
W20260709 13:14:00.317514  2582 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:14:00.315274  2466 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:00.319206  2394 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:00.381110 28127 leader_election.cc:336] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.623s (SENT)
I20260709 13:14:00.382715 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:00.381748 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:00.384025 28127 leader_election.cc:336] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.738s (SENT)
I20260709 13:14:00.372702 28764 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
W20260709 13:14:00.384512 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.323251  2432 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.386847 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:00.385829 28764 pending_rounds.cc:85] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 13
I20260709 13:14:00.287259  2486 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:14:00.388897 28153 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:00.288628  2154 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.374758  2484 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:14:00.355300  2096 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:00.375247  2553 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:00.315449  2564 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 13:14:00.376420  2580 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:00.387084  2571 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:00.374318  2005 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:00.302057  2530 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.229909  2545 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.400149  2432 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.421872 28231 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:00.387796  2325 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:00.305965  2063 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.381327 28233 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.315002  2341 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.399791  2466 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:00.418563  2453 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:00.412084  2339 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 13:14:00.386807  2273 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.411918  2526 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:14:00.452167 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.423292 28728 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:00.427389 28764 pending_rounds.cc:107] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 17.14
W20260709 13:14:00.607790 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.607869 28764 raft_consensus.cc:2917] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Aborting config change with OpId 17.14: Aborted: Op aborted by new leader
W20260709 13:14:00.608326 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.608358 28764 raft_consensus.cc:2889] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 18 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:00.525925 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 1156) took 1199 ms. Trace:
W20260709 13:14:00.559383 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 233257us
I20260709 13:14:00.587669 28757 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:00.609637 28761 rpcz_store.cc:276] 0709 13:13:59.326643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.328445 (+  1802us) service_pool.cc:224] Handling call
0709 13:13:59.328881 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.533085 (+204204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.533116 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.533709 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:14:00.307744 (+774035us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:00.307834 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.307952 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.525643 (+217691us) raft_consensus.cc:1685] finished
0709 13:14:00.525672 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.525851 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19945,"raft.queue_time_us":43268,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":29589,"thread_start_us":877422,"threads_started":2,"wal-append.queue_time_us":662804}
I20260709 13:14:00.446818  2580 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.440640  2562 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:00.563998 28127 leader_election.cc:336] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.874s (SENT)
I20260709 13:14:00.442768  2492 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:00.446453  2588 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:00.423794  2531 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.586809  2545 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:00.613008 28762 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 18 LEARNER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:00.610219 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:14:00.614151 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.596603  2564 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:00.617934 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:00.615165 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 191946us
I20260709 13:14:00.626513 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:00.606875  2341 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:00.602986  2063 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:00.622643 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.603402  2511 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 13:14:00.346568  2510 consensus_queue.cc:1175] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0faaace2b36c4bcebf2fa6835235e92b" responder_term: 18 status { last_received { term: 17 index: 13 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:14:00.636345 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 72395.1us
I20260709 13:14:00.619580  2588 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:00.627636  2562 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.646724 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:00.616504  2580 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:00.615186  2526 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:00.614944  2453 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:00.527223  2436 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.683910 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:00.625715  2429 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:14:00.624326 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.650620 28127 leader_election.cc:336] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.887s (SENT)
I20260709 13:14:00.685307 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:00.686489 28127 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.969s (SENT)
W20260709 13:14:00.687322 28127 leader_election.cc:336] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.023s (SENT)
W20260709 13:14:00.688058 28127 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.088s (SENT)
I20260709 13:14:00.634473  2005 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:00.689878  2005 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:00.690122  2597 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:14:00.691243  2581 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:14:00.691007  2005 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.692114 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3168) took 1171 ms. Trace:
I20260709 13:14:00.693435  2325 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:00.625932  2564 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:00.688416 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:00.698645 28127 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.091s (SENT)
I20260709 13:14:00.699446 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:00.676412  2583 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 13:14:00.648033  2531 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:00.692515 28160 rpcz_store.cc:276] 0709 13:13:59.521069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.908252 (+387183us) service_pool.cc:224] Handling call
0709 13:13:59.908718 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:00.210926 (+302208us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:00.264531 (+ 53605us) log.cc:844] Serialized 14 byte log entry
0709 13:14:00.443944 (+179413us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:00.480326 (+ 36382us) log.cc:844] Serialized 26 byte log entry
0709 13:14:00.537773 (+ 57447us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:00.537837 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.537945 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.691904 (+153959us) raft_consensus.cc:1685] finished
0709 13:14:00.691923 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.692054 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70223,"prepare.queue_time_us":58114,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":4216,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":41219,"thread_start_us":480071,"threads_started":2,"wal-append.queue_time_us":425421}
I20260709 13:14:00.677345  2599 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:00.677558  2584 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:14:00.664945  2134 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.647843  2492 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:00.652042  2588 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:00.681711  2339 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:00.666601  2578 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:00.648363  2443 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.058s
W20260709 13:14:00.717432  2593 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 13:14:00.716917  2096 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:00.706182  2429 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:00.758052 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 147868us
W20260709 13:14:00.784644 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.702072  2523 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:14:00.664419  2466 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:00.790097  2591 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:14:00.731225  2134 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.805461 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3169) took 1284 ms. Trace:
I20260709 13:14:00.790053  2492 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:00.805970 28156 rpcz_store.cc:276] 0709 13:13:59.521378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.962574 (+441196us) service_pool.cc:224] Handling call
0709 13:13:59.962906 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.963232 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.963259 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.963717 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:14:00.157070 (+193353us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:00.157110 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.157174 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.805144 (+647970us) raft_consensus.cc:1685] finished
0709 13:14:00.805177 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.805362 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10,"raft.queue_time_us":3524,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":741,"thread_start_us":192986,"threads_started":1,"wal-append.queue_time_us":337776}
I20260709 13:14:00.807196  2432 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:00.726821  2599 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.813330 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:00.790643  2530 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:00.817186 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.819166 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.802383  2154 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:00.825910 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.784336  2564 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:00.831180 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 148036us
I20260709 13:14:00.831142  2154 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:00.844405  2606 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.844800  2154 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.844949  2608 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.875713 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:14:00.877084 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.884212 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:00.830886  2619 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:14:00.888202  2619 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:00.888513  2627 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:00.890097  2523 consensus_queue.cc:1243] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:14:00.891731 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:00.892530 28232 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:00.806057  2526 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:00.900080 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 201460us
I20260709 13:14:00.828377  2436 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:14:00.900954 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:00.854301  2601 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:00.929502 28161 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:00.930498 28158 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:14:00.911201 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 83335us
I20260709 13:14:00.801808  2453 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.873555  2273 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:00.854909  2616 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:00.888708  2619 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:00.924883 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:00.952075 28160 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:00.925803 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:00.901067  2578 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:00.962056 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:00.932605 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:00.936233  2602 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:14:00.937026  2618 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:14:00.940181  2127 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:00.941023  2628 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:00.886300  2606 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:00.944952  2603 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:14:00.960660  2619 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:00.946244  2615 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:14:00.897846  2608 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.865649  2429 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:00.949997  2599 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.005089 28761 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:14:00.973269 28153 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:14:00.935715  2436 consensus_queue.cc:1243] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:00.900895  2613 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:14:00.933612  2562 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:01.032114  2540 consensus_meta.cc:287] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.000s
I20260709 13:14:00.945551  2377 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:14:00.988855 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 70519.8us
W20260709 13:14:00.997820  2568 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.041s	user 0.007s	sys 0.000s
I20260709 13:14:00.960281  2616 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:01.002059  2621 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:01.002287  2134 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:01.002820  2063 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:01.003383  2624 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.014279  2580 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:00.991323  2628 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.015937 28227 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:14:01.080019 28227 pending_rounds.cc:85] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 7
I20260709 13:14:01.080441 28227 pending_rounds.cc:107] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:14:01.080799 28227 raft_consensus.cc:2889] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:01.039383  2608 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.049778  2420 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:01.053622  2154 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:01.062618  2377 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:00.977116  2578 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:00.986918  2127 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:01.102025 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:00.994678 28160 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:01.012385  2606 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.082516  2624 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:01.086577  2394 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.001s
I20260709 13:14:01.127749 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:01.123529 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 202318us
I20260709 13:14:01.082782  2420 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:01.130606 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.098484  2127 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:01.135362 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:01.136437 28160 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:01.138823 28228 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:14:01.139278 28228 pending_rounds.cc:85] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 13
I20260709 13:14:01.139544 28228 pending_rounds.cc:107] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 17.14
I20260709 13:14:01.139835 28228 raft_consensus.cc:2917] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Aborting config change with OpId 17.14: Aborted: Op aborted by new leader
I20260709 13:14:01.140239 28228 raft_consensus.cc:2889] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:01.142908 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:01.154567 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 222837us
I20260709 13:14:01.068964  2540 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.074574  2621 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:01.160918 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.174475 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:01.132658  2394 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:01.177266 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.178328 28228 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:01.179584 28235 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:01.192688 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.152570  2613 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:14:01.179113  2526 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:01.205287 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.165644  2436 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:14:01.156351  2377 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.160025  2578 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:01.190629  2063 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:14:01.218080  2574 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.272126  2624 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:01.280853  2633 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:14:01.284852  2377 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:01.287006 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 1461) took 1034 ms. Trace:
I20260709 13:14:01.287469 28757 rpcz_store.cc:276] 0709 13:14:00.252172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.252538 (+   366us) service_pool.cc:224] Handling call
0709 13:14:01.286759 (+1034221us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b5400531e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:01.286935 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607102720}
I20260709 13:14:01.177381  2420 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.201431  2626 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:14:01.313469 28228 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:14:01.321293 28228 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:14:01.298810  2580 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:01.336004 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.337623  2394 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:01.344098  2545 consensus_meta.cc:287] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.715s	user 0.006s	sys 0.004s
W20260709 13:14:01.350617  2634 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 13:14:01.350793 28127 leader_election.cc:336] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.229s (SENT)
W20260709 13:14:01.328586  2588 consensus_meta.cc:287] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.001s
I20260709 13:14:01.361176  2588 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.363449  2588 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.366305 28230 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:14:01.366847 28230 pending_rounds.cc:85] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
I20260709 13:14:01.367110 28230 pending_rounds.cc:107] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:14:01.367410 28230 raft_consensus.cc:2889] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:01.370216 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.352305  2621 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.302439  2616 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:01.383800 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.326215  2420 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:01.279269  2574 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.377315  2628 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.406138  2616 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.437295  2588 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:01.493510  2622 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:14:01.494714  2647 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:14:01.496043 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3415) took 1323 ms. Trace:
I20260709 13:14:01.496402 28226 rpcz_store.cc:276] 0709 13:14:00.172561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.174207 (+  1646us) service_pool.cc:224] Handling call
0709 13:14:00.174639 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:00.181426 (+  6787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.181457 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:00.181933 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.059425 (+877492us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b180028b220. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:14:01.271699 (+212274us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:01.271758 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.271838 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:01.495719 (+223881us) raft_consensus.cc:1685] finished
0709 13:14:01.495739 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:01.495985 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6288,"raft.queue_time_us":7094,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":14113,"spinlock_wait_cycles":2846592,"thread_start_us":212053,"threads_started":1,"wal-append.queue_time_us":1195738}
I20260709 13:14:01.497433  2575 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:01.498027  2575 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:01.518388  2595 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:01.519264  2601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.009s	sys 0.004s
W20260709 13:14:01.519749  2601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.010s	sys 0.004s
I20260709 13:14:01.520133  2601 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:01.510633  2640 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
W20260709 13:14:01.530208  2642 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:14:01.531276 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3422) took 1156 ms. Trace:
I20260709 13:14:01.535584 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1290) took 1975 ms. Trace:
I20260709 13:14:01.537009 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 1424) took 1232 ms. Trace:
I20260709 13:14:01.502228  2540 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.547693 28157 rpcz_store.cc:276] 0709 13:13:59.560474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.963790 (+403316us) service_pool.cc:224] Handling call
0709 13:13:59.964072 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:00.132317 (+168245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.132375 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:00.132440 (+    65us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:00.132502 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.535197 (+1402695us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54003449d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:01.535270 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:01.535497 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386267392}
I20260709 13:14:01.547463 28229 rpcz_store.cc:276] 0709 13:14:00.374661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.047162 (+672501us) service_pool.cc:224] Handling call
0709 13:14:01.047616 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.048102 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.048145 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.058653 (+ 10508us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.166021 (+107368us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:01.166352 (+   331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.166436 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:01.531035 (+364599us) raft_consensus.cc:1685] finished
0709 13:14:01.531060 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:01.531208 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9522,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":5055,"thread_start_us":106685,"threads_started":1,"wal-append.queue_time_us":427552}
I20260709 13:14:01.547813 28235 rpcz_store.cc:276] 0709 13:14:00.304502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.304678 (+   176us) service_pool.cc:224] Handling call
0709 13:14:01.536681 (+1232003us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b5400391cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:01.536889 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1948931328}
W20260709 13:14:01.528105  2637 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
W20260709 13:14:01.541455  2645 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:14:01.509569  2616 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:01.510214  2588 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.509472  2575 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.563179  2574 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.562978  2616 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:01.563963  2127 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:01.579360 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2607) took 1282 ms. Trace:
I20260709 13:14:01.579890 28084 rpcz_store.cc:276] 0709 13:14:00.297145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.297261 (+   116us) service_pool.cc:224] Handling call
0709 13:14:00.297637 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.494229 (+1196592us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b18003e3bc0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59ce66f060 00007f59ce66fbfb 00007f59ce67a64b
0709 13:14:01.494429 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.494458 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.495002 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.511754 (+ 16752us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:01.511823 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.512052 (+   229us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:01.579008 (+ 66956us) raft_consensus.cc:1685] finished
0709 13:14:01.579031 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:01.579287 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12083,"raft.queue_time_us":123879,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5494,"spinlock_wait_cycles":4821888,"thread_start_us":148263,"threads_started":2,"wal-append.queue_time_us":66536}
W20260709 13:14:01.584789 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.554782  2619 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:01.587980  2649 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.146s	user 0.000s	sys 0.002s
I20260709 13:14:01.593190 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:01.595571 28084 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:01.597700  2572 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.598311  2572 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:01.600262  2621 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.665648  2621 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.642495  2572 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:01.662720 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3410) took 1514 ms. Trace:
I20260709 13:14:01.671660 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:01.603693  2403 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.675418  2403 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:01.676105 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.676448  2607 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:01.677273  2607 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:01.677892  2403 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:01.678762 28233 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:01.673710 28230 rpcz_store.cc:276] 0709 13:14:00.148816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.148954 (+   138us) service_pool.cc:224] Handling call
0709 13:14:00.149239 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.378415 (+1229176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.662447 (+284032us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003b4054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:01.662646 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2749070720}
I20260709 13:14:01.656260  2523 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.682034  2492 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:01.682123  2523 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.684063 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.656925  2134 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.685066 28757 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:01.685434  2134 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.687840  2134 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.688184 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.655224  2611 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.689582  2611 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.682690  2492 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:01.692226 28233 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:01.692395  2611 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.692145  2492 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.693401  2626 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.693920  2626 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.694012  2492 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:01.653146  2323 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:01.695509  2323 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:01.696184  2626 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.697091 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 1162) took 1013 ms. Trace:
I20260709 13:14:01.697350 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:01.697486 28761 rpcz_store.cc:276] 0709 13:14:00.683444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.683656 (+   212us) service_pool.cc:224] Handling call
0709 13:14:01.696934 (+1013278us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54007008d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:01.697039 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736933760}
W20260709 13:14:01.663450 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:01.626273 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.699011 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:01.693004 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3409) took 1544 ms. Trace:
I20260709 13:14:01.701005 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:01.702051 28764 raft_consensus.cc:2393] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:01.661473 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1417) took 2087 ms. Trace:
I20260709 13:14:01.702440 28234 rpcz_store.cc:276] 0709 13:13:59.573742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.574006 (+   264us) service_pool.cc:224] Handling call
0709 13:13:59.574412 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:00.166212 (+591800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.661188 (+1494976us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54000de3d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:01.661408 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1198434688}
I20260709 13:14:01.706349 28079 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:01.707206  2626 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.707403 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:01.707801  2626 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.708889 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:01.708978 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:01.709659 28078 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:01.708128 28082 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:01.697978 28158 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:01.710697 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:01.711416 28084 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:01.717438 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:01.719044 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.719265 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:01.723381 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.723158 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.725406 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:01.730350 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.732445 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.732589 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.733887 28226 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:14:01.736665 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.737448 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:01.698022  2644 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260709 13:14:01.659723  2643 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.328s	user 0.001s	sys 0.001s
I20260709 13:14:01.739712 28226 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:01.749840 28226 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:01.744868 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.671896  2341 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:01.752626 28757 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:14:01.743206 28161 raft_consensus.cc:2410] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:14:01.742962 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.741096 28764 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:14:01.747077 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:01.753705 28228 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:01.754026 28158 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:01.747792 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.754145 28082 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:01.754559 28229 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:14:01.747928 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:01.747349 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:01.643638  2588 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:01.641631  2653 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:14:01.716863  2626 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.668130  2621 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.758437 28078 raft_consensus.cc:2468] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
W20260709 13:14:01.758919 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.697783  2523 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.646546  2619 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:01.782819 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:01.761235 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.743631  2636 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.739209  2607 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.750875 28226 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:01.733418  2127 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:01.700766 28233 rpcz_store.cc:276] 0709 13:14:00.148466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.148658 (+   192us) service_pool.cc:224] Handling call
0709 13:14:00.149134 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.678310 (+1529176us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54002eb6a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce5eca1d 00007f59ce5d31a4 00007f59ce655447 00007f59ce66dfe7 00007f59ce66f7de 00007f59ce679b49 00007f59ce67a416 00007f59ce676d8f 00007f59ce67683e 00007f59d363b419
0709 13:14:01.692747 (+ 14437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.692944 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656774016}
I20260709 13:14:01.776521 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:01.641350  2628 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.733022  2562 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.793285 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:01.796814  2646 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:14:01.696570  2323 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.798343  2511 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.799299 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.800153 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.803668 28226 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:01.834942 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:01.803951 28235 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:01.761478  2341 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:01.808768 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.844686 28230 raft_consensus.cc:2368] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19.
I20260709 13:14:01.845651 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.846642 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:01.847249 28228 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:01.848295  2273 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.848451  2429 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:01.849046  2273 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.849610  2341 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:01.850174  2377 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:01.850592  2377 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.850734  2341 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:01.850972  2377 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:01.851752  2273 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:01.849730  2429 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.853793 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.585317  2601 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:01.854583  2592 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:01.818042  2607 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.858803  2607 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.833319 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.838472 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.838580 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.841063  2453 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:01.860420 28761 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:01.842159  2492 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:14:01.845463  2511 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:01.852881 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.854918 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.870394 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.871241 28230 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:01.841627  2544 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:01.862164 28229 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:01.878440 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48588.6us
I20260709 13:14:01.837853  2096 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:14:01.830402 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55395.8us
I20260709 13:14:01.832394  2619 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.866235 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.805944  2636 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:01.741683 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:01.909821 28084 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:01.893540 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:01.911437 28230 raft_consensus.cc:2410] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:14:01.886961 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:01.899075 28127 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.522s (SENT)
I20260709 13:14:01.912346 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:01.900456 28228 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:01.809376 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:01.905997 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 60870.8us
I20260709 13:14:01.867033 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.829993  2628 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.915376 28229 raft_consensus.cc:2368] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:01.909693 28049 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:01.854610  2429 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:01.835269  2134 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:14:01.865141 28226 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:14:01.919809 28226 pending_rounds.cc:85] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 12
I20260709 13:14:01.920135 28226 pending_rounds.cc:107] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:14:01.920440 28226 raft_consensus.cc:2889] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:01.922114 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.922721 28226 raft_consensus.cc:2376] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19. Re-sending same reply.
I20260709 13:14:01.923588 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.913805 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.917771 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.904628 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:01.832662  2562 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.927976 28125 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:01.928799  2486 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:01.929067  2562 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:01.925625 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.929931 28230 raft_consensus.cc:2368] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:01.929724  2154 raft_consensus.cc:2804] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:01.930599  2154 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:01.930893 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.932014 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.931674  2154 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:01.934998 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.935545 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.936638 28228 raft_consensus.cc:2410] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:14:01.938055 28228 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:14:01.938476 28228 pending_rounds.cc:85] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 11
I20260709 13:14:01.938755 28228 pending_rounds.cc:107] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:14:01.939054 28228 raft_consensus.cc:2889] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:01.846015  2127 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:14:01.929225  2486 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:01.926082  2571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.322s	user 0.014s	sys 0.013s
I20260709 13:14:01.903234  2096 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.916879 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:01.912151  2610 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:01.941267 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:01.941828 28228 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:01.945494 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:01.860072  2595 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 13:14:01.946883 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:01.947583 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:01.977593 28229 raft_consensus.cc:2368] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:01.972198 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:01.914424  2544 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:01.917759  2596 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.979877 28084 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 23.
W20260709 13:14:01.982182 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 67298.9us
I20260709 13:14:01.916827  2432 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:01.992705 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:01.969812  2469 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:01.997586  2469 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:01.997783 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.998613 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.913177  2619 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:01.979358  2096 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:01.997009 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 55832.8us
I20260709 13:14:02.010877 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:01.973562  2486 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:01.976991  2571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.380s	user 0.014s	sys 0.013s
I20260709 13:14:01.990716  2580 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:02.013506 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:02.014900 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:01.944947 28226 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:02.020375 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:01.994843  2532 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:02.024511 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:02.026134 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.011610  2469 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:01.994951  2403 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
W20260709 13:14:02.026693  2605 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.231s	user 0.004s	sys 0.001s
I20260709 13:14:01.998881  2596 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.018659  2572 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:02.023895  2595 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:02.039882 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:02.021394  2580 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:02.021867  2571 raft_consensus.cc:697] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:02.041724 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.026523  2511 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:02.035565  2134 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:02.045888 28228 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:01.993321  2610 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:02.057192 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:02.058125 28228 raft_consensus.cc:2376] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19. Re-sending same reply.
I20260709 13:14:02.059278 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:02.040305  2607 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:02.038693  2627 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.044171  2580 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:02.073400  2628 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:02.074873  2619 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:02.078861  2627 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.092149  2627 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:02.087074 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.096589  2323 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:02.100584 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3407) took 1955 ms. Trace:
I20260709 13:14:02.101159 28227 rpcz_store.cc:276] 0709 13:14:00.145419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.146120 (+   701us) service_pool.cc:224] Handling call
0709 13:14:00.146541 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.081485 (+934944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.100249 (+1018764us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b54001f4fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:02.100510 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1973538176}
W20260709 13:14:02.102464  2657 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.213s	user 0.002s	sys 0.000s
I20260709 13:14:02.102723 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:02.104182 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:02.050920  2134 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.105584 28227 raft_consensus.cc:2368] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:02.113076 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:02.115432 28127 leader_election.cc:336] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.588s (SENT)
I20260709 13:14:02.116039 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:02.113823 28228 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:14:02.118127 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:02.125193 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:02.118168 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:02.119355 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 270217us
I20260709 13:14:02.065356  2607 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:02.122668 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.126965 28228 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:14:02.116041  2134 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:02.048837  2544 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:02.185593 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.242754 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:02.163254 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.247852 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:02.143362 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.244190 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:02.267174 28127 leader_election.cc:336] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.613s (SENT)
I20260709 13:14:02.324949 28227 raft_consensus.cc:2368] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:02.326313 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:02.327145 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:02.395573 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.442550 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:02.445124 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 196709us
I20260709 13:14:02.447831 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:02.448616 28084 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:02.449870 28078 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:02.182684 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.122634  2628 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:02.225374  2663 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:14:02.463697 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:02.459439 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 64878.2us
I20260709 13:14:02.469269 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:02.470112 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:02.141567  2607 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:02.229033  2636 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:02.201710  2154 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:02.458154 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:02.442836  2432 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:02.512337  2651 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.019s	user 0.024s	sys 0.015s
I20260709 13:14:02.460665  2621 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:02.459964  2523 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:02.519944 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.154717  2453 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.339s
I20260709 13:14:02.465754  2492 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:02.243547  2377 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:02.534018 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.467216  2619 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:14:02.543673  2667 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:02.567246  2667 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:02.567996 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.569557  2667 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:02.217908  2571 consensus_queue.cc:237] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:02.571905  2486 consensus_meta.cc:287] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.001s
I20260709 13:14:02.466085  2606 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:14:02.528895  2621 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:02.549407  2666 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:14:02.574864  2621 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.575179  2667 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:02.575706  2667 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.532918  2523 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.578462  2667 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.579584  2523 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.581895  2613 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:02.582455  2613 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:02.583514  2613 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.553440  2628 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:02.585104  2628 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.586810  2610 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:02.588157  2628 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.588653  2564 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:02.590397  2596 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:02.592072  2575 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:02.554859  2377 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.537011  2492 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.529191  2432 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:02.541307  2664 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
W20260709 13:14:02.563395  2662 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.475s	user 0.001s	sys 0.000s
W20260709 13:14:02.564402  2341 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.713s	user 0.012s	sys 0.000s
I20260709 13:14:02.538393  2484 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.572453  2486 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.517688  2154 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:02.613276  2096 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.001s
W20260709 13:14:02.708878 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.032s	sys 0.112s
W20260709 13:14:02.773123 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.033s	sys 0.112s
W20260709 13:14:02.774199 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 753834us
I20260709 13:14:02.714872 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2816) took 1015 ms. Trace:
W20260709 13:14:02.479630 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 454375us
I20260709 13:14:02.782635  2674 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:14:02.719098 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:02.784801 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.619575  2484 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.722397 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:02.725905 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:02.787930 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 65991.1us
I20260709 13:14:02.730331 28233 raft_consensus.cc:2410] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 13:14:02.733349 28234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.021s	sys 0.054s
W20260709 13:14:02.789705  2688 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:14:02.790171 28234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.022s	sys 0.054s
I20260709 13:14:02.659475  2492 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.618690  2613 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:02.793114  2492 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:02.794108  2492 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:02.675282  2607 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.795042 28762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1468) took 1114 ms. Trace:
I20260709 13:14:02.797175  2680 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:02.799113  2607 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.798622  2680 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.770558  2523 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:02.800894  2523 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.789979  2628 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:02.805032  2628 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.805835  2621 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:02.807083  2686 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:14:02.806910  2621 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.795055  2492 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.810619  2680 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.810839  2432 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:02.811237 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.809155  2667 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:02.767772  2096 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.777005 28159 rpcz_store.cc:276] 0709 13:14:01.698919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.699414 (+   495us) service_pool.cc:224] Handling call
0709 13:14:01.699754 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:01.717127 (+ 17373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.717161 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:01.745626 (+ 28465us) log.cc:844] Serialized 52 byte log entry
0709 13:14:02.032745 (+287119us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:02.045538 (+ 12793us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.053572 (+  8034us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.057486 (+  3914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.057610 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.713890 (+656280us) raft_consensus.cc:1685] finished
0709 13:14:02.713974 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.714516 (+   542us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16920,"prepare.queue_time_us":1117551,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":27356,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":24461,"thread_start_us":226884,"threads_started":1,"wal-append.queue_time_us":449553}
I20260709 13:14:02.804423  2154 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.808465  2628 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.811689  2492 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.804477  2523 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.820248  2486 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:02.840279  2571 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:02.841325  2571 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:02.795761 28762 rpcz_store.cc:276] 0709 13:14:01.680699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.680856 (+   157us) service_pool.cc:224] Handling call
0709 13:14:01.681188 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.681547 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.681576 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.682054 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.946733 (+264679us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:01.946786 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.946873 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.794361 (+847488us) raft_consensus.cc:1685] finished
0709 13:14:02.794417 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.794751 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1790,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":312,"thread_start_us":264296,"threads_started":1,"wal-append.queue_time_us":362502}
I20260709 13:14:02.848111  2617 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:02.848939  2682 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.849434  2617 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.850015  2682 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.856721  2682 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:02.826788 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:02.862926  2571 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:02.877364 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:02.879369  2484 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.879556 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:02.884953 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:14:02.885732 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.886013 28226 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:02.886319 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:02.886296 28156 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.007s	sys 0.007s
I20260709 13:14:02.833570  2667 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.896986 28230 raft_consensus.cc:1217] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Deduplicated request from leader. Original: 16.13->[18.14-18.14]   Dedup: 18.14->[]
I20260709 13:14:02.897787 28230 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:14:02.895027 28156 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.007s	sys 0.008s
I20260709 13:14:02.901958  2377 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.892413  2621 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:02.912703 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.892199  2617 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:02.917420 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:02.918428 28160 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:02.909277  2096 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.924948  2674 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:02.928359 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3195) took 1772 ms. Trace:
I20260709 13:14:02.930104 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:02.929065 28156 rpcz_store.cc:276] 0709 13:14:01.155805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.155981 (+   176us) service_pool.cc:224] Handling call
0709 13:14:01.165830 (+  9849us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.166213 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:01.930421 (+764208us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b180042cd60. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:14:02.141988 (+211567us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.916752 (+774764us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:02.917843 (+  1091us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.918135 (+   292us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:02.918210 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.918359 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.927817 (+  9458us) raft_consensus.cc:1685] finished
0709 13:14:02.927977 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.928242 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":140654,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":4449,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":6327,"spinlock_wait_cycles":2310400,"thread_start_us":975968,"threads_started":2,"wal-append.queue_time_us":714217}
W20260709 13:14:02.920326  2469 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.876s	user 0.012s	sys 0.010s
I20260709 13:14:02.931001  2667 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:02.909751  2377 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.931927 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:02.933485  2377 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:02.931579  2469 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.936230  2469 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:02.936854 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.937918  2377 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:02.938263  2580 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.790s	user 0.015s	sys 0.000s
I20260709 13:14:02.938892  2377 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.939008  2580 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.942361  2377 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:02.943912  2674 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:02.947743  2674 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:02.954072 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:02.955091 28078 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:14:02.956660 28079 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:02.958104 28156 raft_consensus.cc:1217] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 18.11->[19.12-21.13]   Dedup: 19.12->[21.13-21.13]
I20260709 13:14:02.958956 28156 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: [19.12-21.13]
I20260709 13:14:02.958773 28728 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:02.920668  2486 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:02.962460  2377 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:02.963115  2377 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.963620  2377 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:02.967137 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:02.969282 28160 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 16: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:02.942406  2532 raft_consensus.cc:1064] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:14:02.971118  2532 raft_consensus.cc:1068] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:14:02.973064 28156 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: [17.10-19.11]
W20260709 13:14:02.930121 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 94584.4us
I20260709 13:14:02.976223 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1432) took 1149 ms. Trace:
I20260709 13:14:02.976691 28232 rpcz_store.cc:276] 0709 13:14:01.739792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.744898 (+  5106us) service_pool.cc:224] Handling call
0709 13:14:01.745219 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:01.748080 (+  2861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.748113 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:01.748720 (+   607us) log.cc:844] Serialized 52 byte log entry
0709 13:14:02.530777 (+782057us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b180022a680. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:14:02.732242 (+201465us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:02.827642 (+ 95400us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.870713 (+ 43071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.873894 (+  3181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.878044 (+  4150us) raft_consensus.cc:1685] finished
0709 13:14:02.888011 (+  9967us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.889302 (+  1291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7898,"prepare.queue_time_us":139019,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":6267,"raft.queue_time_us":14100,"raft.run_cpu_time_us":430,"raft.run_wall_time_us":30573,"spinlock_wait_cycles":25808000,"thread_start_us":292081,"threads_started":2,"wal-append.queue_time_us":892549}
I20260709 13:14:02.931042 28228 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
W20260709 13:14:02.981806 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.982355 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:02.985073  2486 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:03.001073 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.996265 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2610) took 1018 ms. Trace:
I20260709 13:14:02.988615 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:03.002607 28229 rpcz_store.cc:276] 0709 13:14:01.997067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.997490 (+   423us) service_pool.cc:224] Handling call
0709 13:14:02.995884 (+998394us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b5400286c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:02.996062 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1767190400}
I20260709 13:14:02.990803  2154 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 13:14:03.003988  2486 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:02.982348 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:02.997100 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 220553us
I20260709 13:14:02.998848  2572 raft_consensus.cc:1064] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
W20260709 13:14:03.009634  2572 raft_consensus.cc:1068] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:14:02.989030  2617 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:14:03.011171 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2588) took 1201 ms. Trace:
I20260709 13:14:02.992276 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2602) took 1163 ms. Trace:
I20260709 13:14:03.011679 28230 rpcz_store.cc:276] 0709 13:14:01.809867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.935967 (+126100us) service_pool.cc:224] Handling call
0709 13:14:02.896547 (+960580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.898211 (+  1664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.010891 (+112680us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b54003ca0d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:03.011090 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2188996864}
I20260709 13:14:03.012677 28233 rpcz_store.cc:276] 0709 13:14:01.828817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.917455 (+ 88638us) service_pool.cc:224] Handling call
0709 13:14:02.991980 (+1074525us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54000e9054. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:02.992187 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1844461440}
I20260709 13:14:03.023253 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:03.012205  2636 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.023820 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:03.023550 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:03.030292 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:03.032228  2636 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.034844 28235 raft_consensus.cc:2410] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
W20260709 13:14:03.036319 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.027937  2572 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:14:03.036132 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:03.028043 28232 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:03.044999 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:03.046293  2484 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:03.048336  2484 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:03.048791  2484 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:03.056172 28227 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:14:03.062672 28235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2590) took 1252 ms (client timeout 1522 ms). Trace:
I20260709 13:14:03.036041  2377 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.066160 28156 raft_consensus.cc:2410] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:14:03.064474 28235 rpcz_store.cc:269] 0709 13:14:01.810339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.838218 (+ 27879us) service_pool.cc:224] Handling call
0709 13:14:03.055144 (+1216926us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400305f54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:03.062600 (+  7456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2699608448}
I20260709 13:14:03.069156 28160 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:03.047755  2580 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:03.071951 28226 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:14:03.081208 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:03.087548 28154 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
W20260709 13:14:03.092454 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.092957 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:03.093504  2627 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:03.093858 28159 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:14:03.094326  2627 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:03.097360  2661 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:03.100665 28762 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:03.098265  2661 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.108171  2661 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:03.109772 27982 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 15 to 22, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 22 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.121604 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:03.124128 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:03.124945  2661 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:03.076339  2154 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:03.124862  2611 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:14:03.136247 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49539us
I20260709 13:14:03.129082  2154 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.132802 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:03.141009  2154 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.143725  2154 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:03.154523  2453 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:03.158509 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:03.159526 28159 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:03.160971  2627 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:03.161597 28049 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:03.170778 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.174059  2453 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:03.161266 27982 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 19 to 21, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 21 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:03.176379 28160 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:03.175079  2453 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.187103 28156 raft_consensus.cc:2410] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:14:03.189702 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.192859  2612 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:03.194124 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 169313us
I20260709 13:14:03.179302  2636 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:03.184662  2469 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:03.202059 28762 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:03.178455  2484 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.208683 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.210191 28761 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:03.210335 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.211602 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.212497 28763 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:14:03.212838 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.214578 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.214969  2453 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:03.215401 28760 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 24.
I20260709 13:14:03.216789 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.217315 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.218371 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.216790 28079 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:03.219146 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.220072 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.220649 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.223493 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:03.225309 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.225668 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.225778 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.226759 28755 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:03.227537 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:03.228036 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.228880 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:03.235603 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 111615us
I20260709 13:14:03.205948 27982 catalog_manager.cc:5697] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 13 to 16, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 16 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.204325  2469 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:03.240422 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.241170 28755 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 23.
I20260709 13:14:03.242233 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.242528 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.237309  2154 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:03.246649 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.248208 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.249186 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.249922 28080 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:14:03.250555  2469 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.252002  2484 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:03.254400  2134 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.796s	user 0.018s	sys 0.007s
I20260709 13:14:03.255003  2134 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:03.255646 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.256472 28077 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:14:03.257562  2698 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:14:03.257074 27982 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.258116 28080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25d46d07733d450e897e3a72d765e511"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:03.258894 28080 raft_consensus.cc:606] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:03.260540 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:03.268859 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:03.269053 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61971.8us
I20260709 13:14:03.271157 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:03.278563 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:03.281158 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.283784 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:03.285006  2612 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:03.259716 27982 catalog_manager.cc:5697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 9 to 12, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 12 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:03.227581  2377 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:03.286679 28077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:03.289964 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:03.294111 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.296168 28077 raft_consensus.cc:606] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:03.296700 28077 raft_consensus.cc:614] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:03.296092 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.300119 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:03.301851 28127 leader_election.cc:400] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:03.302350 28127 leader_election.cc:403] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:14:03.304934 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:03.316234 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.303607 27982 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 14 to 17, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 17 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:03.327698 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.329209 28233 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:14:03.308313  2154 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:03.333210 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:03.324482  2134 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:03.344285 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 60655.4us
W20260709 13:14:03.345273  2575 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.705s	user 0.011s	sys 0.010s
I20260709 13:14:03.342788  2580 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:03.349032 27982 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 14 to 17, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 17 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.350996 27982 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.354691 28085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c654c99561449cb823642a1b85879ac"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:03.357059 28085 raft_consensus.cc:606] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:03.334232  2096 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.354084 27982 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 17, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 17 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:03.365806 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:03.363770 28233 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:14:03.375110  2682 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:03.375329  2484 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:03.375635  2682 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:03.375766  2484 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:03.362708  2575 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.376113  2682 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:03.376470  2575 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.378964  2575 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:03.376387  2096 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.369530  2154 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:03.382198 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:03.376200  2484 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:03.381300 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:03.389783 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:03.369971  2675 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:03.395272 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:03.396085 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 52742.8us
I20260709 13:14:03.396373 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:03.396713 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:03.394469 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:03.407398 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 174799us
I20260709 13:14:03.404227 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:03.412127 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.415616 28232 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:03.416930  2681 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:03.418146 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:03.418473 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.419539 28082 raft_consensus.cc:2410] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:14:03.424531 28233 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
W20260709 13:14:03.425899  2683 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:14:03.426775 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.430171  2580 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:03.430276 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.430958  2580 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.431252 28080 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:03.431505  2580 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:03.431999 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.433446 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.433036 27982 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 11 to 13, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 13 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:03.433959 28084 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:03.434104 28085 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:03.436236 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:03.437634  2617 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:03.438052  2617 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:03.438439  2617 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:03.439978 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:03.440798 28235 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:03.440948  2486 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:03.442483 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:03.435132 27982 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 14 to 18, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 18 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:03.451326 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:03.457104 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45772.5us
I20260709 13:14:03.418164  2096 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:03.467792 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.473665 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.448505  2453 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:03.479403 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.480520 28079 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:03.487506 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:03.490021 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:03.491685 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:14:03.497330  2707 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:14:03.499310  2564 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:03.499922  2564 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:03.500332  2564 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:03.500204 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:03.500885 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.501215 28078 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:03.501456 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.499182 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.491737 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:03.502216 28079 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:03.508517 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.506700 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:03.500886 27982 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 13 to 19, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 19 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.508140 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.461736  2611 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:03.516332 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.525694 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:03.527000 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 89346.7us
I20260709 13:14:03.530748 28234 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:03.531107 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.534583 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.536091 28078 raft_consensus.cc:2368] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19.
I20260709 13:14:03.538931 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.540092 28079 raft_consensus.cc:2368] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
I20260709 13:14:03.483151  2675 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:03.491734  2484 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.500403  2287 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 13:14:03.526806  2612 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:03.557472 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:03.559476 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.573985  2287 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:03.574760  2287 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:03.563858  2612 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:03.576179  2612 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:03.578388  2484 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:03.565629 27982 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 12 to 14, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 14 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:03.581897  2287 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.584722  2715 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:03.585434  2715 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:03.585914  2715 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:03.546283  2486 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:03.589183  2486 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:03.590605  2486 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:03.596117 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.596647  2675 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:03.596928 28081 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:03.545257  2469 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:03.602739 28235 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:03.603824 28232 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:03.602746 28077 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
W20260709 13:14:03.627317 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 101823us
W20260709 13:14:03.629299 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 137695us
W20260709 13:14:03.604589  2701 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:14:03.640529 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:03.639917 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 186429us
I20260709 13:14:03.595479  2486 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:03.640223 27982 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index -1 to 11, term changed from 14 to 16, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 16 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:14:03.650640 28083 raft_consensus.cc:2410] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:14:03.651032 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:03.605659  2154 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:03.670246 28084 raft_consensus.cc:1217] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Deduplicated request from leader. Original: 20.15->[22.16-22.16]   Dedup: 22.16->[]
I20260709 13:14:03.566330  2611 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.681150 28084 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:14:03.695840 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:03.701491 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 209891us
W20260709 13:14:03.711648 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50146.2us
W20260709 13:14:03.713227 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 234071us
I20260709 13:14:03.655658  2627 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:03.716878 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:03.697173  2704 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.251s	user 0.004s	sys 0.000s
I20260709 13:14:03.700033  2154 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:03.719838 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.755342 27982 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 13 to 17, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 17 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:03.718732  2627 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:03.758915 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:03.768235 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:03.765930 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.766031 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:03.776032 28077 raft_consensus.cc:1217] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Deduplicated request from leader. Original: 12.7->[13.8-13.8]   Dedup: 13.8->[]
I20260709 13:14:03.776607 28077 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:14:03.725353  2154 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:03.707633  2675 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:14:03.770108 28759 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 24.
W20260709 13:14:03.781690 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 119580us
I20260709 13:14:03.779138 28730 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:03.800047 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.782614  2453 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:03.785938  2096 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:14:03.787882  2713 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:14:03.792832  2601 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:03.793174  2592 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:03.794574 28080 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:03.795603 28084 raft_consensus.cc:1217] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Deduplicated request from leader. Original: 17.10->[18.11-18.11]   Dedup: 18.11->[]
I20260709 13:14:03.796375  2714 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:03.773204  2627 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:03.820137 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.797257 27982 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 3 to 8, term changed from 10 to 14, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195), 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:03.797544  2154 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:03.823558 28084 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:14:03.798417  2674 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:03.813577  2453 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:03.844888 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:03.845584 28226 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:03.847669 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.842461  2575 raft_consensus.cc:1064] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:14:03.818622  2675 consensus_queue.cc:1243] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:03.865406 28232 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:03.859125 28235 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:03.861852 28759 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:03.862031 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73200.1us
I20260709 13:14:03.854758 27982 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:03.875917 28084 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:03.881402 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.882817 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.883210 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.886231 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:03.871922 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:03.843566  2724 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:03.890669 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.895213 28082 raft_consensus.cc:2410] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
W20260709 13:14:03.897089 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:03.871749  2575 raft_consensus.cc:1068] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Unable to promote non-voter 70996712c3c942468a6006602de6477f: Service unavailable: leader transfer in progress
W20260709 13:14:03.915126  2717 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:14:03.917011 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:03.917389 28077 raft_consensus.cc:1217] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Deduplicated request from leader. Original: 16.9->[17.10-19.11]   Dedup: 17.10->[19.11-19.11]
I20260709 13:14:03.918090 28077 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 13:14:03.918430 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:03.919085 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:03.925042 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 84376.3us
I20260709 13:14:03.926000 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:03.927079 28127 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:03.929075  2714 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:03.929487  2714 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:03.929884  2714 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:03.940851 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.943445 28127 leader_election.cc:400] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:03.943897 28127 leader_election.cc:403] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:03.945214  2096 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:03.948097  2453 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:03.957319  2611 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:03.958243  2453 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:03.915161 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:03.969204 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 62207us
I20260709 13:14:03.970645 28226 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:14:03.975572 28081 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:03.977526 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:03.987609 28082 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:14:03.993008 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:04.001012 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:03.902132  2724 raft_consensus.cc:697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:03.979326  2695 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 13:14:03.978762  2682 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.004s
I20260709 13:14:04.001902 28159 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:03.974985  2675 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:04.014251 28079 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:04.010515 28156 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:04.018087 28154 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:03.925069  2721 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:03.989801  2718 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:03.899947  2617 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.017226  2682 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:04.023783 28160 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:14:04.025099 28157 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:04.025740 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 194352us
I20260709 13:14:04.039791 28124 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:04.041231 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:04.029738 28161 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:04.033226 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:04.033658  2720 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260709 13:14:04.034202 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:04.046046  2682 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:04.047871 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:04.048928 28158 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:04.056010 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:04.053769 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 71018.6us
I20260709 13:14:04.054550  2730 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:04.061026 28153 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:04.051493  2601 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:03.984088 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1686970379998911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 13:14:04.095109 28199 connection.cc:256] RPC call timeout handler was delayed by -1.13089703799983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
W20260709 13:14:04.095450 28199 connection.cc:256] RPC call timeout handler was delayed by -1.13089703799983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
W20260709 13:14:04.095731 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1272970379998242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
I20260709 13:14:03.992123  2564 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.063503  2675 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:04.088876  2721 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:04.114111  2721 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:04.114558  2721 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:04.115350 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:04.121248 28759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:14:04.121150  2728 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:14:04.121943 28759 raft_consensus.cc:606] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:04.122285 28759 raft_consensus.cc:614] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:04.123723  2723 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:14:04.125164  2710 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.409s	user 0.000s	sys 0.003s
W20260709 13:14:04.125985 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 197392us
I20260709 13:14:04.126892 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:04.127767 28759 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:14:04.103314  2724 consensus_queue.cc:237] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.128331 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:04.135367  2675 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:04.140089 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 312171us
I20260709 13:14:04.065069  2730 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.143379  2730 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:14:04.102218 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1227970379998169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
W20260709 13:14:04.144924 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1101970379997965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
W20260709 13:14:04.145337 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0705970379999599s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
I20260709 13:14:04.140692  2740 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:04.124213  2723 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:04.176326  2744 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.176525  2741 raft_consensus.cc:993] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
W20260709 13:14:04.177579  2722 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.340s	user 0.004s	sys 0.000s
W20260709 13:14:04.178278  2486 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.001s
I20260709 13:14:04.181535  2617 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:04.182509 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 54265.3us
I20260709 13:14:04.110016  2592 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.064312  2612 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:04.185242 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 144088us
W20260709 13:14:04.185552  2580 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.753s	user 0.004s	sys 0.011s
I20260709 13:14:04.218727 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:04.138271  2601 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:04.190115 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 129477us
I20260709 13:14:04.194087  2469 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:04.188601  2744 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.132272  2738 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:04.238304  2738 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.238706  2738 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:04.186280  2723 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:04.209383  2734 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:04.245184  2734 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:04.245891  2564 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:04.247320  2734 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:04.210002  2733 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:04.210479  2636 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:04.249137  2636 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:04.249473  2636 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:04.249996  2730 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.250303  2636 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.251456  2636 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:04.252122  2730 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:04.211642 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:04.253415 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.253970  2612 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:04.212162  2735 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:04.255146 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:04.255252  2735 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:04.256254 28228 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:04.258085 28229 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:04.258210  2735 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:04.212492 28199 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.554s (SENT)
I20260709 13:14:04.260558 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:04.260944  2592 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:04.261658 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49247.6us
W20260709 13:14:04.262396 28199 leader_election.cc:336] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.596s (SENT)
I20260709 13:14:04.263141  2592 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:04.263363 28199 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.596s (SENT) [suppressed 27 similar messages]
I20260709 13:14:04.212713  2729 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.215055 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:04.186422 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:04.216003 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:04.209375  2715 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.623s	user 0.013s	sys 0.000s
I20260709 13:14:04.229712  2732 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.220288  2287 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:14:04.234858  2096 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:04.235563  2469 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.199919  2740 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:04.236627 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:04.237802  2744 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:04.260113 28226 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:04.264142 28228 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:04.215899  2575 raft_consensus.cc:1081] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:14:04.268852 28083 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 17. Ops: []
W20260709 13:14:04.272089 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 57090.5us
I20260709 13:14:04.272927 28227 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:04.276512  2729 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:04.277164  2134 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:04.248255  2733 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:04.248389  2627 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:04.300711 28082 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:14:04.281682 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.227990  2484 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:04.269266  2727 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:04.309921 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:04.311913 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:04.225574  2580 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.278407  2617 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:04.317260 28084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:04.318735 28084 raft_consensus.cc:606] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:04.325433 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 109023us
I20260709 13:14:04.331269  2733 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:04.332901  2685 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:14:04.334687 28079 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:04.337194 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:04.338225  2714 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.342686 28757 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:04.343544  2729 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:04.344877 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:04.297534  2134 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:04.350378  2134 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:04.352159  2134 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:04.354368  2469 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:04.355144  2486 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:04.275771  2715 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:04.356875 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.358441  2715 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:04.359555 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 64877.6us
W20260709 13:14:04.362892 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.365458  2580 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:04.374372  2154 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.000s
I20260709 13:14:04.375361  2627 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:04.375159  2154 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:04.358347 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:04.328577 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:04.387005 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:04.426573 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:04.431069 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:04.433796 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:04.434798 28082 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 25
W20260709 13:14:04.429855 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 101399us
I20260709 13:14:04.322417  2484 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:04.444216 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:04.322116  2675 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.123s
I20260709 13:14:04.314350  2721 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.469877 28759 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:04.443825  2287 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:04.467669 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:04.470830 28154 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:04.468611 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 205248us
I20260709 13:14:04.475651 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:04.483533 28082 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 25.
I20260709 13:14:04.448799  2744 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:04.500361 28084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
W20260709 13:14:04.500687 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.501114 28084 raft_consensus.cc:606] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:04.501513 28084 raft_consensus.cc:614] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:04.321353  2732 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.502287 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:04.467916  2714 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:04.507262 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:04.510213 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:04.515182 28077 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:04.517795 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:04.518559 28762 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:14:04.520538 28226 raft_consensus.cc:1217] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Deduplicated request from leader. Original: 16.9->[17.10-20.12]   Dedup: 17.10->[18.11-20.12]
I20260709 13:14:04.520913 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:04.448819  2715 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:04.520967 28232 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:04.528789 28233 raft_consensus.cc:1240] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:04.521921 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 294984us
I20260709 13:14:04.487061  2453 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:04.522951 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:04.531144 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 54338.2us
I20260709 13:14:04.521153 28226 pending_rounds.cc:85] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
I20260709 13:14:04.477372  2377 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:14:04.500082  2744 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.532547 28226 pending_rounds.cc:107] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:14:04.542528 28226 raft_consensus.cc:2889] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:04.490401  2724 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:14:04.527730  2747 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:14:04.549822 28199 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.605s (SENT)
W20260709 13:14:04.547067 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 321868us
I20260709 13:14:04.535224  2287 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:04.536269  2732 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:04.537247  2675 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:04.551239 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:04.552248 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:04.557682 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.541779  2744 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:04.545033  2714 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:04.552634  2725 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.766s	user 0.001s	sys 0.003s
I20260709 13:14:04.567296 28155 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:04.586382  2745 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:14:04.564937  2675 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.588366 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:04.595826 28154 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:04.565694  2747 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:04.553074  2715 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.548389  2738 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.571607  2154 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:04.546689  2484 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.583478  2601 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:04.596226  2675 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:04.571800  2714 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:04.696877  2287 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.699887  2747 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:04.701129 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 160217us
W20260709 13:14:04.702297  2763 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:14:04.703358  2754 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:14:04.702320 28199 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.619s (SENT)
I20260709 13:14:04.705508 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:04.712306 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 150704us
W20260709 13:14:04.713688 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 108190us
I20260709 13:14:04.727842 28081 raft_consensus.cc:1240] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:04.692408  2486 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:04.721316  2757 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:04.722319  2601 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:14:04.725350 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 193561us
I20260709 13:14:04.743830  2721 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:04.700712  2723 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.749048  2757 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:04.782537 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55773.4us
I20260709 13:14:04.784615 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:04.785633 28160 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:14:04.787577 28078 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 25: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:04.799849 28154 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:04.806234 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 227811us
I20260709 13:14:04.809571  2769 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:04.810848  2769 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:04.811342  2769 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:04.815830  2765 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:14:04.785297  2757 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:04.824312 28158 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:04.774708  2453 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:04.825639 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 123379us
I20260709 13:14:04.831018 28161 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:04.836122 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:04.837404 28156 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:04.838642 28153 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:14:04.842070 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:04.843750 28227 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c654c99561449cb823642a1b85879ac"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39712
I20260709 13:14:04.873988 28227 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:04.874476 28227 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:04.874596 28199 leader_election.cc:336] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.663s (SENT)
I20260709 13:14:04.875365 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:04.877945 28233 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:04.846146 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:04.850309  2723 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:04.812458  2769 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.880635 28234 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:04.855072 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:04.881772 28159 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:14:04.856499 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:04.857266 28158 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:14:04.857479 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:04.762128  2486 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:04.861657 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.886371 28157 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:04.944383 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:04.945319 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:04.949326 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:04.952035 28160 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 21. Current term is 22. Ops: []
W20260709 13:14:04.953409 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.962267  2758 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:04.962786  2758 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:04.963780  2758 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:04.965154 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:05.022528 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:05.058792 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 98867.1us
I20260709 13:14:05.055073 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:04.954253  2777 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:14:05.068811 28227 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:05.118184  2588 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.165s
W20260709 13:14:05.119313 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.137444  2453 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:05.148983 28159 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:14:04.948518  2738 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:05.067539  2154 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:05.143940  2744 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.005s
I20260709 13:14:05.105964  2758 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:05.210268 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:05.952814 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.238113 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:05.955524 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:05.956436 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.580046 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:05.959040  2715 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:05.960616 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 993160us
I20260709 13:14:04.967984  2729 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:05.369975 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:05.965108 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.960819  2769 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:05.610834 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.969255 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1311) took 1140 ms. Trace:
W20260709 13:14:05.966862 28730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.023s	user 0.010s	sys 0.001s
W20260709 13:14:05.973162 28730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.010s	sys 0.001s
I20260709 13:14:05.952929 28759 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:05.970902 28227 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:05.972983 28156 rpcz_store.cc:276] 0709 13:14:04.829123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.836821 (+  7698us) service_pool.cc:224] Handling call
0709 13:14:04.837192 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.883865 (+ 46673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.969169 (+1085304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:05.109876  2743 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:05.994771 28755 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 25: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:05.995134 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:05.999168 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:05.465509  2636 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.214s	user 0.017s	sys 0.000s
W20260709 13:14:05.971359  2748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.003s	sys 0.005s
W20260709 13:14:06.025494  2748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.004s	sys 0.008s
W20260709 13:14:05.220376  2747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.011s	sys 0.011s
W20260709 13:14:06.027416  2757 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.079s	user 0.008s	sys 0.005s
W20260709 13:14:06.027702  2747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.323s	user 0.025s	sys 0.026s
I20260709 13:14:06.027983  2757 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:06.031104  2757 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.031628  2747 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.240062  2588 consensus_queue.cc:1243] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:14:06.032894  2592 consensus_queue.cc:1059] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:06.032758  2747 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:05.970300  2768 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.866s	user 0.002s	sys 0.000s
I20260709 13:14:06.008576 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:05.973279  2453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.001s	sys 0.006s
W20260709 13:14:06.039608  2453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.006s	sys 0.006s
W20260709 13:14:06.009934 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 1.13532e+06us
I20260709 13:14:06.007977  2755 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:06.017194  2403 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:14:06.018134 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:05.973084  2715 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:05.976713  2714 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.005998 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:39712 (request call id 3479) took 1131 ms. Trace:
I20260709 13:14:06.051364 28158 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:06.031891  2757 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.051591 28227 rpcz_store.cc:276] 0709 13:14:04.843398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.843580 (+   182us) service_pool.cc:224] Handling call
0709 13:14:05.974367 (+1130787us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:06.000599  2154 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:06.057214 28199 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.763s (SENT)
W20260709 13:14:05.234189  2675 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.000s
I20260709 13:14:06.036897 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1467) took 1255 ms. Trace:
I20260709 13:14:06.038067  2484 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:06.038864  2788 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:14:05.747602  2721 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.008s	sys 0.017s
I20260709 13:14:05.514515  2469 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:06.075551  2721 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.106s	user 0.008s	sys 0.024s
I20260709 13:14:06.075735  2469 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.078408 28083 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:06.040695 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1318) took 1183 ms. Trace:
I20260709 13:14:05.447813  2727 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:05.630620  2486 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.052158 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1321) took 1194 ms. Trace:
I20260709 13:14:06.057727 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:06.061681 28235 rpcz_store.cc:276] 0709 13:14:04.781127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.781307 (+   180us) service_pool.cc:224] Handling call
0709 13:14:04.805212 (+ 23905us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:04.819158 (+ 13946us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:04.819204 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:04.847253 (+ 28049us) log.cc:844] Serialized 26 byte log entry
0709 13:14:05.297866 (+450613us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:05.297946 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.298073 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.036557 (+738484us) raft_consensus.cc:1685] finished
0709 13:14:06.036578 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.036821 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":386,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":9591,"thread_start_us":390378,"threads_started":1,"wal-append.queue_time_us":1176932}
I20260709 13:14:05.976922  2744 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.068045  2747 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.071964  2788 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.073194  2484 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.053591  2715 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:06.056867  2714 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.101819  2714 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.104173  2714 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:06.049161  2755 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:06.079532 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.081470  2636 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:06.080127 28158 rpcz_store.cc:276] 0709 13:14:04.858105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.994638 (+1136533us) service_pool.cc:224] Handling call
0709 13:14:06.052100 (+ 57462us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:06.106932  2715 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.083763 28154 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:06.089532 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:06.090188  2727 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.111822  2588 raft_consensus.cc:1064] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:14:06.100693  2403 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.115505  2403 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:06.121093 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1323) took 1250 ms. Trace:
I20260709 13:14:06.115135 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:06.004648  2743 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.121616 28154 rpcz_store.cc:276] 0709 13:14:04.858739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.083150 (+1224411us) service_pool.cc:224] Handling call
0709 13:14:06.083534 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:06.108476 (+ 24942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.108705 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:06.079193 28157 rpcz_store.cc:276] 0709 13:14:04.857184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.985921 (+1128737us) service_pool.cc:224] Handling call
0709 13:14:06.040609 (+ 54688us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:06.099357 28083 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:06.100350  2757 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:06.134910  2757 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.137146  2755 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:06.138923  2727 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:06.140681  2484 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.142689  2788 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.149377  2469 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:06.150358  2611 raft_consensus.cc:993] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:14:06.150893  2611 raft_consensus.cc:1081] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:14:06.155367 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:06.157191 28762 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:06.157373 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:06.157967 28764 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:06.159571 28760 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
W20260709 13:14:06.210551 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 48123.4us
I20260709 13:14:06.209908 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:06.159508  2748 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:06.060957  2675 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:06.219537 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.162827  2636 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.223141 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:06.221014 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:06.225584 28230 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:06.229535 28228 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:06.165513  2154 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.232622 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3486) took 1364 ms. Trace:
I20260709 13:14:06.235024 28762 raft_consensus.cc:2393] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 24: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:06.235306 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:06.233104 28230 rpcz_store.cc:276] 0709 13:14:04.868299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.203472 (+1335173us) service_pool.cc:224] Handling call
0709 13:14:06.225455 (+ 21983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:06.232381 (+  6926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.232561 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:06.233994 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:06.239042 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:06.240489 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1319) took 1383 ms. Trace:
I20260709 13:14:06.240932 28159 rpcz_store.cc:276] 0709 13:14:04.857507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.238804 (+1381297us) service_pool.cc:224] Handling call
0709 13:14:06.240419 (+  1615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:06.241801 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:06.217154  2748 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:06.248478 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1325) took 1389 ms. Trace:
I20260709 13:14:06.249190 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:06.247926 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:06.248937 28155 rpcz_store.cc:276] 0709 13:14:04.859347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.215291 (+1355944us) service_pool.cc:224] Handling call
0709 13:14:06.248399 (+ 33108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:06.253079 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:06.233671  2636 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.253839 28086 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:06.254664 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:06.254954 28233 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:14:06.235819  2727 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.259358 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 1191) took 1093 ms. Trace:
I20260709 13:14:06.260084 28761 rpcz_store.cc:276] 0709 13:14:05.161994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.162109 (+   115us) service_pool.cc:224] Handling call
0709 13:14:06.254506 (+1092397us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:14:06.255397 (+   891us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15640960}
I20260709 13:14:06.261117 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:06.257104 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:06.269394 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:06.262054  2783 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 13:14:06.276304 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 193867us
I20260709 13:14:06.266681  2757 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.281426 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 1194) took 1118 ms. Trace:
I20260709 13:14:06.282591 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:06.279177 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:06.281840 28760 rpcz_store.cc:276] 0709 13:14:05.162985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.163102 (+   117us) service_pool.cc:224] Handling call
0709 13:14:06.281243 (+1118141us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540043ab54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:06.281360 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":342222720}
I20260709 13:14:06.285207 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:06.285661 28199 leader_election.cc:336] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.935s (SENT)
I20260709 13:14:06.285991 28081 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:06.288918  2744 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:06.270243  2758 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:14:06.271274  2729 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 13:14:06.267257  2727 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:06.294193  2154 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:06.298804  2154 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:06.300242  2154 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:06.368096  2636 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:06.373484  2636 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:06.373926  2636 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:06.374784  2636 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.376222  2636 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:06.379093  2588 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 13, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.380231  2727 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:06.381589  2403 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.299566  2748 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:06.383165  2748 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:06.292984  2757 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.385941 28764 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:06.386643 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 1195) took 1223 ms. Trace:
I20260709 13:14:06.387105 28764 rpcz_store.cc:276] 0709 13:14:05.163320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.163430 (+   110us) service_pool.cc:224] Handling call
0709 13:14:06.157226 (+993796us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:14:06.386565 (+229339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5610752}
I20260709 13:14:06.390690  2743 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:06.391299 28124 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Timed out: RequestConsensusVote RPC to 127.26.83.196:46855 timed out after 1.789s (SENT)
I20260709 13:14:06.413161 28759 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:06.414669  2757 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:06.415378 28758 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:14:06.433785 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 1192) took 1271 ms. Trace:
I20260709 13:14:06.369383  2744 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:06.434319 28758 rpcz_store.cc:276] 0709 13:14:05.162329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.162477 (+   148us) service_pool.cc:224] Handling call
0709 13:14:06.114969 (+952492us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:14:06.433671 (+318702us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23599488}
W20260709 13:14:06.444765 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.452581 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:06.453940 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 1198) took 1115 ms. Trace:
I20260709 13:14:06.454422 28763 rpcz_store.cc:276] 0709 13:14:05.338535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.963725 (+625190us) service_pool.cc:224] Handling call
0709 13:14:06.452380 (+488655us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:14:06.453834 (+  1454us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17419136}
I20260709 13:14:06.457275 28077 raft_consensus.cc:3055] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:06.457852 28077 raft_consensus.cc:740] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:06.458992 28077 consensus_queue.cc:260] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.461071 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:06.461966 28756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 1197) took 1298 ms. Trace:
I20260709 13:14:06.462309 28756 rpcz_store.cc:276] 0709 13:14:05.163883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.447521 (+283638us) service_pool.cc:224] Handling call
0709 13:14:06.460767 (+1013246us) spinlock_profiling.cc:243] Waited 8.09 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:14:06.461899 (+  1132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18613632}
W20260709 13:14:06.470562 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.487584 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:06.490761 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 1189) took 1329 ms. Trace:
I20260709 13:14:06.491207 28757 rpcz_store.cc:276] 0709 13:14:05.161163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.161301 (+   138us) service_pool.cc:224] Handling call
0709 13:14:06.471764 (+1310463us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5800003998. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:14:06.490698 (+ 18934us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28548480}
W20260709 13:14:06.506444 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:06.471468 28126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.359s	user 0.005s	sys 0.021s
W20260709 13:14:06.508261 28126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.396s	user 0.005s	sys 0.021s
W20260709 13:14:06.509393 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 534302us
I20260709 13:14:06.517355 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:06.520866 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.012s	sys 0.035s
W20260709 13:14:06.521370 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.012s	sys 0.035s
W20260709 13:14:06.521778 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 548166us
I20260709 13:14:06.534409 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:06.538165 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:06.539650 28155 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:06.540599 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:06.541538 28158 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:06.535208 28157 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:06.547396 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:06.547560 28199 leader_election.cc:336] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 2.103s (SENT)
I20260709 13:14:06.548049 28757 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:14:06.548090 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:06.551674 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:06.551750 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:06.554353 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 163097us
I20260709 13:14:06.554824 28156 raft_consensus.cc:2368] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24.
I20260709 13:14:06.512974  2757 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.557541 28728 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:06.560137 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.552841  2403 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.565402 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:06.471961  2744 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.570997 28156 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 22: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:06.553011  2715 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:06.556344  2727 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:06.549165  2792 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:06.571445 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:06.591107 28153 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:14:06.591846 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54479.8us
I20260709 13:14:06.597530 28155 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 23: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:06.569378  2685 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:06.607111 28077 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:06.608316 28153 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:14:06.603950 28727 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.013s	sys 0.070s
W20260709 13:14:06.612265 28727 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.013s	sys 0.070s
I20260709 13:14:06.562439  2757 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:06.634758 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 79269.6us
W20260709 13:14:06.635758 28124 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Timed out: RequestConsensusVote RPC to 127.26.83.196:46855 timed out after 2.080s (SENT)
W20260709 13:14:06.635963 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.636264 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:06.639420 28155 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:06.603624  2721 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:06.577570  2403 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.605063  2796 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:06.657384 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 401105us
I20260709 13:14:06.615644  2685 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:06.670249 28199 leader_election.cc:336] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 2.195s (SENT)
I20260709 13:14:06.598629  2715 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.673565 28077 raft_consensus.cc:1275] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:14:06.674000 28077 pending_rounds.cc:85] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 14
I20260709 13:14:06.674285 28077 pending_rounds.cc:107] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:14:06.674533 28077 raft_consensus.cc:2889] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:06.674989 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:06.675612 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.677110 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:06.677487 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:06.679016 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:06.680194 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.681768 28077 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:14:06.688576  2675 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:06.691069 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:06.692174 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:06.692813 28763 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:06.650758  2721 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:06.653488  2714 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.669358  2799 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:06.701629 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:06.706331 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:06.707160 28153 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:06.693162  2782 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:14:06.678354  2747 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:06.708841 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.598824  2792 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.705617 28756 raft_consensus.cc:2376] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16. Re-sending same reply.
I20260709 13:14:06.714951 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:06.696044  2727 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:06.715422 28050 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.763s (SENT)
I20260709 13:14:06.709782 28230 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:06.729452 28227 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:06.698168  2796 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:06.719538 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64028.5us
I20260709 13:14:06.709045  2714 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:06.768500  2805 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:06.769519  2805 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:06.774438 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:06.774857 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:06.786854 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 77173us
I20260709 13:14:06.722711 28758 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:06.696535  2721 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:06.714962 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:06.726303  2723 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
W20260709 13:14:06.829505 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 74175.6us
I20260709 13:14:06.830083 28153 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:06.831496 28078 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 13:14:06.781030 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 65657.4us
I20260709 13:14:06.792636 28764 raft_consensus.cc:2393] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:06.784052 28763 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
W20260709 13:14:06.798636 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 162916us
I20260709 13:14:06.799309 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:06.721392  2744 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:06.820096 28154 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:06.836306 28198 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:06.821815 28757 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:06.703790  2729 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:14:06.840963 28050 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.769s (SENT)
I20260709 13:14:06.841589 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:06.841977 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:06.842419 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:06.847286 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:06.847679 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:06.782750  2796 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.853945 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:06.880544 28155 raft_consensus.cc:2368] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24.
I20260709 13:14:06.729532  2747 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.854630  2744 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:14:06.893846 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 341006us
I20260709 13:14:06.847357  2813 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:06.829157  2727 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:06.781237  2685 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:06.905149  2744 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:06.900784  2796 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:14:06.907123 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55853.4us
I20260709 13:14:06.853224  2816 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:06.839871  2770 consensus_queue.cc:1059] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:14:06.931284 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:06.876579  2743 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:06.848651  2580 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
W20260709 13:14:06.921975 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 80076.6us
I20260709 13:14:06.936456 28124 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:06.879736  2729 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:06.931541 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:06.936319 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:06.880035  2742 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:14:06.946346 28077 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:06.924881  2813 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:07.157099 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 220707us
W20260709 13:14:07.078732 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.157907 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 85344.9us
I20260709 13:14:06.932678  2714 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:06.937290  2792 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:07.108474  2810 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:14:07.057727  2757 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:07.118137  2801 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.391s	user 0.001s	sys 0.000s
W20260709 13:14:07.113602  2811 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 13:14:07.061394  2784 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
W20260709 13:14:07.053427  2154 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.668s	user 0.013s	sys 0.000s
I20260709 13:14:07.176262 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:07.205762 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:07.064558  2636 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.001s
W20260709 13:14:07.178988 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.209504 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:07.129714  2823 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:14:07.181449 28226 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:07.131152  2723 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:14:07.181855 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 243877us
I20260709 13:14:07.183130 28229 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:07.183410 28235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25d46d07733d450e897e3a72d765e511"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39712
I20260709 13:14:07.243654 28235 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:07.244153 28235 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:07.183785 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:07.245007 28233 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:07.175376 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 342165us
W20260709 13:14:07.245682 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.246158 28050 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.779s (SENT)
I20260709 13:14:07.128670  2727 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:07.125890  2747 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:07.193599 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.161023  2715 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:07.070647  2828 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:07.287933 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.175138  2403 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:06.948349  2805 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:06.935986  2592 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.244s
I20260709 13:14:07.298936 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.247891  2723 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:07.188095  2821 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.003s
I20260709 13:14:07.079005  2770 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.126s
I20260709 13:14:07.243304  2154 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:07.231310 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.242169 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:07.253539 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 417271us
W20260709 13:14:07.273123 28050 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.795s (SENT)
I20260709 13:14:07.321077 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:06.943954  2795 consensus_queue.cc:1048] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.282s
I20260709 13:14:07.249330 28159 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:06.937943  2816 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:07.278925  2729 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:07.271494 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 337723us
I20260709 13:14:07.305399  2732 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.129392  2744 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:07.271531  2792 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:07.334257 28235 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:07.207755  2486 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:14:07.227953 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:07.317695 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:07.322772 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.320153 28161 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:14:07.325596 28759 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:07.195168 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:07.328347 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:07.374861 28234 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:07.329502 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.330494 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:07.324213 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:07.332067 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.117506  2807 consensus_queue.cc:1059] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 13:14:07.388612 28155 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 21: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:07.388914 28231 raft_consensus.cc:2410] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
W20260709 13:14:07.391923  2840 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:14:07.393267  2486 consensus_queue.cc:1243] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:14:07.393771 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 79015.9us
I20260709 13:14:07.394861 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:07.395830  2833 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:07.397526  2807 raft_consensus.cc:3055] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:07.397958  2807 raft_consensus.cc:740] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:07.398921  2807 consensus_queue.cc:260] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:07.400348  2807 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:07.395871  2729 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:07.402458  2816 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:07.403023 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 74752.5us
I20260709 13:14:07.403321  2834 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:07.388135  2748 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:07.369705  2732 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:07.406068  2788 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:07.406365 28763 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:07.407516 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:07.407825 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:07.408139 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:07.407541  2732 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:07.406097 28756 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:14:07.408174 28755 raft_consensus.cc:2376] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15. Re-sending same reply.
I20260709 13:14:07.413066 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:07.412453 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:07.415357 28763 raft_consensus.cc:2368] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16.
I20260709 13:14:07.415903 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:07.413071 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46272.2us
W20260709 13:14:07.413269 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 140183us
W20260709 13:14:07.393357  2827 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:14:07.415689  2820 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.299s	user 0.005s	sys 0.000s
I20260709 13:14:07.364773  2792 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.408164  2835 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:07.408516  2813 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:07.411590  2838 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:07.429091 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.429405  2154 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:07.430263  2753 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:07.430344 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:07.394161  2486 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.433101 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:07.434425 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.434691 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:07.435310 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:07.437041  2721 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.597s	user 0.005s	sys 0.007s
I20260709 13:14:07.574498  2721 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:07.437373 28156 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:07.439033 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:07.440188 28152 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:07.440873 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.442103 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:07.448344 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:07.450162 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:07.465073  2831 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.146s	user 0.003s	sys 0.000s
I20260709 13:14:07.470937 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.477635 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:07.479007 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:07.478570 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 60679us
I20260709 13:14:07.484508 28233 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
W20260709 13:14:07.502817 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 186820us
I20260709 13:14:07.413628 28235 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:07.558709 28050 leader_election.cc:336] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.061s (SENT)
I20260709 13:14:07.564066 28158 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:14:07.403065  2816 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:07.559770  2792 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:07.571650  2807 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:07.621917 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:07.622220 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 191946us
I20260709 13:14:07.572863  2486 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:07.577099 28154 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 22: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:07.577916 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:07.627676 28156 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:07.563390  2782 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:07.580070 28082 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:07.579470 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.631515 28152 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 13.
I20260709 13:14:07.586015 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.590381 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:07.593689 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:07.632953 28084 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:07.578042 28160 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:07.580564 28758 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 25: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:07.583680 28757 raft_consensus.cc:2376] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18. Re-sending same reply.
I20260709 13:14:07.583251 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.579797 28161 raft_consensus.cc:2376] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17. Re-sending same reply.
I20260709 13:14:07.619065 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.585662 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:07.625661 28729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.001s	sys 0.025s
I20260709 13:14:07.578441 28159 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:07.625957 28124 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:07.636122 28729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.001s	sys 0.025s
I20260709 13:14:07.570415  2753 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:07.643183 28124 leader_election.cc:400] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16.
I20260709 13:14:07.643784 28124 leader_election.cc:403] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:14:07.634280 28155 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:07.576701  2721 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:07.584421  2805 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:14:07.646106 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.645920 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61836.5us
W20260709 13:14:07.646324 28127 connection.cc:256] RPC call timeout handler was delayed by -1.3037011310000253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1507s for a response
I20260709 13:14:07.599857  2816 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:07.650544 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.655527 28158 raft_consensus.cc:2393] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 21: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:07.656811 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.657732 28158 raft_consensus.cc:2368] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24.
W20260709 13:14:07.659286 28127 connection.cc:256] RPC call timeout handler was delayed by -1.2776011309999831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 13:14:07.659933 28127 connection.cc:256] RPC call timeout handler was delayed by -1.2371011309999176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 13:14:07.660359 28127 connection.cc:256] RPC call timeout handler was delayed by -1.2335011309999118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
W20260709 13:14:07.660775 28127 connection.cc:256] RPC call timeout handler was delayed by -1.1183011309999529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
W20260709 13:14:07.661170 28127 connection.cc:256] RPC call timeout handler was delayed by -1.1111011309999412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
W20260709 13:14:07.661609 28127 connection.cc:256] RPC call timeout handler was delayed by -1.1039011309999296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
W20260709 13:14:07.664306 28127 connection.cc:256] RPC call timeout handler was delayed by -1.0760011309998845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
I20260709 13:14:07.673628 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.707407 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:07.628466  2826 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:14:07.683106 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 124437us
W20260709 13:14:07.677824 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.675958 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 90385.4us
I20260709 13:14:07.822022 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.830468 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:08.085973 28160 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:07.845563 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:07.744609 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.786818 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:07.709523  2782 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:07.745540 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.947696 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.074488 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.582543  2723 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.106874 28159 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:08.107880 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:08.106891 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:07.674547  2837 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
W20260709 13:14:08.077739 28050 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.090s (SENT)
I20260709 13:14:08.118271 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:07.707230  2715 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 14, Time since last communication: 0.216s
I20260709 13:14:07.662878  2748 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:07.646479  2675 raft_consensus.cc:1027] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: attempt to promote peer 70996712c3c942468a6006602de6477f: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:14:07.654029  2816 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:08.076769 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 424302us
I20260709 13:14:07.662305  2747 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:14:08.079972 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 436980us
I20260709 13:14:07.637301  2742 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:08.153081  2742 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:07.585703  2824 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:14:07.637081  2796 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.625017  2828 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.202s
I20260709 13:14:08.104241 28155 raft_consensus.cc:2368] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24.
I20260709 13:14:08.079418  2757 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.140266 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:08.147827 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:08.212448 28758 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 25.
I20260709 13:14:08.155303 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:08.131671  2807 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:08.168216  2782 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:08.169821 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3219) took 1231 ms. Trace:
W20260709 13:14:08.183746 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 44780.6us
I20260709 13:14:08.248652 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:08.250665 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:08.231334 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 153604us
I20260709 13:14:08.244367 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:08.266191 28050 leader_election.cc:336] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.546s (SENT)
I20260709 13:14:08.267014 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:08.263870 28155 raft_consensus.cc:2376] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 16. Re-sending same reply.
I20260709 13:14:08.246594 28153 rpcz_store.cc:276] 0709 13:14:06.938415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.938605 (+   190us) service_pool.cc:224] Handling call
0709 13:14:06.938939 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:06.946015 (+  7076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:06.946046 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:07.166017 (+219971us) log.cc:844] Serialized 26 byte log entry
0709 13:14:07.503825 (+337808us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:07.503887 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.503974 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:08.169486 (+665512us) raft_consensus.cc:1685] finished
0709 13:14:08.169519 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.169732 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84435,"raft.queue_time_us":117489,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":8346,"thread_start_us":326760,"threads_started":1,"wal-append.queue_time_us":976994}
I20260709 13:14:08.283377 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:08.182000  2747 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:08.294656 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:08.298873 28155 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:14:08.279506  2484 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.050s	sys 0.072s
I20260709 13:14:08.226012  2403 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:08.302919 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:08.303898 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:08.279079  2816 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:08.269428  2828 consensus_queue.cc:1243] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:14:08.278726  2469 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:08.269161  2748 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:08.155138  2837 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.112197  2753 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:08.213616  2796 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.320779  2747 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.273233  2757 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.315018 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:08.328111 28084 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.000s
I20260709 13:14:08.489454 28084 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
W20260709 13:14:08.308049  2484 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.051s	sys 0.072s
W20260709 13:14:08.338587 28082 consensus_meta.cc:287] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.000s
I20260709 13:14:08.492511 28082 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:08.285290  2846 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.271447  2807 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.417179 28155 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:08.318876  2469 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:08.419745  2845 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:14:08.420123  2826 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:08.422868  2753 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:08.424717 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 92422.4us
I20260709 13:14:08.428069  2742 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:08.557024 28078 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.973s	user 0.010s	sys 0.009s
W20260709 13:14:08.557616 28078 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.973s	user 0.010s	sys 0.009s
W20260709 13:14:08.447371 28156 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.819s	user 0.010s	sys 0.001s
I20260709 13:14:08.448694 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:08.559021 28156 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:08.559667 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3244) took 1090 ms. Trace:
W20260709 13:14:08.302295  2752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.017s	sys 0.094s
I20260709 13:14:08.556900 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1477) took 1068 ms. Trace:
I20260709 13:14:08.461944 28160 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:08.496404  2794 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:08.497440  2486 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:08.497697 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 231728us
I20260709 13:14:08.505548 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2669) took 1051 ms. Trace:
I20260709 13:14:08.553555 28154 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 23: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:08.428406  2805 consensus_queue.cc:1059] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:14:08.560926 28159 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
W20260709 13:14:08.581246 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.557708 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3240) took 1121 ms. Trace:
I20260709 13:14:08.560019 28156 rpcz_store.cc:276] 0709 13:14:07.469319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.577576 (+108257us) service_pool.cc:224] Handling call
0709 13:14:08.559604 (+982028us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:08.562572 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 428617us
W20260709 13:14:08.564720 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 490130us
I20260709 13:14:08.561594 28757 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 18 LEARNER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:14:08.539431  2469 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:08.572396 28082 rpcz_store.cc:276] 0709 13:14:07.441632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.441812 (+   180us) service_pool.cc:224] Handling call
0709 13:14:08.493246 (+1051434us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:08.587738 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.604055 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:08.589838 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.569237  2752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.021s	sys 0.101s
I20260709 13:14:08.551980  2592 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:14:08.552606  2813 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.963s	user 0.008s	sys 0.009s
I20260709 13:14:08.547475  2826 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:08.581780 28158 rpcz_store.cc:276] 0709 13:14:07.435965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.658324 (+222359us) service_pool.cc:224] Handling call
0709 13:14:07.658654 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.658880 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:14:07.711186 (+ 52306us) log.cc:844] Serialized 14 byte log entry
0709 13:14:08.557207 (+846021us) log.cc:844] Serialized 14 byte log entry
0709 13:14:08.557355 (+   148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:08.557398 (+    43us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:08.557449 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.557516 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.557653 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35045,"prepare.queue_time_us":64960,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":92,"raft.queue_time_us":17964,"spinlock_wait_cycles":19780480,"thread_start_us":470517,"threads_started":1}
W20260709 13:14:08.593459 28127 leader_election.cc:336] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.536s (SENT)
I20260709 13:14:08.564889  2723 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:08.561064  2799 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16.
I20260709 13:14:08.606627  2782 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:08.627988 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:08.627247 28127 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:08.538239  2846 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.638515  2846 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:08.640173  2846 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:08.623951  2813 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.008s	sys 0.009s
I20260709 13:14:08.681389  2757 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:08.630712  2805 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:08.682621 28201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.013s	sys 0.011s
I20260709 13:14:08.635715  2782 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:08.683982 28728 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.000s	sys 0.024s
W20260709 13:14:08.684197 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 90806.5us
W20260709 13:14:08.687160 28201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.013s	sys 0.011s
W20260709 13:14:08.690001 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 89854.2us
W20260709 13:14:08.691030 28728 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.000s	sys 0.024s
I20260709 13:14:08.692005 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:08.692250 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 86245.5us
I20260709 13:14:08.678272  2154 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:08.693008 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1488) took 1319 ms. Trace:
I20260709 13:14:08.645757  2403 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
W20260709 13:14:08.688295  2848 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:14:08.696935 28230 rpcz_store.cc:276] 0709 13:14:07.374026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.374777 (+   751us) service_pool.cc:224] Handling call
0709 13:14:07.375221 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.375622 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.375660 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:07.376212 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:14:07.509918 (+133706us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:07.509977 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.510075 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:08.692697 (+1182622us) raft_consensus.cc:1685] finished
0709 13:14:08.692743 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.692932 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8612,"raft.queue_time_us":11391,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":98,"thread_start_us":133322,"threads_started":1,"wal-append.queue_time_us":835401}
I20260709 13:14:08.692065  2782 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:14:08.762995 28127 leader_election.cc:336] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.320s (SENT)
I20260709 13:14:08.650512  2846 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:08.760604  2792 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.132s	user 0.015s	sys 0.000s
I20260709 13:14:08.693202  2805 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:08.767176  2860 raft_consensus.cc:3055] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:08.779495 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:08.782589 28728 connection.cc:256] RPC call timeout handler was delayed by -1.2800830539999879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29960000000000003s for a response
W20260709 13:14:08.784178 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 171503us
I20260709 13:14:08.790457 28124 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:08.797307  2863 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 13:14:08.797423  2854 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:14:08.798552  2486 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:08.798774 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1481) took 1182 ms. Trace:
I20260709 13:14:08.799376  2486 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:08.800064  2486 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:08.696377  2826 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.799279 28077 rpcz_store.cc:276] 0709 13:14:07.616184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.616360 (+   176us) service_pool.cc:224] Handling call
0709 13:14:07.616843 (+   483us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:07.617323 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.617351 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:07.630753 (+ 13402us) log.cc:844] Serialized 52 byte log entry
0709 13:14:08.333623 (+702870us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:08.333665 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.333759 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:08.798437 (+464678us) raft_consensus.cc:1685] finished
0709 13:14:08.798463 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.798686 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":7154,"thread_start_us":248615,"threads_started":1,"wal-append.queue_time_us":944441}
W20260709 13:14:08.784123  2853 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:14:08.784399  2807 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:08.812561 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53250.6us
I20260709 13:14:08.796478  2868 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:14:08.776227  2860 raft_consensus.cc:740] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:08.819459  2866 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:08.819563  2868 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:08.810871  2856 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:14:08.820438  2868 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:08.823731  2715 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.814808 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:08.812426 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:08.562657 28085 rpcz_store.cc:276] 0709 13:14:07.480782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.482524 (+  1742us) service_pool.cc:224] Handling call
0709 13:14:07.482864 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.483089 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:07.995320 (+512231us) log.cc:844] Serialized 14 byte log entry
0709 13:14:08.548137 (+552817us) log.cc:844] Serialized 14 byte log entry
0709 13:14:08.548462 (+   325us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:08.548531 (+    69us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:08.548599 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.548719 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.548906 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1063630,"raft.queue_time_us":8217,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":9470,"thread_start_us":920104,"threads_started":2}
I20260709 13:14:08.819509  2860 consensus_queue.cc:260] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:08.828323  2860 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:08.829406  2869 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:08.829806  2869 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:08.830253 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3237) took 1395 ms. Trace:
I20260709 13:14:08.830592  2869 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.830838 28154 rpcz_store.cc:276] 0709 13:14:07.435213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.630050 (+194837us) service_pool.cc:224] Handling call
0709 13:14:08.829953 (+1199903us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b540010a754. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:08.830140 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071506688}
I20260709 13:14:08.774209  2792 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:08.842463  2792 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:08.806597  2837 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:08.846776  2805 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:08.847123 28759 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:14:08.801020  2486 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.850530 28758 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:08.851286  2844 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:08.853080 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2665) took 1638 ms. Trace:
I20260709 13:14:08.853614 28152 raft_consensus.cc:2368] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20.
W20260709 13:14:08.824013  2816 consensus_meta.cc:287] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 13:14:08.859974 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1483) took 1232 ms. Trace:
I20260709 13:14:08.793677  2796 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:08.861780 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.863268 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.820120  2866 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.860401 28083 rpcz_store.cc:276] 0709 13:14:07.627757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.627885 (+   128us) service_pool.cc:224] Handling call
0709 13:14:07.637185 (+  9300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:08.156054 (+518869us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:08.156089 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:08.156141 (+    52us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:08.156193 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.859733 (+703540us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b54000c99d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:08.859788 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.859921 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1192743168}
W20260709 13:14:08.862082 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.853677 28078 rpcz_store.cc:276] 0709 13:14:07.214946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.215451 (+   505us) service_pool.cc:224] Handling call
0709 13:14:07.322534 (+107083us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.366340 (+ 43806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.366460 (+   120us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:07.511733 (+145273us) log.cc:844] Serialized 26 byte log entry
0709 13:14:08.558012 (+1046279us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:14:08.558055 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.558136 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:08.852808 (+294672us) raft_consensus.cc:1685] finished
0709 13:14:08.852840 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.853009 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4068,"raft.queue_time_us":4171,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":8368,"thread_start_us":973791,"threads_started":1,"wal-append.queue_time_us":1108239}
I20260709 13:14:08.877437 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:08.856757  2873 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.886101 28232 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:08.891826 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.894728 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 195608us
I20260709 13:14:08.889207 28125 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:08.902180 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 111811us
I20260709 13:14:08.903337 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:08.904289 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:08.917824 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:08.921252 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 229303us
W20260709 13:14:08.891449  2861 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.168s	user 0.003s	sys 0.000s
I20260709 13:14:08.884066  2829 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:08.934284 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:08.937161 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:08.866999  2835 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:14:08.876232  2742 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:14:08.880004  2723 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:14:08.959954 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:08.960597 28228 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:14:08.961829 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:08.955873 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:08.950630 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3238) took 1515 ms. Trace:
I20260709 13:14:08.955022 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:08.957719 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:08.958293 28226 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:08.965193 28078 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:08.905114  2837 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:08.904536  2871 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.976037 28231 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 13:14:08.976285 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.964619 28152 rpcz_store.cc:276] 0709 13:14:07.435469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.644341 (+208872us) service_pool.cc:224] Handling call
0709 13:14:08.936570 (+1292229us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b5400053554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:08.950546 (+ 13976us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1766396672}
I20260709 13:14:08.824510  2715 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.926044  2807 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.927480  2844 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:08.980213 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:08.899343  2764 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 13:14:08.886718  2873 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.960175  2829 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:14:08.993137 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 89461.8us
I20260709 13:14:08.994884 28154 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 24: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:09.002651 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:09.006778 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:14:09.009406 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 75209.1us
I20260709 13:14:09.010784 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:08.946259  2867 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.066s
I20260709 13:14:09.011561 28231 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:14:08.989777  2837 raft_consensus.cc:697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:09.001099  2846 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:09.030927 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:08.986876  2807 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:08.981642  2871 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:09.018971  2805 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:09.020978  2860 raft_consensus.cc:2764] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:14:09.029686  2864 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:14:08.990365  2469 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:08.995307  2844 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.050472 28157 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:14:09.056881  2742 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:09.060782  2729 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:09.011530  2872 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:14:09.049520  2846 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.046241  2805 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:09.083979 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:09.054558  2888 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:09.090167  2888 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:09.084076  2875 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:14:09.072252  2729 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:09.097277 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1482) took 1472 ms. Trace:
I20260709 13:14:09.094718 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:09.103610 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.104029 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:09.095909 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 137757us
I20260709 13:14:09.110450 28078 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:09.087133  2805 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:09.091012  2879 raft_consensus.cc:3055] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:09.079516  2860 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:14:09.091962  2872 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:14:09.120100 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 109380us
I20260709 13:14:09.118346 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:09.089394  2866 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:09.110114  2484 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:14:09.126159 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 132263us
I20260709 13:14:09.122400 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:09.096992  2802 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:14:09.110090 28086 rpcz_store.cc:276] 0709 13:14:07.625309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.625456 (+   147us) service_pool.cc:224] Handling call
0709 13:14:07.665231 (+ 39775us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:08.090736 (+425505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:08.090782 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:08.103603 (+ 12821us) log.cc:844] Serialized 26 byte log entry
0709 13:14:08.203498 (+ 99895us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b1800109cc0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:14:08.611875 (+408377us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:08.611917 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.611986 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:09.085709 (+473723us) raft_consensus.cc:1685] finished
0709 13:14:09.085733 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:09.097189 (+ 11456us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":424961,"raft.queue_time_us":437815,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":13841,"spinlock_wait_cycles":8328448,"thread_start_us":408165,"threads_started":1,"wal-append.queue_time_us":665838}
I20260709 13:14:09.121520  2715 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:09.124347  2871 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:09.124465  2885 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:14:09.125092  2866 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:09.101935  2469 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:09.112694  2729 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:09.114208  2824 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:09.131291 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:09.140129  2807 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:09.140486  2876 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:14:09.141459  2869 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:09.142823 28086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:09.162961 28086 raft_consensus.cc:606] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:09.087314  2846 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:09.166177 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48573.2us
I20260709 13:14:09.157078 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:09.188446 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1341) took 1518 ms. Trace:
I20260709 13:14:09.102502  2878 consensus_queue.cc:1059] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:14:09.189102 28155 rpcz_store.cc:276] 0709 13:14:07.669648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.545292 (+875644us) service_pool.cc:224] Handling call
0709 13:14:08.546068 (+   776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:08.566565 (+ 20497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:08.566615 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:08.608793 (+ 42178us) log.cc:844] Serialized 26 byte log entry
0709 13:14:08.810902 (+202109us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:08.954077 (+143175us) log.cc:844] Serialized 14 byte log entry
0709 13:14:08.954331 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.954425 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:09.176273 (+221848us) raft_consensus.cc:1685] finished
0709 13:14:09.176298 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:09.188353 (+ 12055us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18458,"prepare.queue_time_us":142881,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":29484,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":42698,"thread_start_us":162065,"threads_started":1,"wal-append.queue_time_us":270474}
I20260709 13:14:09.196377  2757 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:09.199316  2757 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:09.200328  2846 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:09.200172  2757 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:09.074112  2829 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:09.122670  2872 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:09.119843  2879 raft_consensus.cc:740] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:09.136265  2866 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:09.096858  2888 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.177542  2813 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:09.178504 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:09.178830  2889 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.002s
I20260709 13:14:09.184942  2877 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:09.187186  2900 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:09.187330  2805 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.117475  2837 consensus_queue.cc:237] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:09.178850  2844 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:09.201328  2878 consensus_queue.cc:1243] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:09.198176  2782 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.303522 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:09.304226  2782 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:09.305397 28078 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 25. Current term is 26. Ops: []
I20260709 13:14:09.241241  2898 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 13:14:09.306762 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:09.239192  2753 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.677s	user 0.013s	sys 0.005s
I20260709 13:14:09.245091  2469 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:09.247143 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.308557 28226 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:09.203889  2872 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:09.206506  2866 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:09.249388 28157 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1334) took 1581 ms (client timeout 1777 ms). Trace:
W20260709 13:14:09.317642 28157 rpcz_store.cc:269] 0709 13:14:07.667831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.107563 (+439732us) service_pool.cc:224] Handling call
0709 13:14:09.249201 (+1141638us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b54000a28d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:09.249325 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1737402240}
I20260709 13:14:09.250175 28161 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:09.250491  2805 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:09.222910  2900 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.255522 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.261893 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:09.262202  2873 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:09.262108  2743 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:09.265844 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:09.269699 28229 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:09.272570  2729 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:09.279341 28159 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1328) took 1613 ms (client timeout 1818 ms). Trace:
I20260709 13:14:09.282658 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:09.337499 28152 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:09.340795  2879 consensus_queue.cc:260] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:09.342562  2879 raft_consensus.cc:3060] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:09.240269 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61856us
I20260709 13:14:09.345516  2888 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:09.164147 28728 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:09.300540  2826 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:09.301272  2723 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:14:09.312008 28127 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.791s (SENT) [suppressed 54 similar messages]
I20260709 13:14:09.313830 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:09.314252 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:09.336859 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.354988 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.297310  2877 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:09.356184 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.357100 28155 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
W20260709 13:14:09.336954 28159 rpcz_store.cc:269] 0709 13:14:07.666314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.107718 (+441404us) service_pool.cc:224] Handling call
0709 13:14:09.279126 (+1171408us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b540033e354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:09.279272 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":802420480}
I20260709 13:14:09.360291 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.349241 28078 raft_consensus.cc:606] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:09.362957 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.364583 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:09.321264 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:09.331573 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.334779 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.330603  2900 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:09.376566 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.376765 28157 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:09.377406 28229 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:09.363822 28158 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:09.372349 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:09.337361  2870 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:09.382376 28155 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 19: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:09.354389  2723 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:09.381871 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.384817 28158 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:09.388967  2903 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.015s
W20260709 13:14:09.393113 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 233637us
I20260709 13:14:09.393472 28152 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:09.421481 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.421546 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:09.398530 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1329) took 1732 ms. Trace:
I20260709 13:14:09.400537 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.425129  2902 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:09.425345  2748 raft_consensus.cc:993] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:14:09.425850  2748 raft_consensus.cc:1081] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:14:09.373543  2873 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:09.413144 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:09.420084 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.403823 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.422675 28161 rpcz_store.cc:276] 0709 13:14:07.666564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.436590 (+770026us) service_pool.cc:224] Handling call
0709 13:14:09.249817 (+813227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:09.319049 (+ 69232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:09.398265 (+ 79216us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b54000a28d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:09.398455 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1869235072}
I20260709 13:14:09.424865 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.433894 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.435181 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:09.436007 28152 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:09.435374 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:14:09.436815 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 89550.3us
I20260709 13:14:09.437067 28232 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:14:09.441383 28227 raft_consensus.cc:1217] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Deduplicated request from leader. Original: 12.11->[15.12-15.12]   Dedup: 15.12->[]
I20260709 13:14:09.442163 28227 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:14:09.437479 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:09.447556 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.448616 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.449110 28232 raft_consensus.cc:1217] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Deduplicated request from leader. Original: 16.13->[17.14-19.15]   Dedup: 19.15->[]
I20260709 13:14:09.449450 28227 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:09.451195 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.453171 28232 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 19. Current term is 20. Ops: [17.14-19.15]
I20260709 13:14:09.454193 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:09.455569 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.457134  2898 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:09.457667  2592 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:09.458101  2592 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:09.458909  2592 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:09.459293 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.459888 28226 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:09.460114  2592 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:09.460054 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.461120 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.461326 28158 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:14:09.461787 28155 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 13.
I20260709 13:14:09.462704 28226 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:09.464879 28226 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:09.466424 28226 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39712
I20260709 13:14:09.466893 28226 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:09.467218 28226 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:09.472092 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.476112 28161 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:09.478091  2484 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:09.461800 28730 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:09.480042 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.482761 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.480757 28231 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:09.485186 28154 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:14:09.489319 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:09.492561  2893 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:09.493115  2893 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:09.493517  2893 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:09.495566 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:09.496187 28159 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:09.496892  2794 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:14:09.497254 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.497375  2794 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:09.498513  2794 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.499680 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:09.505604 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.506897 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.507409  2813 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:09.507913  2813 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.508276  2813 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:09.508667 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.520509 28154 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:09.508885  2898 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:09.525120 28234 raft_consensus.cc:2410] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:14:09.509773  2723 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.510303 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.511302  2877 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:09.513275 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.515410 28078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c654c99561449cb823642a1b85879ac"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
W20260709 13:14:09.516420 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.422510  2870 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.449903  2752 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:09.509459 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.527098 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:09.528070 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.514619 28226 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:09.533491 28159 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:14:09.533077 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:09.553653 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.554061 28226 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:09.553013 28161 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:09.540134 28152 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:09.560256 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:09.563189 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.566226 28235 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:09.568058 28231 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:09.569478 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.590415 28229 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:14:09.591259 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.569736 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.571004 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.567337  2799 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:09.597478  2799 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.599748  2799 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:09.575641 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.578764  2900 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.579524 28227 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:09.601032 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.572105  2866 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:09.602353 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.581303  2729 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:09.562162  2747 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 13:14:09.620435  2747 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 13:14:09.582780 27982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 194) took 16985 ms (client timeout 15000 ms). Trace:
I20260709 13:14:09.585314  2862 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:14:09.624045 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.547104  2588 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:14:09.584940 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 50897.6us
I20260709 13:14:09.624969 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.593429 28235 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 19: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:09.565117  2744 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:09.593832  2891 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:09.601117 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.603174  2866 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:09.604151 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:09.622124 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.622723 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.638410 28232 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:09.637239 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.640199 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.641371 28235 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:09.622979 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.620875  2747 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:09.629376  2723 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:09.647451 28157 raft_consensus.cc:1240] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 21. Ops: []
I20260709 13:14:09.600952  2403 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:09.635393 28227 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:09.651437 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.580587 28233 raft_consensus.cc:1275] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:14:09.636205 28086 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:14:09.652583 28233 pending_rounds.cc:85] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 9
I20260709 13:14:09.652928 28233 pending_rounds.cc:107] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:14:09.653175 28233 raft_consensus.cc:2889] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:09.650905  2675 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:09.634940  2891 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:09.660503 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.634280  2873 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 13:14:09.702968 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.668851  2891 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:09.671514  2794 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:09.674453 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.677593 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.718263 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:09.718675  2900 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:09.718905 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.719751 28234 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:14:09.678795 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.721745 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.722769 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.722810  2893 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.724179 28763 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:09.683564 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.725531 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.726106  2870 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:09.726641 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.685248 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.659093  2403 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.732657  2403 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:09.732877  2866 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.733517  2866 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:09.735397 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.627535  2872 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.741178  2872 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:09.621886 27982 rpcz_store.cc:269] 0709 13:13:52.597349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.597885 (+   536us) service_pool.cc:224] Handling call
0709 13:14:09.582682 (+16984797us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:04.141749 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:04.142493 (+   744us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:04.142548 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:14:04.142582 (+    34us) tablet.cc:662] Decoding operations
0709 13:14:04.144094 (+  1512us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:04.144187 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:04.144216 (+    29us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:14:04.192023 (+ 47807us) tablet.cc:701] Row locks acquired
0709 13:14:04.192080 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:14:04.192269 (+   189us) write_op.cc:270] Start()
0709 13:14:04.192430 (+   161us) write_op.cc:276] Timestamp: P: 1783602844192203 usec, L: 0
0709 13:14:04.192481 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:14:04.193309 (+   828us) log.cc:844] Serialized 6791 byte log entry
0709 13:14:04.820469 (+627160us) op_driver.cc:464] REPLICATION: finished
0709 13:14:06.020931 (+1200462us) write_op.cc:301] APPLY: starting
0709 13:14:06.021104 (+   173us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:06.383815 (+362711us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:06.383945 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:07.446392 (+1062447us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:07.446924 (+   532us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:14:07.446991 (+    67us) write_op.cc:312] APPLY: finished
0709 13:14:07.449397 (+  2406us) log.cc:844] Serialized 156 byte log entry
0709 13:14:07.707567 (+258170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:08.081224 (+373657us) write_op.cc:454] Released schema lock
0709 13:14:08.081345 (+   121us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:08.083700 (+  2355us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:08.083753 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":186381,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":1159351,"apply.run_cpu_time_us":54997,"apply.run_wall_time_us":2062758,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75425,"num_ops":17,"prepare.queue_time_us":185543,"prepare.run_cpu_time_us":60508,"prepare.run_wall_time_us":283251,"raft.queue_time_us":197010,"raft.run_cpu_time_us":21991,"raft.run_wall_time_us":516469,"replication_time_us":615460,"thread_start_us":1115210,"threads_started":5,"wal-append.queue_time_us":994289,"wal-append.run_cpu_time_us":13883,"wal-append.run_wall_time_us":2307000}]]}
W20260709 13:14:09.704174 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 181170us
I20260709 13:14:09.713033 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.688015 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.746234 28233 raft_consensus.cc:2376] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18. Re-sending same reply.
I20260709 13:14:09.749203 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:09.750130  2835 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:09.756139 28233 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:14:09.759325  2835 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:09.720438 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:09.769815 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.770481 28152 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
I20260709 13:14:09.738082  2753 raft_consensus.cc:993] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:14:09.761718 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:09.723856 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.765976 28235 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:09.801636 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:09.802256 28232 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:09.735836  2866 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:09.661541 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.792196 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:09.798502 28234 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:09.804807 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:09.811873 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:09.812000 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:09.812750 28161 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:09.814851 28077 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:14:09.815400 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:09.857131  2469 raft_consensus.cc:1081] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:14:09.706914  2675 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:09.857623 28760 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:14:10.013098 28086 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:14:09.985337  2864 log.cc:927] Time spent T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:14:09.930959  2898 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:10.012362 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 250693us
I20260709 13:14:10.014650  2898 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.972867  2824 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 13:14:10.017585  2824 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:10.016170 28226 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:10.018097  2824 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:10.018352 28230 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:39712
I20260709 13:14:09.857959  2799 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:10.018980 28230 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:10.019011  2824 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.019706 28230 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:09.915776 28755 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:10.020807 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:10.021653 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:09.960990 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 97693.5us
I20260709 13:14:10.028334 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:09.858778 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:10.023394  2675 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:09.902176 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:09.951433 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:09.954207  2835 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 13:14:09.892426  2753 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.133s
W20260709 13:14:10.030552 28050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.001s	sys 0.014s
W20260709 13:14:10.045570 28050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.002s	sys 0.015s
I20260709 13:14:10.037241 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:10.046154 28226 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.041221 28086 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:14:10.042088 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:10.031719 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 80387.9us
I20260709 13:14:09.805744  2802 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.051281 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:10.052299 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:10.047814 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 726618us
I20260709 13:14:10.049794 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:09.940222  2826 consensus_meta.cc:287] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.000s
I20260709 13:14:10.040172  2916 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:10.041673 28230 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:10.060597 28078 raft_consensus.cc:606] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:10.079686 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.126150 28078 raft_consensus.cc:614] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:10.126632 28227 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:10.126155 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 48534.8us
I20260709 13:14:10.083446 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:10.086609 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.027209  2893 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:10.090699 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:10.089161 28226 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
W20260709 13:14:10.040167 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.092110 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:10.044682  2782 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:10.091555  2742 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 13:14:10.090448  2816 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:10.122576  2898 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:10.139528 28153 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:10.074391  2824 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:10.128278 28234 raft_consensus.cc:2376] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18. Re-sending same reply.
I20260709 13:14:10.159054 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:10.162655 28230 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:10.183473 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 91474.1us
I20260709 13:14:10.201704 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:10.202791 28230 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:14:10.153023  2911 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:14:10.214879 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:10.215440 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 124811us
I20260709 13:14:10.145686  2916 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.216325 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:10.081513  2802 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:10.155256  2816 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:10.211359  2888 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.865s	user 0.011s	sys 0.003s
I20260709 13:14:10.231335 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:10.144924  2154 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:14:10.143409  2611 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.084s
I20260709 13:14:10.128296  2829 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.131s
I20260709 13:14:10.156632  2891 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.232947  2816 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:09.813570  2747 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.227111  2916 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:10.228411  2403 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:10.285233 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:10.286129 28232 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:10.287101 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:10.291409  2802 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:10.293466  2813 consensus_meta.cc:287] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.746s	user 0.007s	sys 0.004s
I20260709 13:14:10.231830  2757 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:10.280594 28758 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:10.288403 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:10.325771 28127 leader_election.cc:336] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.507s (SENT)
I20260709 13:14:10.293928  2813 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:10.326469 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.327901  2757 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:10.329778 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.331022  2757 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:10.333045 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.334349  2757 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:10.335996 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.337144  2757 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:10.299278  2721 consensus_queue.cc:1175] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "70996712c3c942468a6006602de6477f" responder_term: 25 status { last_received { term: 25 index: 18 } last_committed_idx: 17 last_received_current_leader { term: 25 index: 18 } }
I20260709 13:14:10.338910 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:10.341089 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:10.342046 28232 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 19: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:10.343060 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:10.351704 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:10.359580 28228 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:10.362577 28228 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:14:10.341208  2912 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:10.344434 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:10.377463 28226 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:10.378886 28226 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:10.380954 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45084.7us
W20260709 13:14:10.370499  2922 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:14:10.381807 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.381690 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:10.398757 28230 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:10.401003 28228 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:14:10.403831 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.370364  2912 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:10.422710 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 318875us
I20260709 13:14:10.427641 28226 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:10.430377 28226 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:10.434995 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 203952us
I20260709 13:14:10.422675  2829 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:10.422959  2757 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:10.452179 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:10.465077 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:10.452749  2917 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:14:10.453051  2866 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:14:10.462028  2921 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:14:10.476575 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:10.482378 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 100754us
W20260709 13:14:10.471310  2929 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:14:10.483708 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:10.488243 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:10.494894 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 91178.8us
I20260709 13:14:10.498936 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:10.500377 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 48313.7us
I20260709 13:14:10.475529  2866 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:10.493726  2924 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 13:14:10.508908  2928 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:14:10.509923  2943 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:10.473968  2912 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:10.520671  2940 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:10.521638  2940 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:10.522028  2940 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:10.522410  2940 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:10.503929  2813 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:10.506722  2933 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:10.507027  2866 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.535542 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43764 (request call id 56) took 1021 ms. Trace:
I20260709 13:14:10.560328 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:10.764021 28158 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:10.764302 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:10.610327 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.590544 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1330) took 2923 ms. Trace:
I20260709 13:14:10.763283 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:10.587472 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:10.767191 28156 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:10.766388 28153 rpcz_store.cc:276] 0709 13:14:07.666816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.535306 (+868490us) service_pool.cc:224] Handling call
0709 13:14:08.552308 (+ 17002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:10.155446 (+1603138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.590281 (+434835us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400342bd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:10.590472 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3649033600}
I20260709 13:14:10.630703 28082 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:10.588294 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:10.560401 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:10.683836 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 195626us
I20260709 13:14:10.769402 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.767371 28160 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:10.537042 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.560123 28157 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:14:10.771016 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1348) took 1005 ms. Trace:
W20260709 13:14:10.764910 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:10.764914 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.776826 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:10.536084 28078 rpcz_store.cc:276] 0709 13:14:09.514585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.515207 (+   622us) service_pool.cc:224] Handling call
0709 13:14:10.535280 (+1020073us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54003a93d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce671853 00007f59d363f60a 00007f59ccbc57d4 00007f59ccbc5762 00007f59ccbc5691 00007f59ccbc560c 00007f59ccbc48c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:10.535435 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1192068864}
I20260709 13:14:10.769519 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:10.781769 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.780094 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:10.785624 28157 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:10.786409 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:10.786540 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 555277us
I20260709 13:14:10.787428 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:10.771575 28159 rpcz_store.cc:276] 0709 13:14:09.765754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.765865 (+   111us) service_pool.cc:224] Handling call
0709 13:14:09.766151 (+   286us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:09.766493 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:09.766520 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:09.767083 (+   563us) log.cc:844] Serialized 52 byte log entry
0709 13:14:10.045177 (+278094us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:10.442617 (+397440us) log.cc:844] Serialized 14 byte log entry
0709 13:14:10.466112 (+ 23495us) log.cc:844] Serialized 14 byte log entry
0709 13:14:10.770575 (+304463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.770680 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:10.770741 (+    61us) raft_consensus.cc:1685] finished
0709 13:14:10.770761 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:10.770942 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57478,"prepare.queue_time_us":401418,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":1809,"raft.queue_time_us":82826,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":18984,"thread_start_us":887455,"threads_started":3,"wal-append.queue_time_us":471665}
W20260709 13:14:10.703202  2919 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.428s	user 0.002s	sys 0.000s
I20260709 13:14:10.799027 28758 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:14:10.799886 28127 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.801234 28228 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 22. Current term is 23. Ops: []
W20260709 13:14:10.653944  2925 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.289s	user 0.002s	sys 0.000s
I20260709 13:14:10.801571 28226 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:14:10.815771 28226 pending_rounds.cc:85] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 9
I20260709 13:14:10.816098 28226 pending_rounds.cc:107] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:14:10.591898  2893 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:10.587674  2895 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:10.651547  2910 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:10.593456  2891 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:10.545014  2744 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:10.697397  2799 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:10.763305  2747 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:10.590445  2792 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.792208  2154 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:10.558701  2933 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.785987  2868 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:14:10.647081  2943 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.174s
I20260709 13:14:10.590787  2816 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.647078  2829 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.178s
I20260709 13:14:10.803364  2912 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.802935  2837 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:10.536383  2916 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.801793  2588 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:10.792663  2888 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.819114 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:10.792701  2721 consensus_queue.cc:1048] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
W20260709 13:14:10.876925 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57881.9us
I20260709 13:14:10.871040 28158 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:10.823201 28157 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:10.820784 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.837311 28228 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:10.839881 28083 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:10.840205 28757 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:14:10.857456 28160 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:10.869549 28763 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:10.821269  2893 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:10.882220  2916 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:10.872651 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 85405.8us
I20260709 13:14:10.836655 27977 catalog_manager.cc:5697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:10.884656 28156 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:10.861730 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:10.822296 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.903077 28159 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
W20260709 13:14:10.885908 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 105804us
W20260709 13:14:10.834395 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 397843us
W20260709 13:14:10.892663 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 47739.8us
W20260709 13:14:10.892719 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72024.8us
I20260709 13:14:10.906354 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1351) took 1133 ms. Trace:
I20260709 13:14:10.816447 28226 raft_consensus.cc:2889] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:10.896418 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:10.857359  2723 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.908938  2951 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:10.907299 28154 rpcz_store.cc:276] 0709 13:14:09.772757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.809152 (+ 36395us) service_pool.cc:224] Handling call
0709 13:14:09.857623 (+ 48471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:09.858010 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:09.858040 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:10.022228 (+164188us) log.cc:844] Serialized 26 byte log entry
0709 13:14:10.379897 (+357669us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:10.503875 (+123978us) log.cc:844] Serialized 14 byte log entry
0709 13:14:10.906049 (+402174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.906127 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:10.906168 (+    41us) raft_consensus.cc:1685] finished
0709 13:14:10.906178 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:10.906297 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18979,"prepare.queue_time_us":84867,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":6309,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":9802,"thread_start_us":854928,"threads_started":3,"wal-append.queue_time_us":446921}
I20260709 13:14:10.909698  2951 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.833117  2744 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:10.908699  2723 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.837527  2911 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.870363  2588 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:10.831270  2945 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:10.898672  2888 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.830718  2910 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:10.955614  2910 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:10.956604  2910 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:10.833298  2799 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:10.962041  2799 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.963140  2799 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:10.963636  2871 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:10.964051  2871 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:10.831521  2942 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:10.919543  2744 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:10.830539  2895 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:10.966166  2895 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.967160  2895 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:10.891309  2675 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.925370  2403 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:10.969027  2675 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.875105  2802 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:10.971737  2802 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:10.974246  2802 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:10.891822  2782 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:10.924398 28050 leader_election.cc:400] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24.
W20260709 13:14:10.976275 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59571.7us
I20260709 13:14:10.977634 28050 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.980108 28050 leader_election.cc:400] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24.
W20260709 13:14:10.872498  2877 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.010s	sys 0.043s
I20260709 13:14:10.837677  2933 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:10.980957  2877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.010s	sys 0.044s
I20260709 13:14:10.981556  2877 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.981995  2877 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:10.982594  2933 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:10.983652  2592 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:10.984009  2877 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:10.984112  2592 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.897078  2846 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:10.984558  2592 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:10.901074  2753 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.833724  2792 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:10.994712 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 172487us
W20260709 13:14:10.994982 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48386.1us
I20260709 13:14:10.995556 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:10.996841 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:10.997287 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.946828 28763 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:10.929768  2945 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:10.930473  2826 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:10.921751  2794 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.027s
I20260709 13:14:10.926384  2588 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:10.986115 27977 catalog_manager.cc:5697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 12 to 17, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 17 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:10.949007  2951 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:10.951697  2888 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:10.827739  2866 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:10.901307  2835 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:10.998776 28763 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:11.004685 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:10.899710  2953 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:11.008862 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:10.984555  2877 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:11.011615 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.031338 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.012966 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.032331  2866 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.032626 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.029877  2835 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.033119  2792 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.007344  2942 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:11.006150  2945 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:11.033811 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.035065 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.014302 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.029053 28757 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:11.015445 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.017099 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.016677 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.018924 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:11.026209 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 78965.8us
I20260709 13:14:11.037374 28084 raft_consensus.cc:2368] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
I20260709 13:14:11.038168 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.039178 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.017550 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.000412  2933 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:11.033514  2835 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:10.930856  2154 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:14:11.042169 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.046528 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.042232 28081 raft_consensus.cc:2410] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:14:10.929668  2911 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.044528 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.049120 28757 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:14:11.049697 28757 raft_consensus.cc:606] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.001312  2611 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:14:11.045699 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.053668 28084 raft_consensus.cc:2368] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 22. Current term is 24.
I20260709 13:14:11.053584 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.054572 28228 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:11.046168 28085 raft_consensus.cc:2368] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
I20260709 13:14:11.056087 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:11.045095 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 134999us
I20260709 13:14:11.051702 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.005580  2484 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 13:14:11.052542 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.059230 28080 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:11.025926  2846 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:11.006677  2940 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.056836 28084 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:14:11.006116  2753 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.057714 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:11.060062 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.063339 28080 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:14:11.064077  2951 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:11.064524 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:11.065491  2951 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:11.065444 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:11.066018  2951 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:11.066418  2951 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:11.066555 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:11.006209  2893 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.055686  2829 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:11.047550  2792 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:11.061672  2866 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:14:11.071476 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.101511 28086 raft_consensus.cc:2368] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 23. Current term is 24.
I20260709 13:14:11.101022 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.071014 28080 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:11.103178 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.104671 28227 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:14:11.106086 28728 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.106019  2945 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.107306 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.107843 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:11.108610  2945 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.109436  2945 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:11.109568  2788 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:11.109896  2945 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:11.110181  2788 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:11.110555  2916 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:11.111038  2916 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:11.111133  2788 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.111931  2916 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.112913  2932 raft_consensus.cc:993] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: : Instructing follower 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:14:11.113384  2742 raft_consensus.cc:1081] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Signalling peer 11f10c12c03e47dcac88b050d4c1431b to start an election
I20260709 13:14:11.116608 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.090039 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.116761 28229 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:11.090799 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.099102 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:11.099102 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.117388 28759 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:11.122260 28229 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e"
dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:44070
I20260709 13:14:11.122896 28229 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:11.123245 28229 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:11.124155 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:11.124665 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:11.126358 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.127624 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:11.128312 28226 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:11.131626 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.126794 28085 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:11.135471 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.136188 28085 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:11.141993 28082 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:14:11.062297  2675 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:11.095331  2950 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:14:11.145313 28228 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:11.135773 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:11.147431 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.149238 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.154884 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:11.160212 28234 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:11.043275  2877 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.155459 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.081967  2893 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:11.082022  2952 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:11.161979 28085 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:11.097067  2946 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.081463  2723 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.127931  2933 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:11.115010  2945 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:11.166507 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.158627 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:11.087127  2866 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:11.167785 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:11.155695 28229 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.075491  2911 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:11.170615 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.170651 28085 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:11.153549  2954 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.199s	user 0.004s	sys 0.000s
I20260709 13:14:11.093981  2154 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:11.223723 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 156184us
I20260709 13:14:11.224045 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.152164  2592 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.132936  2826 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:11.145133  2792 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:11.133541  2835 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.153396  2829 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:11.177003  2942 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:11.074733  2951 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:11.100606  2892 consensus_queue.cc:1175] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "11f10c12c03e47dcac88b050d4c1431b" responder_term: 23 status { last_received { term: 22 index: 16 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:11.181577  2946 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.083648  2846 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.246313  2945 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.172163  2747 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:11.169677  2952 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:11.248685 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:11.248898  2952 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.095831 27977 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:11.171974  2484 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:11.250257  2484 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.250702  2484 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:11.251255  2752 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:11.251811  2752 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.168920  2957 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:14:11.169838  2744 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.195282 28226 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
W20260709 13:14:11.198297 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.198292  2588 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.254966  2744 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.232175  2802 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.024s
I20260709 13:14:11.252463 27977 catalog_manager.cc:5697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:11.256431  2588 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.202944 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.257505 28086 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 19: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:11.246021  2592 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:11.242398  2867 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 13:14:11.258342 28086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:11.257205 27977 catalog_manager.cc:5697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 8 to 13, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 13 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:11.214687 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:11.259634 28161 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:11.217358 28229 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.219923 28228 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:11.230571  2826 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.261305  2826 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:11.219969 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.225005 28227 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:11.225468 28153 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:11.222585 28085 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.265075 28085 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:11.267340 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:11.267021  2871 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.268395  2871 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:11.229146 28160 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:11.229393 28158 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:11.269649  2945 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.224505 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:11.235199 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.270602 28756 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:11.270850  2816 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.215772  2721 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.214603  2866 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:11.271597  2866 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:11.271517  2721 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.234298 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:11.273888  2721 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.243050 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.244172  2907 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:11.274824  2948 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:11.275038  2907 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:11.275497  2907 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:11.275938  2907 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:11.277103  2952 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:11.275516  2403 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:11.233111  2753 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:11.279606  2403 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:11.280364 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.246049 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.241477 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:11.254523 28764 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:14:11.235317 28763 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.281121 28083 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:14:11.282603 28763 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:11.259001 28086 raft_consensus.cc:606] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:11.227412 28082 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:11.222834  2900 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:11.245174 28081 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:11.260510 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:11.281473 28233 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:14:11.286800 28761 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:11.283308 28086 raft_consensus.cc:614] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:11.273926  2942 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:11.209897 28079 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:11.287639 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:11.287798 28049 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:11.288298 28079 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:11.286274 28083 raft_consensus.cc:1240] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:14:11.285986  2900 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.292569  2900 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:11.292022  2942 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.294835  2942 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.295440 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:11.296406 28086 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:11.242259  2940 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.288451  2948 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:11.306025  2948 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.280686  2403 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.314874  2403 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:11.316252  2946 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.314023 28233 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:11.245242  2951 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.287156 28761 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:11.318150 28761 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.319422 28761 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:11.326512 28077 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:11.335444 28156 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 26.
I20260709 13:14:11.341816 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:11.328403 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 85466.3us
W20260709 13:14:11.358649 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:11.359391 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73728.8us
I20260709 13:14:11.360527 28052 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:14:11.352939 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.312397  2877 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.363909 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.367403 28763 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:14:11.368520 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.369062 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.368503 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.374441 28085 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:11.375449 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.371814 28077 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 26: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:11.376354 28085 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:11.265919  2484 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.372802 28756 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:14:11.379284 28759 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:14:11.376530 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 152130us
I20260709 13:14:11.380147 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:11.381523 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:11.381978 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.382380 28760 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:11.382652 28755 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 25.
I20260709 13:14:11.383482 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:11.380635 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:11.387598 27977 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:11.390264  2799 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:11.390806  2799 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.391197  2799 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:11.394248  2940 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:11.394724  2940 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:11.395130  2940 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:11.396672  2835 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:11.397153 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.397928  2956 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:11.398543  2956 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.400780  2588 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:11.401247  2588 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.401618  2588 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:11.402281  2951 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:11.402745  2951 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:11.403139  2951 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:11.404932  2911 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:14:11.407368 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.419988 28233 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:11.401548 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:11.409104 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.410193 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.354070  2592 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:11.416098 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:11.430243 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:11.417564 28228 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:11.425510 28077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:11.359087  2892 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.429279 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:11.381587 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.425954 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:11.433696 28085 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 26.
I20260709 13:14:11.373749  2953 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:11.418550  2794 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:14:11.438747 28761 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:11.370831  2154 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:11.454661 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:11.453636 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:11.423179  2829 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:11.428646  2592 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.374027  2891 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:14:11.377075  2782 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:14:11.484057 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:11.499468 28156 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:11.442121  2945 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:11.442332  2757 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:14:11.447700  2792 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:14:11.503165 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48631us
I20260709 13:14:11.441941  2744 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
W20260709 13:14:11.503788 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 50294.6us
I20260709 13:14:11.509539 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.518821 28156 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:11.517299 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:11.527076 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:11.445827  2892 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.474838  2956 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.499377  2592 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:11.471432  2900 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.457372 27977 catalog_manager.cc:5697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 9 to 15, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 15 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:11.527940  2888 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:11.534269 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:11.514439  2957 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:14:11.537004 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.526176  2945 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:11.508836  2877 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:11.510977  2757 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.470172  2154 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.452765  2953 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.558396 28156 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:11.586889 28158 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:11.536908  2824 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:11.605983 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.606678 28084 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:14:11.541791  2956 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:14:11.592307  2962 log.cc:927] Time spent T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:14:11.616379 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:11.617147 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 125112us
I20260709 13:14:11.614821 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.601814  2918 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:11.590294  2888 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:11.586551  2753 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:11.630234 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:11.631374 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:11.632241 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.634397 28229 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:11.602250 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:11.543421  2903 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:14:11.598922  2757 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:11.626230 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.644168 28152 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:11.609848  2890 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 13:14:11.603093  2154 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.629459 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:11.623826 28153 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 13:14:11.657786 28161 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:14:11.636291 28160 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 23: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:11.602243  2949 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:11.663133 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:11.603688  2957 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:11.652595 28084 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:11.624571  2918 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:11.618773  2824 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.649191  2959 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:11.666538 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.668063 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:11.672295 28161 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:11.673795 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 142245us
I20260709 13:14:11.674605 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:11.681870 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:11.671522  2154 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:11.733073  2154 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.735361  2829 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:11.739374 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:11.698825  2964 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:14:11.697901 27977 catalog_manager.cc:5697] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:11.699723  2918 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.643640  2753 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.667155  2945 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.705343 28161 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:14:11.705935  2957 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.758527 27977 catalog_manager.cc:5697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:11.761067 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:11.761019  2826 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.762202 28161 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 23: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:11.763051 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:11.763588 28161 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
W20260709 13:14:11.765638  2968 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:14:11.767123  2956 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:11.767999  2826 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.707998 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.711385  2866 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.712832  2949 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:14:11.713608  2969 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:14:11.715485  2907 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.717738  2892 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.680971  2916 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:11.698051 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.772717  2900 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.792883  2975 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:11.793345  2975 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:11.794278  2975 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.794911 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:11.795233  2918 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:11.648904  2744 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:14:11.799751 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:11.775602 28226 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:11.800935 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:11.750414 28154 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
W20260709 13:14:11.701967  2893 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.000s
I20260709 13:14:11.654286  2971 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:14:11.752961 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:11.788641 28160 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:11.789266 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.813674 28161 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 19: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:11.807953 28757 raft_consensus.cc:2393] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 19: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:11.783360 28234 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:11.823179 28160 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
W20260709 13:14:11.823973 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.787379 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:11.748477  2959 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:11.679831  2888 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.837666  2888 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:11.838080  2935 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:14:11.839577  2949 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:11.758320  2752 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.758577  2484 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:11.800995 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 73862.6us
I20260709 13:14:11.840685  2949 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.841662 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:11.787106 28127 leader_election.cc:336] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.791s (SENT)
I20260709 13:14:11.842489 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:11.843251  2868 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:11.805613 28077 raft_consensus.cc:606] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:11.843611 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 56535.6us
I20260709 13:14:11.844027 28077 raft_consensus.cc:614] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:14:11.844334 28127 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.855s (SENT)
I20260709 13:14:11.843808  2868 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:11.806718 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:11.847185  2868 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.847915 28079 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 23: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:11.844604  2611 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:14:11.830297 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.779243  2753 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:11.808753  2871 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.003s
I20260709 13:14:11.842542 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:11.753742  2154 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.806020 28759 raft_consensus.cc:2368] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24.
I20260709 13:14:11.746680 28152 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:11.804980  2744 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:11.787206  2907 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:11.831770 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:11.846093 28127 leader_election.cc:336] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.883s (SENT)
I20260709 13:14:11.783114  2866 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.859247 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:11.867040 28226 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:14:11.871201 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:11.744452  2588 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.862489 28049 leader_election.cc:400] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24.
I20260709 13:14:11.873047 28049 leader_election.cc:403] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:14:11.788327  2916 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:11.807796  2976 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:11.865710 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.874430 28161 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:14:11.874778 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.875562 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.851156  2829 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:11.876397 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.799996  2799 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.878477 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:11.866561 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 194748us
W20260709 13:14:11.880671 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2602533629999471s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
I20260709 13:14:11.881433 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:11.884667 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2566533629999412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
I20260709 13:14:11.787247  2951 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.863518  2877 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:11.886484 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:11.887671 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.864763  2744 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:11.869846  2824 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:11.899727 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1774533630000406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 13:14:11.908993 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1702533630000289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 13:14:11.909359 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1018533629999183s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 13:14:11.909642 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0964533629999096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
W20260709 13:14:11.909919 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0937533629999052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
W20260709 13:14:11.910178 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0478533630000584s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18320000000000003s for a response
W20260709 13:14:11.910434 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0307533630000307s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
I20260709 13:14:11.911684 28158 raft_consensus.cc:1240] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:11.880051  2866 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.879542  2953 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.760577 27977 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:11.901691 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:11.884320  2871 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:11.904247 28152 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: [15.13-16.14]
I20260709 13:14:11.892026  2943 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:11.888885  2976 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:11.929823  2951 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:11.932500  2592 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:11.935693  2592 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.935155  2976 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.884439  2940 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:11.942970 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53302.7us
I20260709 13:14:11.946781 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:11.947500 28759 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:14:11.919581  2953 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:11.950459 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.965425 28759 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:11.966171 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:11.968216 28227 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:11.917716  2866 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:11.923894  2971 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:14:11.971580 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.971154 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:11.986160 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:12.001753 28127 leader_election.cc:336] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.071s (SENT)
I20260709 13:14:12.003615 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:11.977807  2799 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:11.959805  2959 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:12.008788  2782 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:12.010164 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:12.011966 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:14:11.984344  2952 consensus_meta.cc:287] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.693s	user 0.007s	sys 0.005s
W20260709 13:14:11.974371  2982 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:14:12.009279  2782 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:12.018733 28127 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.169s (SENT)
I20260709 13:14:11.971393  2877 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:11.972434  2890 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:11.979532  2588 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:12.024950 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 54693.7us
I20260709 13:14:12.026265 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:11.966949  2953 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:12.021943  2611 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:14:11.943380  2916 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:12.024117  2799 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:12.050329 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:12.050535 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:12.051018 28229 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:12.023607 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:12.023087  2742 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:12.035838 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:12.047077 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.048033  2975 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:12.058732 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:12.059404 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:12.031312 28050 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:12.062234 28199 leader_election.cc:400] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
I20260709 13:14:12.071646 28233 raft_consensus.cc:1275] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:12.107923 28199 leader_election.cc:403] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:14:12.035254  2943 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:12.103272 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 79777.3us
I20260709 13:14:12.113018 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:12.109416 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 76765.8us
I20260709 13:14:12.116582 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:12.110658 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48618.1us
I20260709 13:14:12.118041 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:12.120041 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:12.120651 28228 raft_consensus.cc:2376] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21. Re-sending same reply.
I20260709 13:14:12.120998 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:12.122983 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 91787.2us
I20260709 13:14:12.125253 28050 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:12.113507  2890 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:12.121222  2942 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:12.121516  2868 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
W20260709 13:14:12.179005 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53870.6us
I20260709 13:14:12.179697  2983 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:12.114446  2611 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:12.205506  2611 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:12.111826  2742 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:12.207163  2742 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:12.132035  2403 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:12.104447  2829 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:12.191520 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 73578us
I20260709 13:14:12.128600  2792 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:12.037420  2757 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:12.301648 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.132170  2932 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.219928 28050 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:12.186482 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 65546.2us
I20260709 13:14:12.128783  2816 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
W20260709 13:14:12.341701 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.178123  2943 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:12.132190  2994 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:12.112123  2782 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:12.221112  2403 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:12.344144 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 124324us
I20260709 13:14:12.302685  2940 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:12.301887  2890 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:12.186292  2877 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:12.361094  2862 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.354130  2932 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:12.339677  2982 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:14:12.357748  2983 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:14:12.374687 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 105677us
I20260709 13:14:12.383065  2403 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:12.414918 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:12.394760 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:12.396397 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:12.401255 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3578) took 1182 ms. Trace:
I20260709 13:14:12.417138 28157 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:12.417309 28234 rpcz_store.cc:276] 0709 13:14:11.218974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.220841 (+  1867us) service_pool.cc:224] Handling call
0709 13:14:11.221130 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:11.815342 (+594212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:12.401014 (+585672us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b54000fb9d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:12.401185 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1292557184}
I20260709 13:14:12.362892  2826 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.417132 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:12.215574 27977 catalog_manager.cc:5697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:12.405157 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:12.358142  3003 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:14:12.434544 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:12.410764 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:12.448537 28160 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 23: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:12.428277 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:12.395037 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:12.302150  2829 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:12.419567 28079 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:12.415076 28077 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:14:12.451499 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.359373  2799 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:14:12.390566  2866 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:12.452987 28078 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:14:12.421124  2794 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.440229 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:12.482095 28078 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:12.456945 28155 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 23: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:12.460119 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:12.462944 28152 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:12.482844 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:12.455516 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:12.483642 28084 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:12.484205 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:12.484428 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:12.485216 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:12.490693 28157 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:12.451006  2983 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:12.477108  2987 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:14:12.509044 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:12.452837  2953 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:12.521950 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:12.486696  3004 log.cc:927] Time spent T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:14:12.530593 28152 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:12.532111 28084 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
W20260709 13:14:12.525090 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 119028us
I20260709 13:14:12.524541 28082 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 24: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:12.534044 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:12.413957  2862 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:12.506933  2794 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:12.537969 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:12.469359  2826 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:12.540664 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:12.553221  2918 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.757s	user 0.011s	sys 0.001s
W20260709 13:14:12.614008  3001 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:14:12.831019 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.547480  2952 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:12.620071  2862 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:12.613767  2983 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:12.888875 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 350945us
W20260709 13:14:12.888276 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 347653us
I20260709 13:14:12.876955  2943 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.918866 28199 leader_election.cc:400] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 22. Current term is 24.
I20260709 13:14:12.921550 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:12.924067 28085 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:12.918663 28050 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:12.926970 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.922555 28234 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:12.892704  2994 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:12.900835  2953 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.924078  2918 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:12.932539  2757 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:12.937093  2799 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:12.957742  2799 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:12.951750 27977 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:12.958171 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:12.960006  2799 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:12.960203 28199 leader_election.cc:400] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
W20260709 13:14:12.946028  2888 consensus_meta.cc:287] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 1.108s	user 0.011s	sys 0.002s
I20260709 13:14:12.960475 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3277) took 1335 ms. Trace:
I20260709 13:14:12.960578 28199 leader_election.cc:403] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:12.973767 28158 rpcz_store.cc:276] 0709 13:14:11.624972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.625858 (+   886us) service_pool.cc:224] Handling call
0709 13:14:11.626133 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:11.912147 (+286014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:12.960215 (+1048068us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54003449d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:12.960391 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656245632}
I20260709 13:14:12.959388 27977 catalog_manager.cc:5697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 6 to 13, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 13 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:12.963032 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:12.976994 28161 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:12.975821 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:12.522976  2890 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:12.981909  2826 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:12.983050  2829 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:12.955426  2953 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:12.986240  2989 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.014s	user 0.005s	sys 0.000s
I20260709 13:14:12.986469  2953 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:12.988633  2984 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.018s	user 0.006s	sys 0.013s
W20260709 13:14:12.989316  2871 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.063s	user 0.012s	sys 0.000s
I20260709 13:14:12.991122  2871 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 23. Current term is 24.
I20260709 13:14:12.992261  2588 consensus_queue.cc:1059] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:12.993577  2588 raft_consensus.cc:3055] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:12.994175  2588 raft_consensus.cc:740] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:12.995550  2588 consensus_queue.cc:260] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:12.997293  2403 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.000s
I20260709 13:14:12.998001  2403 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:12.935758  2943 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:13.002269  2992 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.029s	user 0.008s	sys 0.008s
I20260709 13:14:12.966234  2974 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:13.004046 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1550) took 1296 ms. Trace:
I20260709 13:14:12.942209  2747 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.010210 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1550) took 1157 ms. Trace:
I20260709 13:14:12.977214  2952 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
I20260709 13:14:12.964540 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:13.097707 28153 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:13.018889 28199 leader_election.cc:400] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 23. Current term is 24.
I20260709 13:14:13.049911 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:12.960654  2868 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:13.094072 28160 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:13.094766 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3275) took 1491 ms. Trace:
I20260709 13:14:13.093894 28227 rpcz_store.cc:276] 0709 13:14:11.707894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.708009 (+   115us) service_pool.cc:224] Handling call
0709 13:14:11.708268 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:11.969562 (+261294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:13.003671 (+1034109us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400305554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:13.003953 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597369216}
I20260709 13:14:13.094921 28761 rpcz_store.cc:276] 0709 13:14:11.853082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.853245 (+   163us) service_pool.cc:224] Handling call
0709 13:14:11.853606 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:11.855476 (+  1870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:11.855509 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:11.863176 (+  7667us) log.cc:844] Serialized 26 byte log entry
0709 13:14:11.961205 (+ 98029us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:11.961247 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:11.961318 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:12.985467 (+1024149us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:13.008776 (+ 23309us) raft_consensus.cc:1685] finished
0709 13:14:13.008809 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:13.010146 (+  1337us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1482,"raft.queue_time_us":5478,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":2304,"spinlock_wait_cycles":68427136,"thread_start_us":61441,"threads_started":1,"wal-append.queue_time_us":95070}
I20260709 13:14:13.100747 28154 rpcz_store.cc:276] 0709 13:14:11.603396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.604351 (+   955us) service_pool.cc:224] Handling call
0709 13:14:13.093755 (+1489404us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000802d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:13.094698 (+   943us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274517248}
I20260709 13:14:13.017613  2994 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:13.014843  2829 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.038920  2932 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.071437  2826 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.097007  2588 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:13.071713  2799 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.071797  2757 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.099393  2868 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:13.109110  2792 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:13.107326  2974 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:13.102609  2747 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.130932  2868 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:13.112905  2829 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.129173  2826 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.132911  2799 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.212383 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:13.211004 27977 catalog_manager.cc:5697] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 7 to 14, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 14 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:13.217016 27977 catalog_manager.cc:5697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 11 to 13, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 13 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:13.225234 27977 catalog_manager.cc:5697] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:13.231985 28051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.017s	sys 0.025s
W20260709 13:14:13.232575 28051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.018s	sys 0.026s
I20260709 13:14:13.022182 28077 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
W20260709 13:14:13.102334 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 52546.9us
I20260709 13:14:13.236620 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:13.244194 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 225291us
W20260709 13:14:13.143102 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 211335us
I20260709 13:14:13.266350 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:13.283027  2403 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.132652  2757 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.171262  2974 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.301334  2943 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.302315  2943 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.303264  2757 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.303105  2943 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.304001  2757 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.172210  2918 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:13.294987  2829 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.295872  2747 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:13.315362 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.315879  2799 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.299881  2826 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:13.326469 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 348001us
I20260709 13:14:13.330240  2888 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:13.331126  2888 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.157106  2794 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.334790  2943 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.299953  2721 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:13.336974  2792 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:13.337430 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56848.5us
I20260709 13:14:13.338053  2792 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.316154  2757 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.340070 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:13.347057  3018 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:13.347355  3019 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.347723  3018 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.348219  3018 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:13.352483  2813 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:13.354553  2757 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.354416  2611 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:13.361586  2403 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:13.380851  2877 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:14:13.381548  2403 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.382792  2877 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:13.383473  2877 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:13.363823 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:13.365558  2994 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:13.365557  2918 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.315902  2895 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.380149  2611 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.393190 28153 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:13.415938  2994 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:13.422235 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1374) took 1009 ms. Trace:
I20260709 13:14:13.422899 28153 rpcz_store.cc:276] 0709 13:14:12.413224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.964304 (+551080us) service_pool.cc:224] Handling call
0709 13:14:13.422081 (+457777us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:13.405247  2943 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.397682 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:13.425259  2943 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.405553 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.427717 28078 raft_consensus.cc:2410] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 25 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:14:13.405083  2792 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.428907  2792 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.429625  2792 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.431219  2611 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:13.410969  2403 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:13.432387  2792 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:13.434901  2960 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:14:13.419582 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55986.8us
W20260709 13:14:13.376163 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 110189us
I20260709 13:14:13.402325 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1547) took 1777 ms. Trace:
I20260709 13:14:13.435531  2960 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:13.423699  2895 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.442549 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:13.442866  2960 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.443565 28226 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:13.443385  2877 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.348268  3019 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.449236  2895 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:13.408814 27977 catalog_manager.cc:5697] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:13.401125  2794 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.453311  2794 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:13.403678 28758 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 1240) took 1639 ms (client timeout 2078 ms). Trace:
I20260709 13:14:13.457567  2802 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:13.416823  2721 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.408775  2799 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.395711  2813 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.462466 28227 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:13.467998 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:13.469583  2943 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.469714 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:13.471549  2868 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.472676  2877 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:13.474668 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1368) took 1063 ms. Trace:
W20260709 13:14:13.475500 28758 rpcz_store.cc:269] 0709 13:14:11.763963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.764108 (+   145us) service_pool.cc:224] Handling call
0709 13:14:13.403216 (+1639108us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54007693d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:13.403448 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3318454016}
I20260709 13:14:13.474805 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:13.477023  2802 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:13.484355 28160 rpcz_store.cc:276] 0709 13:14:12.411620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.017815 (+606195us) service_pool.cc:224] Handling call
0709 13:14:13.093887 (+ 76072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:13.099656 (+  5769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:13.474234 (+374578us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b54000802d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:13.474478 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174012800}
I20260709 13:14:13.490873  2802 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.489550 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.498806  2794 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.499691  2799 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.501974  2721 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.502657  2799 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.503332 28155 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:13.423285  2744 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.506181 28154 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:13.510991 28086 consensus_meta.cc:287] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.029s	user 0.011s	sys 0.014s
I20260709 13:14:13.533821 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:13.536496  3019 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.506527  2960 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:13.543334 28086 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:13.544164 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2859) took 1109 ms. Trace:
I20260709 13:14:13.544934 28161 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:13.545917 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.547986 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.545010 28231 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:13.504647  2721 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:13.505998 27977 catalog_manager.cc:5697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 9 to 16, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 16 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:13.544447 28086 rpcz_store.cc:276] 0709 13:14:12.434425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.439831 (+  5406us) service_pool.cc:224] Handling call
0709 13:14:13.544022 (+1104191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:13.562510 28155 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:13.510871  2813 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:13.545956  2802 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:13.549052  2888 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.564826  2752 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:13.565707 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.566267  2799 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.566730 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.567271 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.567988 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:13.568912 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.569319 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:13.569619  2744 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:13.440369 28755 rpcz_store.cc:276] 0709 13:14:11.625049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.626309 (+  1260us) service_pool.cc:224] Handling call
0709 13:14:11.651143 (+ 24834us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:11.652398 (+  1255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:11.652439 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:11.656907 (+  4468us) log.cc:844] Serialized 26 byte log entry
0709 13:14:11.851642 (+194735us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:11.851695 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:13.401475 (+1549780us) spinlock_profiling.cc:243] Waited 876 us on lock 0x7b54004371d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:13.401598 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:13.401664 (+    66us) raft_consensus.cc:1685] finished
0709 13:14:13.401678 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:13.402040 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2835,"raft.queue_time_us":2202,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":3490,"spinlock_wait_cycles":2015104,"thread_start_us":185083,"threads_started":1,"wal-append.queue_time_us":226302,"wal-append.run_cpu_time_us":1826,"wal-append.run_wall_time_us":1078337}
I20260709 13:14:13.570811 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:13.583580 28758 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:14:13.571772 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.583976 28084 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 21: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:13.586294 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.584942 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:13.588799  2752 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.581264  2867 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.600143 28159 raft_consensus.cc:2368] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 26. Current term is 27.
I20260709 13:14:13.600915 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.613710 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:13.617617 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1372) took 1205 ms. Trace:
I20260709 13:14:13.618086 28159 rpcz_store.cc:276] 0709 13:14:12.412650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.492825 (+1080175us) service_pool.cc:224] Handling call
0709 13:14:13.617561 (+124736us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:13.622594  3033 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:13.623332  3033 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:13.588968  2799 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.621165 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:13.625978 28155 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 14.
I20260709 13:14:13.631809 28078 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:13.633066  3029 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.270s	user 0.002s	sys 0.000s
I20260709 13:14:13.633737  2868 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:13.629864 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 162757us
W20260709 13:14:13.638355 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.638495  3028 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:14:13.639534 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.631588  2867 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.643608  2867 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.641345 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:13.652544 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 67974.1us
I20260709 13:14:13.654497 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:13.656255  2910 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.659124  2910 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.660964 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:13.661629  2910 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.661809 28227 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:13.663007 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.663931 28080 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:14:13.666456 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:13.670228 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.670958 28085 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:13.670992  2799 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.672189 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.672991 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.675225 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:13.678663 28127 leader_election.cc:336] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.354s (SENT)
I20260709 13:14:13.681300 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:13.683212  3013 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:13.683759  3013 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.684275  3013 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:13.687350  2890 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:13.688092  2890 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.688884  2782 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:13.689316 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:13.690369  2892 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:13.690833  2892 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.691197  2892 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:13.693099  2862 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:14:13.694285 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 154746us
I20260709 13:14:13.694638 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.693002  2867 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.696244  2867 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.698278  2866 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.699158  2866 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.701274 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:13.702167 28233 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:13.657013  2868 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:13.657624  2948 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.658358  2753 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:13.707599 28228 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:14:13.710088 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55768.9us
I20260709 13:14:13.684343  3010 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:14:13.719949 28154 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:13.720705 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:13.721035 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:13.721817  2866 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:13.722826  2952 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:13.723204 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.722636  2948 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.723894 28761 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:13.723697  2952 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.683640  2956 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:13.725093 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.725250  2956 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.726325  2952 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:13.727603  3010 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:13.728096  3010 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:13.721884 28230 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:13.730235 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.734136 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:13.708979  2910 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 13:14:13.739499 28127 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.375s (SENT)
I20260709 13:14:13.740829 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:13.742195  2942 consensus_meta.cc:287] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.419s	user 0.019s	sys 0.004s
I20260709 13:14:13.742414 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:13.742760  2942 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:13.736092  2910 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:13.743960 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.744251  2910 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:13.730274  2994 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:13.745491  2994 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:13.745811 28759 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:13.745632  2910 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.747303  2910 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:13.743824 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:13.747664 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.748520 28759 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:13.746765  2994 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.743813 28153 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:13.749538 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:13.754698  2753 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.755388  2753 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.757761  2753 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.758818 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.740923  3028 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:13.761219  3028 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.761699  3028 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:13.749089  2752 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:13.730135  2867 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.767907  3013 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.774595 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.728693  2956 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:13.624711  3033 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.778736 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:13.779623 28226 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:14:13.758524  3018 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.741477  2890 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:13.783330 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49531.1us
I20260709 13:14:13.783481  2948 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:13.787576 28153 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:13.801757 28758 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 27.
I20260709 13:14:13.804369 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:13.816366 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:13.819339 28755 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 25: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:13.819834 28154 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:13.839548  2888 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:13.789206  2867 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:14:13.843039  2867 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:13.843461  2867 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:13.729606  3010 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.847081 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:13.831579 28127 leader_election.cc:336] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.410s (SENT)
W20260709 13:14:13.836450 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 87491.8us
I20260709 13:14:13.845702 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:13.850986 28077 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:13.829857  3019 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:13.853046  3019 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:13.853482  3019 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:13.857811 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:13.830842  2975 consensus_meta.cc:287] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 1.517s	user 0.011s	sys 0.011s
I20260709 13:14:13.844482  2867 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.861608 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:13.889952 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44330.1us
I20260709 13:14:13.893594  2799 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:13.899952 28049 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:13.871856  2890 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:13.884691  3013 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:13.914507 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 44769.4us
I20260709 13:14:13.916091  2753 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:13.923583 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:13.924471 28227 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:13.923583 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:13.923800  3010 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:13.925915 28155 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:13.925642 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:13.913239  2888 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 13:14:13.933184  2799 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:13.933872  3033 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.934059  2799 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:13.934651  3033 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.933085 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.939548  3018 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:13.940126  2799 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.943642 28228 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:13.888633  2752 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:14:13.948227 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:13.952275 28233 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 13:14:13.949589 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 122066us
I20260709 13:14:13.950166 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:13.951253  3033 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:13.957660 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:13.913239  2890 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:13.967360 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 44930.2us
I20260709 13:14:13.924494  3024 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:13.973148  3024 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:13.973644  3024 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:13.974900 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:13.976892 28153 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:14:13.979120 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:13.970594 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:13.987661  2403 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:13.988123  2403 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:13.988544  2403 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:13.930137  3028 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:13.990360 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:13.991016 28077 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:13.991149  3028 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:13.991609 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:13.968586  2890 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:13.993427 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:13.923653  2753 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.991196 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:14.004050 28757 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 26. Current term is 27. Ops: []
I20260709 13:14:14.006435 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:14.007715  2867 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:14.008461 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.957681  2942 consensus_queue.cc:1048] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:14:14.008652 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:13.994360 28125 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:14.020682 28154 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:14.026119 28227 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:14.036705 28077 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:14.037266 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:14.040598 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46323.3us
I20260709 13:14:14.033028 28730 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:14.034446 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:14.011042  2753 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:14.011030  2792 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:14.037606 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:14.047835 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:14.053108 28079 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:14:14.083388 28756 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:14.056056  2871 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.056223 28158 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:14.058861  2721 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:14.048947  2942 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:14.061141  2983 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:14:14.093922 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:14.094624 28758 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:14.082770  2948 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.095414  2983 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:14:14.096047  2983 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:14.095898  2948 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:14.096421 28161 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 26. Current term is 27. Ops: []
I20260709 13:14:14.062071  2862 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:14.097178  2862 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:14.095495 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:14.098081 28153 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:14.098294  2862 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.099750  2943 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:14.064177 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:14.100307  2943 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.066004 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:14.066129 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:14.067791 28234 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:14.067925 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:14.069242 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:14.073045 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.076932  2975 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:14.079427 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:14.023011  2799 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:14.084404 28233 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:14.086567 28081 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:14.093874 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:14.089403  2942 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:14.061587 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:14.108294 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:14.105638 28156 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:14.112632 28086 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:14:14.081264  2792 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.116329 28154 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:14.032166  3028 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:14:14.110164  2942 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:14.107167  2799 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.137204 28230 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:14:14.125922 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:14.137610  2782 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.004s
W20260709 13:14:14.141314 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73563.8us
I20260709 13:14:14.141139  2983 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.141801  2782 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:14.142283  2782 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:14.142457  2983 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:14.137709  2721 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.133390 28084 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:14.146451  3030 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:14.143517  2782 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.147142  2721 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.147701  2721 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:14.132376 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:14.090343  2892 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.149605 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:14.150894  2892 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:14.130080  2975 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.120944  2792 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:14.151998 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:14:14.153234 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.153914  2975 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.154150  2868 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:14.133711  3018 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:14.155102  2868 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:14.155484  3018 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:14.153950  2877 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:14.137651 28230 pending_rounds.cc:85] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 11
I20260709 13:14:14.156554  3018 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.157486 28230 pending_rounds.cc:107] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:14:14.157871 28230 raft_consensus.cc:2889] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:14.146911  3030 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:14.158790  3030 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:14.159431 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:14.145018 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:14.148175 28084 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:14.158885 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:14.141443 28233 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:14.160445 28155 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:14.140164 28158 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:14.159145  2943 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.161754 28233 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:14.171370 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.133980  3013 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.182250 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:14.184785 28230 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:14.138402  2799 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:14.183188 28085 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:14.147163  2935 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
W20260709 13:14:14.222921 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 60928.1us
I20260709 13:14:14.148166  2721 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:14.190991 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.172782  2943 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:14.235098 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.164319  2782 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:14.116859  2871 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.223003 28755 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:14.222829  2971 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:14.227569  2994 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:14.243042 28050 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:14.229185  2910 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:14.237696 28233 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:14.259768 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 100933us
I20260709 13:14:14.273221  2802 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:14.275401 28233 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:14.236498  2890 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:14.278223 28079 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2862) took 1788 ms (client timeout 1758 ms). Trace:
W20260709 13:14:14.281564 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.281975  2721 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.285684 28230 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:14.273859  2802 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:14.224447  3013 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:14.293537 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50608.7us
W20260709 13:14:14.278904 28079 rpcz_store.cc:269] 0709 13:14:12.483755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.483923 (+   168us) service_pool.cc:224] Handling call
0709 13:14:14.241459 (+1757536us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54003a98d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:14.271724 (+ 30265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3370504576}
I20260709 13:14:14.294948 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:14.301312  3033 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:14:14.304157 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.318867  2935 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:14.249924  2943 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.263084  2971 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.320371 28085 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:14.329713 28083 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:14.304482  2948 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:14.338522 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:14.287098  2910 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.276346  3024 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.323758  2935 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:14.295665  3019 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.330487  3015 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:14.356520 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 86776.6us
I20260709 13:14:14.383055 28082 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:14.383601 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 247396us
I20260709 13:14:14.388779 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:14.358280  2935 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:14.383901 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 265623us
I20260709 13:14:14.397812  2592 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:14.405897 28081 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
W20260709 13:14:14.406172  3047 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.220s	user 0.005s	sys 0.000s
I20260709 13:14:14.396351  2403 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.408828  2911 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.409518  2911 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.410002  2403 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.391113  2802 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:14.421397  3046 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:14:14.425138  2721 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.425843  2721 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:14.429633 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:14.430510 28080 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:14.433736  2871 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.438566  2943 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.439416  2943 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:14.446465 28078 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:14.446638  2971 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.449404  2890 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:14.451326  2910 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:14.453269 28083 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2858) took 2019 ms (client timeout 1655 ms). Trace:
W20260709 13:14:14.452638 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 51971.7us
W20260709 13:14:14.459410 28083 rpcz_store.cc:269] 0709 13:14:12.434102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.434269 (+   167us) service_pool.cc:224] Handling call
0709 13:14:14.453008 (+2018739us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54001ad454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:14.453174 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3275121920}
I20260709 13:14:14.365306  3015 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:14.454994  2952 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:14.459925  3010 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.002s
I20260709 13:14:14.454150  3013 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:14.487746 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.491973 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:14.487130 28233 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
W20260709 13:14:14.492700 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 88469.5us
I20260709 13:14:14.493485 28233 pending_rounds.cc:85] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 8
I20260709 13:14:14.493824 28233 pending_rounds.cc:107] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:14:14.494194 28233 raft_consensus.cc:2889] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:14.440085  2943 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:14.535449 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 43577.7us
I20260709 13:14:14.536402 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:14.540136 28127 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.916s (SENT)
I20260709 13:14:14.541426  2877 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:14.542326  2877 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:14.542742  2877 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.543103  2877 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:14.547412 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 202730us
W20260709 13:14:14.548071 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 386646us
I20260709 13:14:14.483367  2792 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:14.557381  3050 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:14.532264  2952 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.560674  2952 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.565699 28728 leader_election.cc:400] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 26. Current term is 27.
I20260709 13:14:14.565562  3010 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:14.566217 28728 leader_election.cc:403] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:14.567377  3052 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 26. Current term is 27.
I20260709 13:14:14.565493 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:14.568678  3052 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:14.569456  3052 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:14.569996 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:14.570734  3052 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.572937  3033 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:14.580672 28159 raft_consensus.cc:2393] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 21: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:14.583721 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:14.584580 28082 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:14.585886  3010 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:14.586409  3010 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.586782  2952 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:14.588853 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:14.586805  3010 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:14.592861  3019 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:14.593078 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:14.593840 28082 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:14.594980  2943 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:14.596480 28233 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:14.597653 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.599997 28231 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:14.600560 28231 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:14.601768 28231 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:14.603310 28231 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:14.605222 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:14.605547  2911 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.606209  2911 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:14.492259  2994 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:14:14.608837  2994 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:14.609270  2994 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:14.603165 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:14.610250  2994 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:14.611639  2994 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:14.614439  2721 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:14.615226  2753 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.012s
I20260709 13:14:14.615720  2753 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:14.617177 28077 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 21: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:14.558132  3050 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.621690 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:14.623970 28227 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:14:14.630157 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:14.630934 28227 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:14.600096  2942 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:14.640065 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:14.640615 28231 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:14.640879 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:14.641093 28230 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:14.641881 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:14.642776 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 292854us
W20260709 13:14:14.640611 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 100542us
I20260709 13:14:14.644802 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:14.645414 28083 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:14:14.637814 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 248712us
I20260709 13:14:14.649480 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:14.564287  3030 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:14.646312 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:14.658167 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:14.665349 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:14.666893 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:14.667966 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:14.650628 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:14.649029 28125 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:14.668789 28758 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 25: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:14.671746 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:14.712263 28227 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
W20260709 13:14:14.713171 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:14.712814 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 63896.4us
W20260709 13:14:14.715903 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 396058us
I20260709 13:14:14.722342 28759 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:14.724164 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:14.637537  2975 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:14:14.730187 28052 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:14.736721 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:14.737283 28125 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:14.646854  3010 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.717793  3048 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:14.734285  3051 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:14.759898 28125 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:14.763055 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:14.779377 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49278.3us
W20260709 13:14:14.805141 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 268809us
I20260709 13:14:14.926889 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:14.809057 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 126865us
W20260709 13:14:14.762853  2799 consensus_meta.cc:287] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.002s
I20260709 13:14:14.877562 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:14.953277 28759 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:14.878484 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:14.879062 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 163964us
I20260709 13:14:14.759617  2744 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.761866  2721 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:14:14.920832 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:14.923683 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:14.748307  2952 raft_consensus.cc:476] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Not starting pre-election -- already a leader
I20260709 13:14:14.925997 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:14.935585  3024 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:14.765990  3051 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.003460  3051 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:15.003743  3010 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:15.004717 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:15.005367  3050 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:15.005481 28764 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:14.997948  2792 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:14.759316  3048 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:14.937523 28155 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:14.751334  2895 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:14:14.830852 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.923558 28083 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 25.
I20260709 13:14:14.942215 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:14.943848 28156 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:14:14.945277  2753 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:14.947906  2935 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.949296 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:14.950953 28154 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:14:14.951341 28158 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:14.953243 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:14.878468 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 118679us
I20260709 13:14:14.955848 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:14.937476  3060 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:14:14.981159 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:14.989629 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49866.3us
I20260709 13:14:14.998538 28227 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 13:14:15.021080 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:14:14.998966 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72217.8us
I20260709 13:14:15.001595 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.001901 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.002100  2948 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:15.001685 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:15.009632 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:14.970283 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.022711 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:15.014667 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61549.1us
I20260709 13:14:15.025090 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:15.026705 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.033308 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.034510 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.037624 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.037736 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:15.040918 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.043155 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.044217 28235 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:15.045100 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:15.045512  3063 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:14:15.045324 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:15.046316 28152 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:14:15.028425  2948 consensus_queue.cc:1243] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:14:15.016474  2895 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:15.053156 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.054524 28228 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:15.054039  2895 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.055516 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.055729  2895 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:15.010704  2983 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:15.054086 28232 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:15.056721 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.063153 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:15.032297  3057 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.268s	user 0.000s	sys 0.002s
I20260709 13:14:15.067862 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:15.081605 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.067883 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.068553 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.082878 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:15.069468 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:15.076611  2782 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:15.077258  2871 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:15.027760  3050 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:15.028497  3060 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.032815  2792 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:15.016927  3048 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:15.094816 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:15.036368  2744 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:14.972905  2799 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:15.080487 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.070310  2948 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:15.084690 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:15.085997 28231 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:15.087798 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 65185.7us
I20260709 13:14:15.090269 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.090833 28156 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:15.057034  2983 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.092566 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:15.093016 28233 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 14: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:15.093549 28227 raft_consensus.cc:2393] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 21: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:15.084327 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:15.060660  2935 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:15.096095 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.102154 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:15.087021  2943 consensus_queue.cc:1059] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:14:15.105993 28235 raft_consensus.cc:2410] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:14:15.107475 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.057075  3059 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:14:15.095160  3060 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:15.076676  2975 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:14:15.116724 28083 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:14:15.117177 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:15.092530  2403 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:15.089504  2952 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:15.129045 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.129372  2952 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.109742  2983 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:15.106575 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:15.121403 28160 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 17: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:15.122670 28157 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:15.108385  3050 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:15.108897  3030 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:15.112614 28082 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:15.123827 28228 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:15.096910  2792 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:15.125155 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.106786  2948 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.185469 28156 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 25.
I20260709 13:14:15.120836 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.187826 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:15.188601 28227 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:15.132270 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.190616 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:15.132992 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.148437  3071 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.197952  3071 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.139405  3062 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:15.201015  3062 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.201560  3062 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:15.150470  3064 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:14:15.151859 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:15.123044  2802 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:15.155848  2877 consensus_meta.cc:287] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.612s	user 0.011s	sys 0.000s
I20260709 13:14:15.157639  2744 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:15.158385  2971 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:15.207371  2971 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:15.208858  3071 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:15.162266 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:15.183003 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.161525 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:15.184432 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:15.209600 28155 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:15.122164  2942 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:15.210194 28161 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:15.208377  2971 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.212512  2971 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:15.184902 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:15.193691  2948 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:15.217716  2918 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.218317  2918 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.216609 28157 pending_rounds.cc:85] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 11
I20260709 13:14:15.230250 28233 pending_rounds.cc:85] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 14
I20260709 13:14:15.245059 28233 pending_rounds.cc:107] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:14:15.245525 28233 raft_consensus.cc:2889] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:15.231086 28157 pending_rounds.cc:107] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:14:15.246814 28157 raft_consensus.cc:2889] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:15.224433 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 467378us
I20260709 13:14:15.252874 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:15.255935 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:15.184144  2792 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:15.256547 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 228376us
I20260709 13:14:15.268709 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:15.277237 28755 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:15.276983 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3302) took 1226 ms. Trace:
I20260709 13:14:15.302806 28728 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:15.220702  2918 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:15.327955 28763 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:15.319805 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:15.319068 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:15.223134  2782 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:15.228341  3033 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.221860  2403 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:15.351609 28227 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:15.343634 28086 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
W20260709 13:14:15.335582 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 234729us
I20260709 13:14:15.221972  3082 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:15.325798 28160 rpcz_store.cc:276] 0709 13:14:14.024491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.024636 (+   145us) service_pool.cc:224] Handling call
0709 13:14:15.250976 (+1226340us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000a37d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:15.251209 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2390928256}
I20260709 13:14:15.326783 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:15.322108  2802 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.354990 28728 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:15.215474  2877 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.322407  3018 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:15.221982  3015 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:15.325188 28127 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.880s (SENT) [suppressed 13 similar messages]
I20260709 13:14:15.350805  3070 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:15.269918  2721 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:15.372862 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:15.375655  3070 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:15.376120  3070 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:15.331598  2942 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.377067  2942 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.379313  2942 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:15.374680 28078 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:15.376654 28124 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:15.384378 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:15.385356 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.386574 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:15.334338  3051 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.351086  2952 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:15.399257 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:15.399623 28228 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 21: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:15.392095 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:15.398214 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.351383  2403 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:15.227523  2948 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.386068 28232 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
W20260709 13:14:15.404004 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 135435us
I20260709 13:14:15.404138 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:15.407874 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:15.408622 28078 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:14:15.412185 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 87046.9us
I20260709 13:14:15.379869  2942 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:15.380131  2867 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:15.374521  3082 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:15.355453  3033 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.371402  2802 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.399904  3018 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 25 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:15.408128  3059 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:15.393002  3015 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:14:15.427742  2952 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:15.421905  2753 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:15.432911  2799 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:15.452504 28086 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:15.433281  2935 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:15.436769  3055 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:15.373970  3050 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.455588 28084 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:14:15.418270  3030 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:14:15.460001 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:15.463521 28085 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:15.464242 28152 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:15.425609  2744 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:14:15.441286  2867 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:15.481694 28161 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:15.485293  2799 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:15.486514  2799 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.490191  2799 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:15.491130  2799 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:15.498453  2799 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.501734  2799 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:15.447539  2975 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:14:15.467061  2942 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.471858  2935 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:15.479347  2910 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.462817  2753 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.503540  2952 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:15.502772  3073 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:15.510351  3028 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:15.524824  2942 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.530305  2952 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.594007  3101 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:14:15.593871 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 140213us
I20260709 13:14:15.596557 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:15.477514  3090 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:14:15.618999  3059 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.506573  3048 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.637202 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:15.628551 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:15.649461 28079 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:15.520450  2799 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.627586 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:15.629420 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:15.630954  2802 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:15.654294 28125 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:15.633431  2935 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.528105  2910 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.525669  3086 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:14:15.621326 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 210408us
I20260709 13:14:15.637441  2867 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.513571  3082 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.538106  3093 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.207s
I20260709 13:14:15.641283  3090 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:15.676823 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:15.641974 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:15.519709  3085 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.250s
I20260709 13:14:15.538102  3033 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:15.541256  2943 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.508013  2721 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.665248 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:15.665380 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:15.665747 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:15.688128 28152 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 25: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:15.666939 28077 raft_consensus.cc:1240] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:15.619972  3059 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.667961 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:15.668402 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:15.621232  2877 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:15.669641  3086 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:15.675319 28156 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:14:15.540990  3018 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.679828 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:15.669979  2994 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 1.058s	user 0.014s	sys 0.001s
I20260709 13:14:15.697912  2994 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:15.698423  2994 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:15.699204 28227 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:14:15.699594 28227 pending_rounds.cc:85] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 12
I20260709 13:14:15.699860 28227 pending_rounds.cc:107] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:14:15.700269 28227 raft_consensus.cc:2889] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:15.677608 28160 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
W20260709 13:14:15.653378  3060 consensus_meta.cc:287] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.002s
I20260709 13:14:15.689764 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:15.702262 28159 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:15.687789 28755 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:15.691262 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:15.691804 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.694000 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:15.689522  2943 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:15.695909  2895 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.001s
I20260709 13:14:15.696535  2942 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:15.697109 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:15.712999 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:15.721051 28079 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:15.666010  2799 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.724638 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:15.724256 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:15.725359 28156 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:14:15.725886 28152 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:15.724145 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:15.728961 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:15.728575 28082 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:15.727005 28229 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:15.698952  2752 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:15.732584 28158 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:15.732861 28758 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:15.735028 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:15.699783  3086 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:15.736498 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:15.723901  3101 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:15.723847  3073 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:15.743280  2952 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:15.743665  2877 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:15.734927  3015 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:15.689332  3090 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:15.823004 28158 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:15.906978 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:15.868170  2952 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:14:15.902427  3100 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:14:15.921603  3113 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:15.933887  3062 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.732s	user 0.014s	sys 0.000s
W20260709 13:14:15.934007  3070 consensus_meta.cc:287] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.000s
W20260709 13:14:15.854003  2983 consensus_meta.cc:287] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.697s	user 0.007s	sys 0.003s
I20260709 13:14:15.731509  3105 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.183s
I20260709 13:14:15.740383  3091 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:15.933880  3048 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:15.844231  3058 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.113s
W20260709 13:14:15.958513 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 223544us
I20260709 13:14:15.712427  3060 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:15.739060  2753 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:15.996512 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 250000us
I20260709 13:14:16.050932 28156 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:16.051098 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:15.905365  2994 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.087400 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:15.985661  2799 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:15.999241  2868 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.094673 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1570) took 1450 ms. Trace:
I20260709 13:14:16.002453  3015 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.119668 28152 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
W20260709 13:14:16.141409 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44009.8us
I20260709 13:14:16.086632  2983 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.125617 28757 rpcz_store.cc:276] 0709 13:14:14.644711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.644893 (+   182us) service_pool.cc:224] Handling call
0709 13:14:14.667625 (+ 22732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:14.924614 (+256989us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:14:14.924652 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:14.977915 (+ 53263us) log.cc:844] Serialized 26 byte log entry
0709 13:14:15.509555 (+531640us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:15.509597 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.509691 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.094422 (+584731us) raft_consensus.cc:1685] finished
0709 13:14:16.094445 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.094609 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":180553,"raft.queue_time_us":39031,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":7926,"thread_start_us":610899,"threads_started":2,"wal-append.queue_time_us":617423}
I20260709 13:14:16.090163  3070 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.099084  2952 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:16.152941 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:16.153663 28080 raft_consensus.cc:2368] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 15. Current term is 16.
I20260709 13:14:16.085234  3062 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.139930  3015 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:16.137269  3091 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:15.997793  3101 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.170253 28086 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 19: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:16.139159  2868 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.167068  3059 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:16.205466  3102 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
I20260709 13:14:16.148005  3098 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
I20260709 13:14:16.220454  3090 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.220664 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1537) took 1096 ms. Trace:
I20260709 13:14:16.181072  2943 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.207394 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:16.221168 28081 rpcz_store.cc:276] 0709 13:14:15.124597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.124779 (+   182us) service_pool.cc:224] Handling call
0709 13:14:15.142116 (+ 17337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:15.142323 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:15.142348 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:15.185043 (+ 42695us) log.cc:844] Serialized 26 byte log entry
0709 13:14:15.653664 (+468621us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:15.653723 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.653813 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.220414 (+566601us) raft_consensus.cc:1685] finished
0709 13:14:16.220435 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.220579 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":335287,"threads_started":1,"wal-append.queue_time_us":492455}
I20260709 13:14:16.222184  2868 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:16.225283  3051 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:16.225436  3060 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:16.228407  2983 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:16.208360  3091 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.183995  2952 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.093015  3113 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.215912 28159 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:16.146438  3112 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:16.292150 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 467949us
W20260709 13:14:16.293161 28235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.007s	sys 0.004s
I20260709 13:14:16.283233  2799 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:16.292968  2792 consensus_meta.cc:287] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.957s	user 0.013s	sys 0.001s
W20260709 13:14:16.308997 28235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.008s	sys 0.005s
I20260709 13:14:16.249521  3112 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.319505 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2729) took 1310 ms. Trace:
W20260709 13:14:16.322398  3050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.038s	sys 0.038s
W20260709 13:14:16.323519  2892 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.003s	sys 0.029s
I20260709 13:14:16.329466 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.312845  2799 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.316340  2792 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.330054 28235 rpcz_store.cc:276] 0709 13:14:15.003789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.132005 (+128216us) service_pool.cc:224] Handling call
0709 13:14:15.132336 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:15.374762 (+242426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:15.374793 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:15.393191 (+ 18398us) log.cc:844] Serialized 26 byte log entry
0709 13:14:15.651030 (+257839us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:15.675817 (+ 24787us) log.cc:844] Serialized 14 byte log entry
0709 13:14:16.313892 (+638075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.314000 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.314065 (+    65us) raft_consensus.cc:1685] finished
0709 13:14:16.314085 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.314268 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17621,"prepare.queue_time_us":23444,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":143329,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":2280,"spinlock_wait_cycles":13184,"thread_start_us":1041048,"threads_started":3,"wal-append.queue_time_us":288001}
W20260709 13:14:16.332384  3086 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.001s
W20260709 13:14:16.333217  3050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.038s	sys 0.038s
W20260709 13:14:16.335260  2892 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.004s	sys 0.029s
I20260709 13:14:16.367810 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.369491 28755 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:16.373425  3113 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:16.375288 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3630) took 1245 ms. Trace:
I20260709 13:14:16.375751 28228 rpcz_store.cc:276] 0709 13:14:15.129989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.184648 (+ 54659us) service_pool.cc:224] Handling call
0709 13:14:16.375035 (+1190387us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54003b4554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:16.375193 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419785984}
W20260709 13:14:16.360595  3120 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
W20260709 13:14:16.359899  3122 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:14:16.384121  3062 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:16.386404 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 98296.6us
W20260709 13:14:16.388360 28127 leader_election.cc:336] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.907s (SENT)
I20260709 13:14:16.394899 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.400352 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:16.401880 28083 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:14:16.362183  3112 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.407579 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.356521  3024 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:16.386193  2799 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.357410  3086 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.402540  3092 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.407272  3070 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:16.409261  2802 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:16.406159  2753 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:16.423861 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:16.427943 28083 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:16.416471  3106 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:16.406939  3085 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:14:16.413930  3114 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:16.437376 28082 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:16.442679 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:16.444337 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:14:16.444630 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 56324.6us
I20260709 13:14:16.433475  3092 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.422377  2892 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.022s
I20260709 13:14:16.449074  3113 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:16.498250 28083 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:16.499372 28728 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:16.496980  2802 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 13:14:16.501041  2802 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:16.501407  2802 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:16.501544  3098 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:16.502206  3098 raft_consensus.cc:697] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:16.503186 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.504174  2799 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:16.505126  2799 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:16.505514  3128 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:14:16.505715  2799 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:16.506282  2799 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:16.506600 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:16.507161  2799 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.508132 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:16.505839  3111 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.509756 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:16.509339  3111 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.510429 28233 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:14:16.512050  3111 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:16.516884 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:16.519910  3104 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 13:14:16.494088 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:16.509405 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:16.511772  3015 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.513490  3086 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.513867  3113 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.513952 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.516052  2799 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:16.509373  2792 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:16.529762  3085 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:16.542822  3085 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.543179  3085 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:16.536141  3091 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:16.542335  3092 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:16.532244  3113 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:16.489382  3106 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:14:16.554330 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.556193 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.564461  3015 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.565824  3091 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:16.570701 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2739) took 1486 ms. Trace:
I20260709 13:14:16.571380 28085 rpcz_store.cc:276] 0709 13:14:15.084330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.084452 (+   122us) service_pool.cc:224] Handling call
0709 13:14:16.570452 (+1486000us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54002684d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:16.570627 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618604928}
I20260709 13:14:16.503185  3098 consensus_queue.cc:237] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.577214  3091 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:16.577958  3091 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.579806 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:16.580786 28761 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 15: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:14:16.581018  3129 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.132s	user 0.003s	sys 0.000s
I20260709 13:14:16.583150  3091 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.585354 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:16.586905 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:16.587612  3117 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:16.587808 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:16.588680 28757 raft_consensus.cc:2410] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:14:16.502231  2802 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.588935 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:16.589527  2802 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:16.593942 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.595336 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:16.588275  3117 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.599504 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:16.601393  3101 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:16.601643  2994 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:16.602118  3101 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:16.602267  2994 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.603178  3101 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.597069  3024 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:16.599711  2952 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:16.604703  2994 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.605723  2952 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.612133  2952 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.615003  3117 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.605921  3081 raft_consensus.cc:3055] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:16.617730  3081 raft_consensus.cc:740] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:16.618661  3081 consensus_queue.cc:260] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.619625 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.621567 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:16.621917 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:16.622947 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:16.623687 28086 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 21: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:16.608877  2877 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:14:16.625550  2877 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:16.626092 28079 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:16.608480  2403 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.627664 28079 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:14:16.626118  2877 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:16.628669  2877 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:16.629848 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:16.630503 28078 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:14:16.632891  3070 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:14:16.635226  2994 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:16.635691  2994 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:16.636080  2994 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:16.612165 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2711) took 1667 ms. Trace:
I20260709 13:14:16.619760 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 1579) took 1377 ms. Trace:
I20260709 13:14:16.640379 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:16.640261 28758 rpcz_store.cc:276] 0709 13:14:15.242492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.242717 (+   225us) service_pool.cc:224] Handling call
0709 13:14:16.619549 (+1376832us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b5400654354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:16.619688 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":918756224}
I20260709 13:14:16.641671 28228 raft_consensus.cc:2393] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:16.620347  3117 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:16.642207 28079 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:14:16.643821 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:16.646605 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:16.649473 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:16.652702 28086 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:14:16.598066 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.632460  3113 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.654675 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:16.669823 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 75923.6us
I20260709 13:14:16.673492 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:16.675328 28233 raft_consensus.cc:2410] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:14:16.677076 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:16.678393 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:16.675843 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:16.600962  3091 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:14:16.683521 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 85509.9us
I20260709 13:14:16.685752  2792 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:16.686781 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:16.623703  2952 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:16.687071  2948 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.688561 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:16.697444 28235 raft_consensus.cc:2410] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:14:16.671906 28756 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:16.638237 28226 rpcz_store.cc:276] 0709 13:14:14.944281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.972053 (+ 27772us) service_pool.cc:224] Handling call
0709 13:14:15.009064 (+ 37011us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:15.092323 (+ 83259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:15.092361 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:15.108498 (+ 16137us) log.cc:844] Serialized 26 byte log entry
0709 13:14:15.185631 (+ 77133us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:15.237370 (+ 51739us) log.cc:844] Serialized 14 byte log entry
0709 13:14:15.371941 (+134571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.372026 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.370616 (+998590us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b18003580c0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a522 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:14:16.609617 (+239001us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b18003580c0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
0709 13:14:16.611908 (+  2291us) raft_consensus.cc:1685] finished
0709 13:14:16.611937 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.612100 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84511,"prepare.queue_time_us":53588,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":41243,"raft.queue_time_us":257198,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":46983,"spinlock_wait_cycles":103268224,"thread_start_us":190535,"threads_started":2,"wal-append.queue_time_us":82319}
I20260709 13:14:16.695741 28081 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
W20260709 13:14:16.701960 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.703083 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.666981  3117 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:16.705324 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:16.708909 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:16.672523  2942 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:16.712246  3082 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:16.731592  3126 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:14:16.733860  3116 consensus_queue.cc:1059] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:14:16.734973  3131 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:14:16.736027  2910 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:16.737644  2592 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:16.738253  2592 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:16.739385  2592 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.740646 28086 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:16.742280 28086 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:14:16.745535  3074 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.389s	user 0.005s	sys 0.005s
I20260709 13:14:16.712479  3112 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:14:16.748484  3112 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:16.748941  3112 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:16.713819 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:16.727900 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:16.750296 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:16.750957 28082 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:16.707013 28124 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:16.762939 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49202.8us
I20260709 13:14:16.767280  3126 raft_consensus.cc:3055] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:16.767720  3126 raft_consensus.cc:740] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:16.768550 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 61606.5us
I20260709 13:14:16.769089  3131 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:16.769490  3131 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:16.770321  3131 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:16.770655 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:16.771467  3131 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:16.729411 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:16.773178  3065 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:16.773527 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 1544) took 1399 ms. Trace:
I20260709 13:14:16.773764  3065 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.773904 28084 rpcz_store.cc:276] 0709 13:14:15.374346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.384110 (+  9764us) service_pool.cc:224] Handling call
0709 13:14:16.773359 (+1389249us) spinlock_profiling.cc:243] Waited 70.3 ms on lock 0x7b54002daad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:16.773473 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161620864}
I20260709 13:14:16.774633  2792 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:16.775267  2792 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:14:16.775817 28080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2737) took 1693 ms (client timeout 2205 ms). Trace:
W20260709 13:14:16.776257 28080 rpcz_store.cc:269] 0709 13:14:15.082390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.089163 (+  6773us) service_pool.cc:224] Handling call
0709 13:14:16.775636 (+1686473us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003a98d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:16.775762 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2389370624}
I20260709 13:14:16.769908 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:16.765457 28228 raft_consensus.cc:2410] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:14:16.771435 28756 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:16.713717  3093 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:16.712811  3117 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:16.780084 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:16.780400 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:16.781481 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:16.779349  3101 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:16.797705  3101 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.727207  3019 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:14:16.706358  3085 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:16.843878 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.776302  2792 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:16.892616 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 112290us
I20260709 13:14:16.901060 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:16.903952 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3627) took 2154 ms. Trace:
I20260709 13:14:16.796576  2592 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:16.766125  3048 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:16.904448 28230 rpcz_store.cc:276] 0709 13:14:14.652613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.652754 (+   141us) service_pool.cc:224] Handling call
0709 13:14:14.678243 (+ 25489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:14.678597 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.678625 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:14.679084 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:14:15.012894 (+333810us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:15.012933 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.012998 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.744998 (+1732000us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b24001df498. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4109c4 00007f59cb40eacb 00007f59cb5c9dc9 00007f59cb5c9a55 00007f59cb5c909d 00007f59ce1fb10e 00007f59ce21288c 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221
0709 13:14:16.753087 (+  8089us) raft_consensus.cc:1685] finished
0709 13:14:16.753117 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.807340 (+ 54223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":8439,"spinlock_wait_cycles":25288576,"thread_start_us":263033,"threads_started":1,"wal-append.queue_time_us":454030}
I20260709 13:14:16.910393 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:16.783078  3082 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:16.969087  3082 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.969553  3082 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:16.970083 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 59776.9us
I20260709 13:14:16.970255  3091 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:16.972488  3019 consensus_queue.cc:1243] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:14:16.794174  2877 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:16.975540 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:16.976609 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:16.977272 28153 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:16.980208 28155 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:14:16.980540 28161 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:14:16.986123 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.792881  3086 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:17.024832 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.024933 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.028163 28155 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:16.891796  2942 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:16.898212  2867 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:17.048139 28156 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:14:17.048000 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.048998 28159 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 17: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:17.050257 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:17.051692 28161 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:14:16.909433  2721 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:17.069778 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 83941.9us
I20260709 13:14:17.054281 28155 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 21: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:16.902519  3065 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:17.044102  2942 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:16.977520  3093 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:16.908843  3101 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:17.079936 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.081223 28156 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 17: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:17.024426  3116 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:17.087199 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:17.088543 28159 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 19: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:17.084884 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:17.089607 28155 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:17.055008  2867 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:17.108816 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:17.110024 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:17.111189 28233 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:17.083267  2799 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.003s
I20260709 13:14:17.078073  2942 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:16.768610  3126 consensus_queue.cc:260] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.126665 28161 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:17.132025 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.133000 28159 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 14.
I20260709 13:14:17.135612 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:17.139603 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:17.116441  2867 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:17.025894  3112 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.159433 28153 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 15.
I20260709 13:14:17.170109 28153 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:14:17.121413  3092 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:17.174351 28158 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:17.143187  2952 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:17.170409  3126 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:17.149482  3072 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 13:14:17.094901  3116 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.179296  3092 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.201138  3112 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:17.204027  2877 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:17.248957 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 428557us
W20260709 13:14:17.254103 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.209800  2952 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.220494  3117 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:17.278643  3148 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.214s	user 0.000s	sys 0.002s
I20260709 13:14:17.290666  3085 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:17.364625 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 91530.2us
I20260709 13:14:17.378701 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:14:17.382148  3155 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:14:17.382390  2952 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:17.383893  3117 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:17.385062  3116 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:17.388612  3085 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:17.418632  3085 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:17.419268  3085 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:17.395601 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 1256) took 1670 ms. Trace:
I20260709 13:14:17.421231 28755 rpcz_store.cc:276] 0709 13:14:15.725370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.726006 (+   636us) service_pool.cc:224] Handling call
0709 13:14:15.734234 (+  8228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:16.370026 (+635792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.395191 (+1025165us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400472cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:17.395504 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460438400}
W20260709 13:14:17.403856  3143 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.642s	user 0.003s	sys 0.006s
I20260709 13:14:17.393390  3165 raft_consensus.cc:2955] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Committing config change with OpId 18.15: config changed from index 11 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:17.424631  2994 consensus_meta.cc:287] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.788s	user 0.008s	sys 0.004s
I20260709 13:14:17.432292  2994 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.410678  3084 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:17.433338  3084 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.406320  3073 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:17.411563  2942 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.404165  3116 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:17.435426  3084 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:17.436894  3116 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.437160 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:17.438061 28081 raft_consensus.cc:2393] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:14:17.438465  3131 consensus_meta.cc:287] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.666s	user 0.011s	sys 0.003s
I20260709 13:14:17.439201  3116 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:17.439414 28230 raft_consensus.cc:2410] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:14:17.443046 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:17.443480  2403 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:17.443768 28153 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:14:17.445199 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3635) took 2082 ms. Trace:
I20260709 13:14:17.445365  2892 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:17.445600 28227 rpcz_store.cc:276] 0709 13:14:15.362436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.362587 (+   151us) service_pool.cc:224] Handling call
0709 13:14:15.389709 (+ 27122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.712760 (+323051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.444969 (+1732209us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54000de8d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:17.445146 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710850816}
I20260709 13:14:17.445930  2892 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.434618  2994 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:17.361239  3082 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.448390  2892 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:17.448967  3082 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:17.406042  3052 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.471577  2892 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.435185  3073 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.472036  3052 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.472846 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:17.473465 28159 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:14:17.474696  3052 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:17.475123  3073 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:17.429795  3117 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.455658  2592 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:17.475772  3117 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.475978  2592 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.477941  3117 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:17.455848  3024 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:17.453436 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.456952 28155 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:17.457104 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:17.478533  3024 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:17.479013 28078 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:17.480037  3144 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.685s	user 0.000s	sys 0.008s
I20260709 13:14:17.472112  2892 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.481038  2592 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:17.482573  3080 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.482728  3060 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:17.483140  3080 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.483372  3060 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.457926 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:17.485342 28079 raft_consensus.cc:2410] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:14:17.461128 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:17.486102  3058 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:17.461906 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:17.487263 28080 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:17.462999 28084 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 13:14:17.488339 28084 pending_rounds.cc:85] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 18
I20260709 13:14:17.488620 28084 pending_rounds.cc:107] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 13:14:17.488914 28084 raft_consensus.cc:2889] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:17.489444  3060 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:17.463896 28082 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:17.411563 28234 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:14:17.497252 28234 pending_rounds.cc:85] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 13
I20260709 13:14:17.497581 28234 pending_rounds.cc:107] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:14:17.497826 28234 raft_consensus.cc:2889] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:17.503125  3113 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:17.483021  2892 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:17.507432  3092 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:14:17.509696  3080 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:17.513792 28078 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:17.514866  2892 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:17.515555  2892 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.518934  3081 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:17.431738 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.454423  2867 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:17.466926 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 88299.8us
W20260709 13:14:17.467872  3109 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.370s	user 0.003s	sys 0.010s
I20260709 13:14:17.413331 28226 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:14:17.470487  3071 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:17.471012  2942 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:17.453332 28235 raft_consensus.cc:2368] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:14:17.486578  3058 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:17.526315 28049 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:17.541062 28081 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:17.527241 28229 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:14:17.529601 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:14:17.545131 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.547390 28159 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:17.549091 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:17.549109 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 248440us
I20260709 13:14:17.548710 28756 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 14. Current term is 15. Ops: [14.13-14.13]
W20260709 13:14:17.550967 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.466095 28153 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:17.553948 28156 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:17.554047 28229 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:17.557634 28756 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:17.560169 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:17.560657 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.561316 28763 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 27.
I20260709 13:14:17.531606  2892 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:17.531651  3071 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:17.559967 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.569528 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:17.570366 28156 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 21: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:17.575026 28153 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 14.
I20260709 13:14:17.577276 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.580998 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1397) took 2407 ms. Trace:
I20260709 13:14:17.571372 28083 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 14.
I20260709 13:14:17.577003 28153 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:17.547545  3172 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:17.561554  3028 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.470609  3085 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.567775  3160 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:17.678407 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.681408 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.682361 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.683415 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:17.684801 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 204728us
I20260709 13:14:17.685222 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:17.685681 28232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.003s	sys 0.040s
I20260709 13:14:17.687484 28761 raft_consensus.cc:1240] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
W20260709 13:14:17.691406 28232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.003s	sys 0.040s
I20260709 13:14:17.686291 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.669274 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.693133 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.589064 28157 rpcz_store.cc:276] 0709 13:14:15.161508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.184390 (+ 22882us) service_pool.cc:224] Handling call
0709 13:14:15.216403 (+ 32013us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:15.351888 (+135485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:15.351921 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:16.515204 (+1163283us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b2c000b54b8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:14:16.515610 (+   406us) log.cc:844] Serialized 26 byte log entry
0709 13:14:16.547454 (+ 31844us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:16.597344 (+ 49890us) log.cc:844] Serialized 14 byte log entry
0709 13:14:16.637119 (+ 39775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.637229 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:17.479485 (+842256us) spinlock_profiling.cc:243] Waited 9.85 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:17.568931 (+ 89446us) raft_consensus.cc:1685] finished
0709 13:14:17.568953 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:17.569084 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":833,"prepare.queue_time_us":48892,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":56499,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":11365,"spinlock_wait_cycles":29407744,"thread_start_us":31380,"threads_started":1,"wal-append.queue_time_us":166330}
I20260709 13:14:17.695732 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.697129 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1399) took 2535 ms. Trace:
I20260709 13:14:17.670048  3113 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:17.670240  3172 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.697624 28154 rpcz_store.cc:276] 0709 13:14:15.162198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.162719 (+   521us) service_pool.cc:224] Handling call
0709 13:14:15.163048 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:15.163414 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:15.163443 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:15.163914 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 13:14:15.618057 (+454143us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:15.618123 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.618232 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:17.424669 (+1806437us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:17.693123 (+268454us) raft_consensus.cc:1685] finished
0709 13:14:17.693140 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:17.697060 (+  3920us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":210,"raft.queue_time_us":164701,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":3691,"spinlock_wait_cycles":52685696,"thread_start_us":453604,"threads_started":1,"wal-append.queue_time_us":793760}
I20260709 13:14:17.579869  2592 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.112s
I20260709 13:14:17.704449 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.707162 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.709403 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:17.716501 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:17.718678 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:17.718842  3112 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.719362 28757 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:17.719497  3112 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.721662 28157 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:17.722329  3112 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:17.723488 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:17.724220  3085 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:17.708125  3082 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.727209 28232 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 21: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:17.727350  3126 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:17.732854 28763 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:17.726079 28226 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:17.704269  3172 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:17.691439  3028 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.745196 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:17.746928 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:17.675618  3058 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.703195  3071 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.695058  3160 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.753537  3058 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:17.755426  3160 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:17.760136  3178 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:14:17.764683 28085 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:17.765529 28764 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:17.765458  2911 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.766434  2911 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.766993 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.768285 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:17.746174  3082 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.769531 28157 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:17.771121  3082 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:17.745692  3098 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 13:14:17.777695 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:17.778980 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:17.779345 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.779469 28161 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:14:17.779965 28156 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:17.780042 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.780632 28153 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:17.780853 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.782356 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:17.782974 28227 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:14:17.783097 28153 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:17.764830  3028 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:17.788429 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:17.785194 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.798135 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.797904  3168 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:14:17.798470 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.805428 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.806505 28228 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
W20260709 13:14:17.807544 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:17.808629 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 63522.8us
I20260709 13:14:17.818280  3028 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:17.819144  3028 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:17.820130 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.807483  2983 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.821224 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.821314  2983 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.822108 28231 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:17.823760 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.824450 28152 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:17.787403  3107 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:17.826084 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:17.827559 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:17.828840  3167 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:17.829424  3167 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.832413 28234 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:17.834303 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.837100  3082 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:17.837844  3082 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:17.839161 28757 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
W20260709 13:14:17.840209 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 51892.1us
I20260709 13:14:17.846163  3024 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.849146  3024 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:17.849012  3160 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:17.753537  3071 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:17.850617  3167 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:17.851976  3098 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:17.852928  3158 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:17.853737  3098 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:17.829761 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:17.857002 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.858227 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:17.857730  3028 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:17.861122 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:17.862099 28232 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
W20260709 13:14:17.864259  3175 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:14:17.866463 28155 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:17.867553 28153 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:17.868196 28152 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:17.840106 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.870366 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:17.872566 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:17.873255 28758 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:14:17.820159 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.875752 28157 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:17.840410  3112 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.804347  3126 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:17.869935 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 53216.3us
I20260709 13:14:17.870736 28235 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:17.898829 28229 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:14:17.899788 28153 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:14:17.894289 28125 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:17.896293 28127 leader_election.cc:400] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
I20260709 13:14:17.900898 28127 leader_election.cc:403] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 13:14:17.902359 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.885421 28761 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:17.907371 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:17.908776 28085 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2882) took 1368 ms (client timeout 1580 ms). Trace:
W20260709 13:14:17.909399 28085 rpcz_store.cc:269] 0709 13:14:16.540380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.572623 (+ 32243us) service_pool.cc:224] Handling call
0709 13:14:17.908540 (+1335917us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002add54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:17.908679 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2687559552}
I20260709 13:14:17.910934 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.840018  3082 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:17.914894 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.915318 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.918030 28229 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:17.918259 28763 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:17.919039 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.917901 28235 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:17.921188 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:17.914530 28198 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:17.924589 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.926069 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.927470 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:17.931537 28755 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:17.924996 28049 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:17.934840 28125 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:17.937242 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.938176 28230 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 24.
I20260709 13:14:17.940196 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.943298 28227 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:17.937326 28049 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:17.905530  2952 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:17.922892  3173 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
I20260709 13:14:17.914340  3131 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:17.944530 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.960856 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.935875  3181 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:14:17.921180  3107 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:17.962107 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:17.952840 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:17.949417  3024 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:14:17.954330 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 261249us
I20260709 13:14:17.914306  3081 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:17.946450 28157 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:17.965464 28081 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:17.965605 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:17.965660 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:17.893075  3065 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:14:17.942546  2942 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:17.950948  3059 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:17.969045 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:17.967859 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.974017 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:17.949715  3080 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:17.953874  2952 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.979382 28154 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:17.923988  3112 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:17.981242 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.973341 28229 raft_consensus.cc:2955] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Committing config change with OpId 18.15: config changed from index 11 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:17.945608  3082 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:17.986364 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:17.986763 28231 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:17.906816  3116 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
W20260709 13:14:17.990010 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.936151  3048 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:14:17.956398  3183 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:17.966841  3181 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.977504  2403 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:17.981861  3107 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:17.994608  3085 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:17.978021  3024 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:18.003129 28764 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:14:18.005288 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:18.006520 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:17.994998  3059 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:17.991571  2952 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:18.024259  3082 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:18.020633 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:18.024854  3082 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.027096  3082 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:18.003880  2983 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:18.014424  2403 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.028332  2403 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:18.015295  3181 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:18.028914  3058 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:18.022012 28230 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:14:18.010007  2975 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:18.055670 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.056581 28083 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 21: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:18.022500 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:17.949054 28125 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:18.043885 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:18.038785  3080 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:18.060142  3048 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:18.063189  3080 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:18.064070  3080 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.065357  3080 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:18.069051 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:18.060575 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 111639us
I20260709 13:14:18.074656 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:18.062676 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:18.062129 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.083951 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:14:18.118845 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44462.7us
I20260709 13:14:18.120208 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:18.122640 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.042908  3160 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:18.018356  2942 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.052s
I20260709 13:14:18.021003  3183 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.117996  3085 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:18.181468 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.151661  3048 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:18.077499  3058 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.097s
I20260709 13:14:18.150138  3160 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:18.172559  3166 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:14:18.189478 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.165280  3091 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:18.166764  2983 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:18.124328  2952 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.179839  2975 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:18.183456  3081 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 14. Current term is 15.
W20260709 13:14:18.236523 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 238469us
I20260709 13:14:18.240108 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.245214 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:18.225973 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 76473.6us
I20260709 13:14:18.173310  3172 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.215945  3160 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:18.267107 28235 raft_consensus.cc:2368] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19.
I20260709 13:14:18.267769 28125 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:18.235496  2975 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:18.277299 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.243960  2983 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:18.175593  3024 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.283696 28085 raft_consensus.cc:2368] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18.
I20260709 13:14:18.284116 28083 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:18.287918 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:18.272851  3091 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:18.230126  3048 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.050s
I20260709 13:14:18.175659  3107 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.182722  3059 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.282166  2975 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:18.258123  2403 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:18.309005 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 393478us
I20260709 13:14:18.282859  2911 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:18.316799 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.359478 28083 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:18.303697  3116 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:18.365051 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 279079us
I20260709 13:14:18.361810  3081 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:18.378659 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:18.407513 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 369573us
I20260709 13:14:18.415625  3172 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:18.417337  3193 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:14:18.421257  2403 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:18.434559 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 60208.3us
I20260709 13:14:18.435590 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:18.403283 28083 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:18.479789 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:18.482337 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46862.5us
I20260709 13:14:18.483410 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:18.489104 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.362270  3028 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.517166  2911 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:18.517833  2911 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:18.518779  2911 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.519531 28083 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:14:18.521750 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.526868 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:18.539264 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:18.548799 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.549777 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:18.571755 28077 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:14:18.573335 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46583.3us
I20260709 13:14:18.360564  2983 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.577050 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.578444 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.494553  3081 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:18.371037  3116 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.514096  2403 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:14:18.607179  3187 log.cc:927] Time spent T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:14:18.609477 28228 raft_consensus.cc:1275] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:18.613471  2952 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:18.616369 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3343) took 1543 ms. Trace:
I20260709 13:14:18.618824 28158 rpcz_store.cc:276] 0709 13:14:17.072892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.377908 (+305016us) service_pool.cc:224] Handling call
0709 13:14:17.378406 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.401476 (+ 23070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.401533 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.824167 (+422634us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b2c000b54b8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:14:17.824662 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:14:18.181664 (+357002us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:18.181706 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.181786 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.616026 (+434240us) raft_consensus.cc:1685] finished
0709 13:14:18.616049 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.616280 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22221,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":153,"spinlock_wait_cycles":120806144,"thread_start_us":356550,"threads_started":1,"wal-append.queue_time_us":542868}
I20260709 13:14:18.576141  3190 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:18.572320  3065 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
W20260709 13:14:18.639823 28730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.005s	sys 0.026s
I20260709 13:14:18.592957  3183 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:18.642948 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.621580  3192 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:18.621202  3191 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:18.645996 28730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.005s	sys 0.026s
W20260709 13:14:18.648694 28727 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.015s	sys 0.015s
I20260709 13:14:18.572438  3028 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.590514  3048 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:14:18.626587  3182 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.628093  2403 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:18.627848  3092 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:14:18.664366 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:18.666953 28084 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:18.662443 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:18.663447 28727 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.015s	sys 0.015s
W20260709 13:14:18.655867 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 366092us
I20260709 13:14:18.680824 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:18.657466 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 617879us
I20260709 13:14:18.637713  3190 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.700528 28155 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:14:18.713208 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:18.712577 28201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.003s	sys 0.010s
I20260709 13:14:18.722769 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:18.725449 28201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.003s	sys 0.010s
I20260709 13:14:18.665635  3194 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:18.666040  3197 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:18.683229  3091 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:18.729429 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:18.727413 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.734912 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:18.725507  3181 consensus_meta.cc:287] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.696s	user 0.012s	sys 0.000s
I20260709 13:14:18.744601 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:18.745499 28159 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
W20260709 13:14:18.732223  3071 consensus_meta.cc:287] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.877s	user 0.008s	sys 0.004s
I20260709 13:14:18.754141 28158 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:18.756773 28153 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
W20260709 13:14:18.760614 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.010s	sys 0.014s
W20260709 13:14:18.761181 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.010s	sys 0.014s
W20260709 13:14:18.761598 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 677675us
I20260709 13:14:18.661464  3191 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.667268  3182 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.773703 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.739594  3197 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:18.774956 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:18.702138  3081 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.770119  3071 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:18.779749  3071 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.780179  2867 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:18.782315  3071 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:18.784053  3028 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:18.788939  2403 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.766219  3059 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:18.793596  3189 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:14:18.793509  3059 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.803690  3205 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.803870  3059 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:18.815481 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:18.743468  3194 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.743304  3181 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.770143  2911 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:18.805474  3195 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:14:18.797380  3197 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:18.836434  3028 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:18.838507  3059 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.806335  3116 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:18.845736 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:18.807825  3206 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:14:18.805879  3183 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:18.854485 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:18.862403 28082 raft_consensus.cc:2393] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 28: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:18.826948  3172 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.869410 28084 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
W20260709 13:14:18.884590 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.809736  3191 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:18.914552  3082 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:18.920192  3082 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.921485  2952 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:18.922073  2952 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:18.926811  3126 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:18.806838  2975 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:18.935957 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.940044 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3657) took 1364 ms. Trace:
W20260709 13:14:18.943868 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 242763us
W20260709 13:14:18.944306 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 110862us
I20260709 13:14:18.820787  3198 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.089s
I20260709 13:14:18.940745 28229 rpcz_store.cc:276] 0709 13:14:17.576016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.972685 (+396669us) service_pool.cc:224] Handling call
0709 13:14:17.972992 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:17.973221 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:14:18.017480 (+ 44259us) log.cc:844] Serialized 14 byte log entry
0709 13:14:18.739617 (+722137us) log.cc:844] Serialized 14 byte log entry
0709 13:14:18.939615 (+199998us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:18.939679 (+    64us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:18.939737 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.939824 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.939973 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30606,"prepare.queue_time_us":270363,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":68674,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":14072,"thread_start_us":846287,"threads_started":3,"wal-append.queue_time_us":649588}
I20260709 13:14:18.957362 28730 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:14:18.958007 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 383592us
I20260709 13:14:18.965677 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:18.974810 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:18.975584 28761 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:18.976709 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:18.977859 28763 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:18.982381 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:18.963263 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2763) took 1014 ms. Trace:
I20260709 13:14:18.983338 28085 rpcz_store.cc:276] 0709 13:14:17.948429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.949524 (+  1095us) service_pool.cc:224] Handling call
0709 13:14:18.962970 (+1013446us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b540027c9d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:18.963184 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729334656}
I20260709 13:14:18.822494  3205 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.915402  3182 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:18.806649  3091 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.987651 28083 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:18.854288  2911 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:18.993150 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:18.959116  3126 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.907226  3059 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:18.994737 28755 raft_consensus.cc:2955] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Committing config change with OpId 18.15: config changed from index 11 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:19.002178 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.005784 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:18.963537  3072 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:19.013429  3223 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:14:18.961483  3211 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:18.993983  3204 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:14:18.929134  3172 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:18.996352  3060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.272s	user 0.014s	sys 0.016s
I20260709 13:14:18.996513 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1423) took 1026 ms. Trace:
I20260709 13:14:18.964579  3028 consensus_queue.cc:1059] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:14:18.992852  3024 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.038942 28763 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:19.031304  3193 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:19.052770 28080 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:19.047746 28157 rpcz_store.cc:276] 0709 13:14:17.970336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.970497 (+   161us) service_pool.cc:224] Handling call
0709 13:14:17.970776 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:18.257744 (+286968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.257774 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:18.271149 (+ 13375us) log.cc:844] Serialized 26 byte log entry
0709 13:14:18.665207 (+394058us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:14:18.992346 (+327139us) log.cc:844] Serialized 14 byte log entry
0709 13:14:18.992765 (+   419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.996254 (+  3489us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.996309 (+    55us) raft_consensus.cc:1685] finished
0709 13:14:18.996324 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.996454 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15646,"raft.queue_time_us":8365,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":15905,"thread_start_us":871782,"threads_started":3,"wal-append.queue_time_us":447104}
W20260709 13:14:19.042124  3060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.318s	user 0.014s	sys 0.016s
I20260709 13:14:19.007295  3126 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.040696  3190 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:19.061967 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3670) took 1044 ms. Trace:
I20260709 13:14:19.049376  3225 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:19.060340  3201 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
I20260709 13:14:19.055613  3072 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:19.065872 28226 rpcz_store.cc:276] 0709 13:14:18.008257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.082160 (+ 73903us) service_pool.cc:224] Handling call
0709 13:14:18.082528 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.082791 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:18.640636 (+557845us) log.cc:844] Serialized 14 byte log entry
0709 13:14:19.052817 (+412181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:19.052872 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:19.052922 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.052987 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.053105 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19501,"prepare.queue_time_us":645405,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":266700,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5919,"thread_start_us":899895,"threads_started":3}
I20260709 13:14:19.057035  3024 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.015571  2952 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.061470  3227 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:19.070044  3080 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.004s	user 0.014s	sys 0.000s
I20260709 13:14:19.045358  3211 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:19.097994  3215 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:14:19.100878 28079 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:19.093796 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:19.062981  3231 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:19.115727  3231 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.116487  3236 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:19.117184  3236 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.118181  3231 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.113834 28226 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:19.119514  3236 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:19.119902  3160 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.840s	user 0.007s	sys 0.004s
I20260709 13:14:19.076874  3225 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.123268  3194 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.122416 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:19.124842  3172 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:19.126498  3126 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:19.128357  3205 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:19.129354  2975 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:19.091640  3072 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.132766 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1422) took 1169 ms. Trace:
I20260709 13:14:19.132958  3072 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:19.133060 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:19.133191 28160 rpcz_store.cc:276] 0709 13:14:17.963168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.963736 (+   568us) service_pool.cc:224] Handling call
0709 13:14:17.978709 (+ 14973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.985327 (+  6618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.985361 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:18.013467 (+ 28106us) log.cc:844] Serialized 26 byte log entry
0709 13:14:18.367605 (+354138us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:18.367653 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.367720 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:19.132554 (+764834us) raft_consensus.cc:1685] finished
0709 13:14:19.132574 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.132694 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13750,"raft.queue_time_us":6244,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":8475,"thread_start_us":307387,"threads_started":1,"wal-append.queue_time_us":621701}
I20260709 13:14:19.136274  3223 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:19.136884  3223 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.061640  3193 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.139289  3223 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:19.145341  3059 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:19.151751 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 186129us
I20260709 13:14:19.146533  3181 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.158473  3112 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.158870  3211 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:19.159900 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:19.160552 28084 raft_consensus.cc:2376] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21. Re-sending same reply.
I20260709 13:14:19.073222  3190 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:19.160051  3224 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:14:19.165688 28049 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:19.127558  2952 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:19.163743 28125 leader_election.cc:400] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18.
I20260709 13:14:19.169219 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:19.169430  3200 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:19.169813 28762 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:19.169939  3200 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.170439  3200 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:19.166260 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:19.173614 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:19.167389 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:19.167551 28763 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:14:19.120352  3160 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.090904  3214 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.090s
I20260709 13:14:19.127417  3225 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:19.158771  3183 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.163533  3059 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.180264  3211 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:19.177670 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:19.183634 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:19.184093 28049 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:19.184192 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 152062us
I20260709 13:14:19.184437 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:19.188943 28084 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:19.188881 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:19.190433 28226 raft_consensus.cc:2410] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
W20260709 13:14:19.190852  3180 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.015s	user 0.006s	sys 0.006s
W20260709 13:14:19.191412  3180 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.006s	sys 0.006s
I20260709 13:14:19.191664 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:19.192754 28226 raft_consensus.cc:2410] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:14:19.192641 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:19.194238 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:19.197849 28226 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:19.198266 28226 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:19.199153 28226 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.201198 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:19.201498 28761 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:19.163317  3227 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:14:19.205441  3227 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:19.181404  3197 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.208768  3197 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:19.209153  2911 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.210052 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 1571) took 1032 ms. Trace:
I20260709 13:14:19.210465 28082 rpcz_store.cc:276] 0709 13:14:18.177858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.178101 (+   243us) service_pool.cc:224] Handling call
0709 13:14:19.209709 (+1031608us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b540017d254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:19.209989 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464268160}
I20260709 13:14:19.212906  3225 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:19.213402  3113 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.213865  3225 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:19.213945  3113 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:19.214339  3225 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:19.214648 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:19.215276  3225 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.219689  3082 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.201052  3059 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.221441 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:19.204710 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:19.222909 28232 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:14:19.223790 28084 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 13:14:19.223762 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:19.226275  3193 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.232116  3082 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:19.238327 28085 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:19.239796  3200 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.167572  3190 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.241753  3024 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.242646  3065 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:19.243999 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.220702 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:19.244874 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 60864.1us
I20260709 13:14:19.245785 28199 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:19.247491 28762 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 28.
I20260709 13:14:19.244825  3240 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:19.277452  3072 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.342232 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:19.342723 28756 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 13:14:19.343947 28125 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:19.277552  3113 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.216591  3225 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:19.281615 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:19.331717 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.346374 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:19.353631 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 107937us
I20260709 13:14:19.364202 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.386031 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:19.386368 28199 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:14:19.386832 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 105339us
W20260709 13:14:19.365541 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 144899us
I20260709 13:14:19.391675 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:19.364744  3160 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:19.189635  3183 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.407123 28085 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 18: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:14:19.407543 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 61257.8us
I20260709 13:14:19.416092 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:19.291781  3248 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:19.226748  2862 consensus_queue.cc:1048] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.160s
I20260709 13:14:19.425622 28082 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
W20260709 13:14:19.364797  3241 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:14:19.365097  3107 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:19.337514  3065 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:19.354971  3240 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:19.365269  3193 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:19.405611 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:19.423630  3091 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:19.440475 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.450016 28758 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:19.451803 28758 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:14:19.422996  3211 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:19.437345 28049 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:19.384114  3024 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.089s
W20260709 13:14:19.455466 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49965.7us
I20260709 13:14:19.456023 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:19.461391 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.429006  2975 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:19.468534 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.465837 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.473063 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.449216  3160 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:14:19.463677  2983 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:14:19.438340  3240 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:19.456657  3107 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:19.449594  3091 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.456959 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:19.478645  3112 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:19.479023  3211 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:19.483040  2911 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:19.482575  3248 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:19.520449 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1413) took 1656 ms. Trace:
W20260709 13:14:19.521579 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.527788 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:19.499361  3231 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:19.525970 28152 rpcz_store.cc:276] 0709 13:14:17.863597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.863779 (+   182us) service_pool.cc:224] Handling call
0709 13:14:17.868019 (+  4240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:17.889214 (+ 21195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.520173 (+1630959us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b54003769d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:19.520368 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8956288}
I20260709 13:14:19.503340  3180 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:19.502066  3113 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:19.514581  3091 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:19.531154 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 75243.5us
I20260709 13:14:19.518482  2911 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.516520  2983 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:19.574266  2983 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.535724 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:19.516491  3112 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:19.576630 28124 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:19.577045 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:19.577802 28763 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 19: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:14:19.531621 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 74808.2us
W20260709 13:14:19.529202 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 82136.7us
I20260709 13:14:19.580181 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:19.578573 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 140556us
I20260709 13:14:19.511317  3065 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.527967  3211 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.541177  3248 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.541661  3239 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:19.542014  3238 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:19.542331  3234 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:19.542897  3059 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:19.530349  2792 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:19.546631  3246 log.cc:927] Time spent T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:14:19.535044  3231 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.536794  3082 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:19.557607 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.561534  3091 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.562552  3193 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:19.567022  3072 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:19.567565  3213 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.553s	user 0.001s	sys 0.001s
I20260709 13:14:19.567786  3200 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.568496  3190 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.569965  3183 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.606740  3239 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.573244  2911 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:19.583595 28758 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:19.583815 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:19.586886 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:19.583297  3229 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.377s	user 0.001s	sys 0.006s
W20260709 13:14:19.674216 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 90480.4us
I20260709 13:14:19.674561 28757 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:19.595664  3065 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:19.623059  2792 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.678017  2792 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.679411 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1615) took 1807 ms. Trace:
I20260709 13:14:19.679863 28227 rpcz_store.cc:276] 0709 13:14:17.872272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.970736 (+ 98464us) service_pool.cc:224] Handling call
0709 13:14:17.971001 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:18.302670 (+331669us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:14:18.622980 (+320310us) log.cc:844] Serialized 14 byte log entry
0709 13:14:19.071632 (+448652us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:19.074339 (+  2707us) log.cc:844] Serialized 26 byte log entry
0709 13:14:19.074521 (+   182us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:19.074559 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.076587 (+  2028us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:19.575013 (+498426us) spinlock_profiling.cc:243] Waited 4.99 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:19.679078 (+104065us) raft_consensus.cc:1685] finished
0709 13:14:19.679213 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.679352 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11903,"prepare.queue_time_us":468419,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":34286,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":6527,"spinlock_wait_cycles":11466112,"thread_start_us":1010580,"threads_started":3,"wal-append.queue_time_us":572790}
I20260709 13:14:19.682108 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:19.579155  3080 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:19.579814  3112 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:19.579481  3205 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:19.597818 28232 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:19.595752 28226 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:19.599759 28228 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:14:19.686439 28228 pending_rounds.cc:85] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 14
I20260709 13:14:19.686739 28228 pending_rounds.cc:107] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:14:19.687001 28228 raft_consensus.cc:2889] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:19.586421  3019 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:19.586289  3257 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:19.652146 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:19.692705 28233 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:14:19.653927 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:19.693503 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:19.600826  3238 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:19.653439 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 47572.6us
I20260709 13:14:19.537922  3180 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.581770  3113 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:14:19.570767  3107 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.675297 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:19.658123  3082 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:19.712594  3205 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:19.695084 28226 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:14:19.713730 28226 pending_rounds.cc:85] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 11
I20260709 13:14:19.713442  3082 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:19.714078 28226 pending_rounds.cc:107] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:14:19.713923  3227 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:19.714422 28226 raft_consensus.cc:2889] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:19.715720 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:19.693892 28234 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:19.658347  3194 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.703179 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:19.704203 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.706815  3180 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.668576  3239 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:19.690495  3019 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:19.727342  3194 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.782424  3194 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:19.783501  3260 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:14:19.785246 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.785636  3231 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:19.786077 28152 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:19.787439 28160 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:19.783550 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.713629  3225 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:14:19.763866 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.731521 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.815435 28157 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:19.830824 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:19.832446 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.833428 28153 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:19.838284 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:19.718560  3205 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.846210 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.703174  3238 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:19.863076 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.726235  3227 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:19.789155  3080 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:19.869843 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.794437  3019 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:19.867563 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.866658 28233 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:19.881173 28226 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:14:19.881805 28226 pending_rounds.cc:85] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
I20260709 13:14:19.882120 28226 pending_rounds.cc:107] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:14:19.882769 28226 raft_consensus.cc:2889] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:19.691470  3193 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.885612 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.808771  3225 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:19.886910 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 139296us
I20260709 13:14:19.888579 28153 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 15.
I20260709 13:14:19.878408 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.889781 28157 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:19.846776  2792 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.854081  2892 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:19.846102  3194 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:19.724656  3113 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.151s
I20260709 13:14:19.894858 28158 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:14:19.912719 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:19.917682 28757 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:19.912834 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.905478 28157 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:19.702538  3257 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:19.894797 28199 connection.cc:256] RPC call timeout handler was delayed by -1.5705611190001036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 13:14:19.918917 28152 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:19.871404  3205 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:19.947955 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.153456 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:19.931093 28161 raft_consensus.cc:2410] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:14:19.958101 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:20.156553 28154 raft_consensus.cc:2368] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18.
I20260709 13:14:19.958747 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:20.269201 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:20.269709 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:20.270010 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1509460250001666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1904s for a response
I20260709 13:14:20.270643 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.271049 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:20.272325 28199 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:20.272748 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.267258 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:20.274006 28085 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 21: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:20.012413 28160 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:20.274551 28158 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 23: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:20.279605 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.285665 28085 raft_consensus.cc:2368] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19.
I20260709 13:14:20.286911 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.290052 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.291051 28086 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:20.292083 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.288026 28085 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:19.892498  3225 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:20.285209 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1567) took 2264 ms. Trace:
I20260709 13:14:20.296427 28081 rpcz_store.cc:276] 0709 13:14:18.005665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.008665 (+  3000us) service_pool.cc:224] Handling call
0709 13:14:18.038246 (+ 29581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.195046 (+156800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.195096 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:18.195153 (+    57us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:18.195207 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.270004 (+2074797us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54001af4d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:20.270081 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.270284 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360005888}
I20260709 13:14:19.895134  2892 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:20.300546 28082 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:14:20.303587 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:20.148708  3193 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.318446 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:20.320369 28086 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:20.321882 28226 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:20.303628 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:19.949738  3198 raft_consensus.cc:3055] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:20.331717  3198 raft_consensus.cc:740] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:20.333268  3198 consensus_queue.cc:260] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.921262  3240 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.337419  3240 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:20.306591 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.338907 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.340034  3239 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:20.340476  3239 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.338740  2892 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.307619 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:20.342944 28757 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:14:20.344127 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2768) took 1185 ms. Trace:
I20260709 13:14:20.259258  3065 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.344574 28084 rpcz_store.cc:276] 0709 13:14:19.159161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.202235 (+ 43074us) service_pool.cc:224] Handling call
0709 13:14:19.223566 (+ 21331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:19.224276 (+   710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.343874 (+1119598us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b540017d250. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:20.344058 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48078080}
I20260709 13:14:20.345093  3065 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.346346 28084 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:20.348225  3065 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:19.938755  3194 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.309296  3227 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:20.309937 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:20.284924  3092 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:19.913049  2792 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.311895 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:20.369220 28158 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:14:20.340859  3239 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:20.307343  3107 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:20.349431 28235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2758) took 2459 ms (client timeout 1954 ms). Trace:
I20260709 13:14:20.015520  3222 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:20.295643  3072 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 13:14:19.955760  2911 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:19.950654  3112 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.358474 28233 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:19.891510  3080 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.360906 28757 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 13:14:20.325935 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:19.959074  3091 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:20.357085  2403 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:20.375594 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:20.371447 28235 rpcz_store.cc:269] 0709 13:14:17.889784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.921711 (+ 31927us) service_pool.cc:224] Handling call
0709 13:14:18.068879 (+147168us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b58001c0598. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:14:20.349058 (+2280179us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54003caad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:20.349277 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446405120}
W20260709 13:14:20.370901 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 67363.3us
I20260709 13:14:20.417850 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:20.408632 28086 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 24.
I20260709 13:14:20.336092  3225 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.393723  3065 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:20.390602  3190 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:20.392213  3222 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:20.312490  3019 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.430859 28085 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 24: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:20.430802 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:20.406715  3257 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.435781 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:20.443900  3211 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:20.447074  3238 consensus_meta.cc:287] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.560s	user 0.004s	sys 0.003s
I20260709 13:14:20.447602  3238 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.449967  2792 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:20.381865  3256 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:14:20.400537  2892 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:20.456868 28199 leader_election.cc:400] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19.
I20260709 13:14:20.457530 28199 leader_election.cc:403] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:20.465972 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:20.398883  3107 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.469292 28763 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:20.471238 28050 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.606s (SENT) [suppressed 17 similar messages]
I20260709 13:14:20.436333  3065 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:20.473480 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:20.475940 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:20.475843  3065 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:20.476985  3065 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.477487  3065 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:20.438310  3092 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:20.481671 28127 connection.cc:256] RPC call timeout handler was delayed by -1.040788842999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
I20260709 13:14:20.450618  3194 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.486368 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:20.471307  3211 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19.
I20260709 13:14:20.488088  3240 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:20.487949  3227 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:20.536440  3240 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.488222  3092 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:20.536898  3227 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:20.489955  2792 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:20.485882  3091 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:20.490638 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:20.536957  3240 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:20.539585  3091 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.516678  2911 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.540803  3080 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.424691  2403 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.472826  3180 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.535879 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1616) took 2485 ms. Trace:
W20260709 13:14:20.536860 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 50589.1us
I20260709 13:14:20.483705  3193 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:20.545419 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:20.541066  3092 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:20.508407 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2893) took 1960 ms. Trace:
W20260709 13:14:20.539754 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49186.4us
I20260709 13:14:20.539656  2792 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.543586 28231 rpcz_store.cc:276] 0709 13:14:18.050069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.083595 (+ 33526us) service_pool.cc:224] Handling call
0709 13:14:18.083911 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:18.084420 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:18.088165 (+  3745us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:19.960375 (+1872210us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b2c0004c898. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:14:19.960839 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:14:20.378027 (+417188us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:20.378081 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.378189 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:20.533206 (+155017us) raft_consensus.cc:1685] finished
0709 13:14:20.533231 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.535804 (+  2573us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109445,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":12041,"spinlock_wait_cycles":66663808,"thread_start_us":416649,"threads_started":1,"wal-append.queue_time_us":470044}
I20260709 13:14:20.551011 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1578) took 2062 ms. Trace:
W20260709 13:14:20.552112 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.549922 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:20.551923 28083 rpcz_store.cc:276] 0709 13:14:18.547991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.548170 (+   179us) service_pool.cc:224] Handling call
0709 13:14:18.548563 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.993702 (+445139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.508043 (+1514341us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b5400297cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:20.508320 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966943104}
I20260709 13:14:20.554380 28152 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:20.551426 28078 rpcz_store.cc:276] 0709 13:14:18.488654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.488867 (+   213us) service_pool.cc:224] Handling call
0709 13:14:18.489273 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.489543 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 13:14:18.707176 (+217633us) log.cc:844] Serialized 14 byte log entry
0709 13:14:20.550478 (+1843302us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b18003e8a20. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a
0709 13:14:20.550677 (+   199us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:20.550724 (+    47us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:14:20.550768 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.550836 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.550957 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":366832,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":4697,"spinlock_wait_cycles":63874944,"thread_start_us":362040,"threads_started":2,"wal-append.queue_time_us":357438,"wal-append.run_cpu_time_us":1950,"wal-append.run_wall_time_us":1189676}
I20260709 13:14:20.557765 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:20.559080 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:20.560086 28231 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:20.561530 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:20.541399  3080 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:20.576445 28231 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:20.577713 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:20.580505 28231 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:14:20.563804 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:20.570359 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1597) took 1597 ms. Trace:
I20260709 13:14:20.537583  3227 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:20.574139 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:20.574819 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:20.585352 28233 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:20.581547 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:20.546444  3194 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:14:20.591817 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.595570 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:20.582842 28755 rpcz_store.cc:276] 0709 13:14:18.973466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.973648 (+   182us) service_pool.cc:224] Handling call
0709 13:14:18.994325 (+ 20677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:18.994604 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:14:19.044637 (+ 50033us) log.cc:844] Serialized 14 byte log entry
0709 13:14:19.963595 (+918958us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b18002cfce0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:14:20.390465 (+426870us) log.cc:844] Serialized 14 byte log entry
0709 13:14:20.569940 (+179475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:20.570007 (+    67us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:20.570063 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.570148 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.570293 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46241,"prepare.queue_time_us":514040,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":67,"raft.queue_time_us":45677,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":3035,"spinlock_wait_cycles":26172672,"thread_start_us":640132,"threads_started":3,"wal-append.queue_time_us":136453,"wal-append.run_cpu_time_us":1286,"wal-append.run_wall_time_us":1050464}
I20260709 13:14:20.551715  3092 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:20.545260  3193 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.556979  3182 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:20.618132 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:20.531419  3222 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.624632  2792 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:20.625723  2792 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:20.625483  3227 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.626108  2792 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:20.626518  2792 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:20.626776  3227 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:20.613188  3092 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:20.634414  3193 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:20.635109  3193 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:20.635828  3193 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:20.636219  3193 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:20.583592  3080 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.637128  3222 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:20.639808  3080 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:20.641381  3019 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:20.642675  2403 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.645334 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1625) took 1432 ms. Trace:
I20260709 13:14:20.645555  3225 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:14:20.645754 28232 rpcz_store.cc:276] 0709 13:14:19.212866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.242444 (+ 29578us) service_pool.cc:224] Handling call
0709 13:14:19.242822 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.684689 (+441867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.645133 (+960444us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54003c9e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:20.645279 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815864704}
I20260709 13:14:20.646198  3225 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:20.647019  3225 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:20.647369  3225 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:20.649513 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:20.650740 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:20.652493 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:20.624804  3205 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.609709  3126 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:20.598906  3194 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:20.636556  3193 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:20.551087  3180 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:20.665346 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 194117us
I20260709 13:14:20.666435 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:20.648161  3225 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.640250  3239 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.668560  3194 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:20.658507  3112 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:20.662323  3238 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.686882 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:20.686755  3273 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:20.687014  3279 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:20.695024 28050 leader_election.cc:336] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.683s (SENT)
I20260709 13:14:20.705081 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:20.648998  3182 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.664961  3126 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:20.671237  3205 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:20.724745 28083 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:14:20.724942 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.726457 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:20.688446  3211 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:20.725689 28086 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:20.702659  3225 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:20.691391  3080 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:20.752914 28081 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:20.740357 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.754565 28078 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 19: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:20.691217  3218 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:20.687587  3290 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:20.710255  3181 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:20.741827 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:20.723588  3126 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:20.758127 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44308.8us
W20260709 13:14:20.774106 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 123423us
I20260709 13:14:20.781628 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:20.766568  3164 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:20.761929  3290 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:20.785475  3279 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:20.788139  3239 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.784855 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:20.797473 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:20.784176  3164 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:20.790978  3131 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:20.791466  3290 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:20.827042 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:20.827507  2403 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:20.828078  2403 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:20.807653 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:20.829195 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.811117 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:20.830797  3238 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:20.831383  3238 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.833362  3180 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.834975 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:20.811826  3182 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:20.812585  3283 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:14:20.836398 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:20.817946 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1606) took 1651 ms. Trace:
I20260709 13:14:20.801003  3211 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:20.803640  3293 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:20.821058 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2896) took 2271 ms. Trace:
W20260709 13:14:20.821182  3281 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:14:20.825228  3239 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.829346  3164 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.806111  3065 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.819831 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:20.839563 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:20.838073 28761 rpcz_store.cc:276] 0709 13:14:19.166574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.166682 (+   108us) service_pool.cc:224] Handling call
0709 13:14:19.166944 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.201901 (+ 34957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.817631 (+1615730us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b54004c5ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:20.817867 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79029504}
I20260709 13:14:20.844918 28079 rpcz_store.cc:276] 0709 13:14:18.550150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.550314 (+   164us) service_pool.cc:224] Handling call
0709 13:14:18.550959 (+   645us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.113364 (+562405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.820798 (+1707434us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54003a98d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:20.820998 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263963904}
I20260709 13:14:20.849576 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2927) took 1019 ms. Trace:
I20260709 13:14:20.850083 28086 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:20.850008 28159 rpcz_store.cc:276] 0709 13:14:19.830399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.835356 (+  4957us) service_pool.cc:224] Handling call
0709 13:14:20.819664 (+984308us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:14:20.849513 (+ 29849us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12483456}
I20260709 13:14:20.850332 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.851862 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.852528 28083 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:20.853111  3239 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.854420  2792 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.855479  3239 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:20.856346 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.843858  3112 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.857620 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.858255 28077 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:20.853447 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.857329  3112 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:20.806447  3131 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:20.862155 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:20.863322 28083 raft_consensus.cc:2393] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 25: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:20.867897  3065 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:20.869550  3293 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:20.870044  3293 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.870424  3293 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:20.858116  3219 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:20.873600  3219 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:20.874050  3219 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:20.874531 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3693) took 1291 ms. Trace:
I20260709 13:14:20.836040  3107 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.874948 28227 rpcz_store.cc:276] 0709 13:14:19.583345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.681822 (+ 98477us) service_pool.cc:224] Handling call
0709 13:14:20.874316 (+1192494us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54000fb9d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:20.874467 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321195904}
I20260709 13:14:20.856492  2792 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:20.847368  3211 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:20.885421 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.837838  3028 consensus_queue.cc:1059] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.037s
I20260709 13:14:20.896631 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:20.860105  3227 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:14:20.895730  3164 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:20.887966  3228 log.cc:927] Time spent T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.674s	user 0.000s	sys 0.015s
I20260709 13:14:20.915196 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2934) took 1013 ms. Trace:
I20260709 13:14:20.918990 28153 rpcz_store.cc:276] 0709 13:14:19.901689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.902805 (+  1116us) service_pool.cc:224] Handling call
0709 13:14:19.903287 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:20.146866 (+243579us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:20.146915 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:20.283141 (+136226us) log.cc:844] Serialized 26 byte log entry
0709 13:14:20.583434 (+300293us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:20.583475 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.583543 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:20.914907 (+331364us) raft_consensus.cc:1685] finished
0709 13:14:20.914936 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.915115 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":365099,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":15982,"thread_start_us":229439,"threads_started":1,"wal-append.queue_time_us":357531}
I20260709 13:14:20.886771  3065 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.889582  3211 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:20.888515  3179 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:20.896294  3274 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:14:20.888935  3197 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:19.327776 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:20.978089 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:20.893918  2403 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:20.984004  3092 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:21.019459 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 302183us
I20260709 13:14:21.003768 28077 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 19: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:20.993500  3112 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:20.997962  3279 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:21.011721 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:21.013403 28079 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:21.014106 28084 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:14:21.025902 28084 pending_rounds.cc:85] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting all ops after (but not including) 12
I20260709 13:14:21.026180 28084 pending_rounds.cc:107] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:14:21.026438 28084 raft_consensus.cc:2889] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:21.026782 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.027398 28160 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:21.029683 28763 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 1278) took 1428 ms. Trace:
I20260709 13:14:21.032217 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.033144 28155 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:21.034359 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:21.017441  3275 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:21.017896  3194 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.018790  3179 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:20.995432  3126 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.091023  3126 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:21.094476  3194 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:21.095883 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.023816 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2895) took 2474 ms. Trace:
I20260709 13:14:21.024315 28127 leader_election.cc:400] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19.
I20260709 13:14:21.096959 28080 rpcz_store.cc:276] 0709 13:14:18.549603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.550614 (+  1011us) service_pool.cc:224] Handling call
0709 13:14:18.578782 (+ 28168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.056541 (+477759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.023593 (+1967052us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b5400267fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:21.023756 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1090172288}
W20260709 13:14:21.097599 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73411.8us
W20260709 13:14:21.070384 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.098646 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:21.099943 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:21.030243 28763 rpcz_store.cc:276] 0709 13:14:19.601393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.601801 (+   408us) service_pool.cc:224] Handling call
0709 13:14:19.602109 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.471700 (+869591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.029456 (+557756us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54006504d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:21.029623 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993867648}
I20260709 13:14:20.996846  3227 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:21.035614 28153 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:21.042600 28152 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:21.104909 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:21.034453  3219 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.041407  2792 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:14:21.110428  3219 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:21.110690  2792 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:21.111145  2792 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:21.111532  2792 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:21.112118 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:20.997422  3240 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.022541  3238 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.115540 28160 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:21.043833 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.117691 28156 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:21.060027 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:21.115609  3227 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.134045  3227 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.145684  3179 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.148111  3092 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.055384  3289 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.098s
I20260709 13:14:21.153673  3289 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:14:21.154047  3289 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:21.154832  3289 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.155998  3289 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:21.029780  3112 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.118072  3275 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:21.157114  3112 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:21.158178 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.160020  3112 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.160373  3240 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:21.160821  3273 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:21.161206  3273 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.161523  3273 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:21.162370 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 103276us
I20260709 13:14:21.162547  3293 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.163384 28125 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:21.164129  3272 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:21.164515  3272 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.101583  3291 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.116878  3290 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.114421  3287 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.172053 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:21.172806 28077 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:21.190055 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:21.168983 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3689) took 1586 ms. Trace:
I20260709 13:14:21.191431 28228 rpcz_store.cc:276] 0709 13:14:19.582147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.582502 (+   355us) service_pool.cc:224] Handling call
0709 13:14:19.582782 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.687456 (+104674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.168709 (+1481253us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b54001c7fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:21.168917 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37979392}
I20260709 13:14:21.169395 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1612) took 1212 ms. Trace:
I20260709 13:14:20.980077  3065 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:21.199055 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.192775 28764 rpcz_store.cc:276] 0709 13:14:19.957480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.960119 (+  2639us) service_pool.cc:224] Handling call
0709 13:14:19.960442 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:19.960838 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:14:19.960865 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:19.961340 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:14:20.542729 (+581389us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:20.542770 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.542851 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:21.169044 (+626193us) raft_consensus.cc:1685] finished
0709 13:14:21.169067 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:21.169334 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1262,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":20522,"thread_start_us":391997,"threads_started":1,"wal-append.queue_time_us":782681}
W20260709 13:14:21.196429  3193 consensus_meta.cc:287] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.001s
I20260709 13:14:21.164908  3272 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:21.164674  3293 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:21.210718 28227 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
W20260709 13:14:21.213557 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.214956 28234 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:21.217540 28233 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:21.218086  3227 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.218117 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:21.167023  3290 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:21.229446 28230 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:21.219365  3092 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:21.232542 28156 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:21.165035  3291 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.230441  3092 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.236981  3091 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.237385  3158 raft_consensus.cc:3055] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:21.246353  3131 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.202009  3287 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.228242  3193 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.232197  3227 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.227066  3082 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:21.280287 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.338508 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.281586 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1633) took 1553 ms. Trace:
I20260709 13:14:21.342600 28226 rpcz_store.cc:276] 0709 13:14:19.728298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.883942 (+155644us) service_pool.cc:224] Handling call
0709 13:14:19.889180 (+  5238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.322391 (+433211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.260545 (+938154us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b54003ca0d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:21.281522 (+ 20977us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":994554752}
W20260709 13:14:21.343547 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.297060 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.300935 28081 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:21.302426  3107 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:21.303395  3179 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.011s
I20260709 13:14:21.321401  3158 raft_consensus.cc:740] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:21.279238  3218 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:21.302767  3165 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:14:21.361631  3065 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:21.389775 28199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.017s	sys 0.046s
W20260709 13:14:21.390179 28199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.017s	sys 0.046s
W20260709 13:14:21.390452 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 550918us
I20260709 13:14:21.367218  3227 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.368512  3158 consensus_queue.cc:260] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.415827 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:21.398479 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:21.418308 28231 raft_consensus.cc:1240] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:21.418684 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:21.369694  3219 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:21.420284 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:21.372700  3107 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:21.375066  3131 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:21.394367 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:21.396323 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.396807 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:21.397059 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:21.401252 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.402870 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.404412 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.405890 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.406908 28758 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:21.407809 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:21.369568 28233 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:21.325518  3194 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:14:21.395396 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:21.423697 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:21.424927 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:21.430706 28078 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:21.448161 28231 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:21.449066 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:21.426692 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:21.427284 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:21.449848 28755 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:21.450131 28229 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:21.333189  3082 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.430130 28759 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:21.430423 28160 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:21.438521 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:21.454841  3082 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:21.455832  3091 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:21.456481 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.456578  3275 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:21.457653 28159 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:14:21.457901 28161 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:21.458451 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.458825 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.431607 28152 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:21.432677 28155 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:21.440308 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:21.445209 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.462710 28085 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:21.456025  3227 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:21.456979 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:21.444548 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.468997 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.441677 28226 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:21.433059 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:21.466706 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:21.467314 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:21.391959  3218 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:21.482339 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:21.483510 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:21.474682 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:21.423384  3240 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:14:21.486613  3240 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:21.486944  3240 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:21.487739  3240 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.489681  3291 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.490993  3092 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.492527  3287 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.487874 28234 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:14:21.493897 28234 pending_rounds.cc:85] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 9
I20260709 13:14:21.494402 28234 pending_rounds.cc:107] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:14:21.476069 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.476390 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.476707 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:21.477315  3082 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:21.496567 28234 raft_consensus.cc:2889] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:21.477726 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:21.497908 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:21.496357  3290 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:21.499419  3290 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:21.499943 28233 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:21.494704  3193 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.500939 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:21.501995  3290 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:21.502458  3290 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:21.502912  3290 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:21.468353  3227 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:21.505293  3227 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:21.506639 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:21.474830  3211 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.509351  3211 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:21.510023  3211 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:21.510591 28230 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:21.512856  3211 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:21.484490 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:21.460814 28124 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:21.517429  3193 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:21.519618 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:21.520596 28079 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 22: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:21.521307 28755 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:21.526999 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:21.517540 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 56843.9us
I20260709 13:14:21.508104 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:21.445931 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:21.535133 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:21.547293 28155 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:21.537989 28229 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
W20260709 13:14:21.549069 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.516152  3273 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.528656 28078 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
W20260709 13:14:21.552106 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 452183us
W20260709 13:14:21.558182 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.539880 28081 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:21.566387 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.567175 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:21.567655 28760 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:21.543407 28085 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:21.544745 28152 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:21.529263 28755 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:14:21.578188 28755 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:21.581616 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:21.583806 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.457782  3275 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:21.590121 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:21.427896  3107 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:21.590657 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:21.588625 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 142819us
I20260709 13:14:21.593888 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.605043 28152 raft_consensus.cc:2368] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 21. Current term is 22.
I20260709 13:14:21.594779 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:21.595346 28153 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:21.616843 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:21.618585  3273 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:21.618785 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 110764us
I20260709 13:14:21.589313  3181 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:21.619293  3273 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:21.604810 28760 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 25.
I20260709 13:14:21.620179 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.456521  3091 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.620461 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:21.620507 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:21.623358 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:21.627389 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:21.632577 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.636938 28757 raft_consensus.cc:2393] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 22: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:21.591965  3289 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:21.626298  3065 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:21.619532  3181 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:21.674638  3289 consensus_queue.cc:1059] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:21.675892  3198 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:21.675616  3181 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.676909  3181 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:21.681532 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.681720 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 61300.3us
I20260709 13:14:21.683070 28159 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:21.683876 28125 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:21.685540 28081 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 13:14:21.688295 28157 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.026s	sys 0.024s
W20260709 13:14:21.688627 28157 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.026s	sys 0.024s
I20260709 13:14:21.586691  3240 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:21.693480 28161 raft_consensus.cc:1240] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:21.601161  3194 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:21.702551 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:21.514921  3211 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.326961 27975 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:21.693539 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:21.716763 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:21.588100  3275 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:21.619760  3273 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.717360 28084 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:21.625269  2892 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:21.583312  3291 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.120s
I20260709 13:14:21.719442 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:21.629038  3091 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:21.706537 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:21.693138  3116 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.721450 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.726047 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:21.708045 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:21.687112  3289 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:21.717990 28159 raft_consensus.cc:2955] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Committing config change with OpId 18.15: config changed from index 11 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:21.716933 28082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.036s	sys 0.053s
I20260709 13:14:21.722438 28160 raft_consensus.cc:2410] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:14:21.719060 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:21.732327 28082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.036s	sys 0.053s
I20260709 13:14:21.722090  3273 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:21.621287  3218 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:21.739563 28077 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.004s	sys 0.028s
I20260709 13:14:21.703873  3065 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:21.715472  3293 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
W20260709 13:14:21.767683 28077 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.004s	sys 0.028s
I20260709 13:14:21.770182 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:21.770759 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 51767.7us
I20260709 13:14:21.716763  3182 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:14:21.739493  3081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.031s	sys 0.042s
I20260709 13:14:21.811009 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:21.812139 28228 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.014s	sys 0.029s
I20260709 13:14:21.731282  3179 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:21.723574  3160 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.812965 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:21.859663 28153 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:21.860924 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.861683 28153 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:21.731106  3194 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:21.813264 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:21.864331 28158 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:21.700403  3290 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.867609  3290 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:21.812491 28228 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.015s	sys 0.030s
I20260709 13:14:21.734962  3291 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:21.718779  3223 consensus_queue.cc:1059] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.052s
I20260709 13:14:21.724089  3092 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:21.857733 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:21.891345 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:21.806604  2792 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.695s	user 0.004s	sys 0.008s
I20260709 13:14:21.735860  3116 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.938753 28764 raft_consensus.cc:2393] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 21: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:21.736395  3164 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.955260  3293 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:21.955612  3164 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.955899  3293 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.957971  3164 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:21.875401  3291 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:21.959664  3291 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:21.835347  3081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.031s	sys 0.043s
I20260709 13:14:21.962649  3081 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:21.963270  3081 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.967190  3291 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:21.971096  3218 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:21.887893  3160 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.972940  3081 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:21.974109  3160 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:21.976779  2792 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:21.978432 28154 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 23: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:21.880580 28153 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
W20260709 13:14:21.720193  3205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.021s	sys 0.031s
I20260709 13:14:21.815263  3176 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:21.979802  2792 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:21.980180  3176 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.980770  3176 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:14:21.983902  3272 consensus_meta.cc:287] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.752s	user 0.009s	sys 0.001s
I20260709 13:14:21.874191  3182 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:21.979835 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:21.993649 28084 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
W20260709 13:14:21.929159 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.006736 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.994375  3198 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.014012  3182 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:22.018945  3193 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.501s	user 0.005s	sys 0.003s
I20260709 13:14:22.019429  3193 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:22.022514  3176 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:22.023329  3227 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.000s
I20260709 13:14:22.023764  3227 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.026973 28152 raft_consensus.cc:1240] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:22.027369 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.014104  3198 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.029572 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:22.034921 28156 raft_consensus.cc:2410] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
W20260709 13:14:21.994589  3205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.024s	sys 0.033s
I20260709 13:14:22.032383 28152 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:14:21.743316  3289 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:21.878650 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:21.888293  3112 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:21.841461  3299 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:21.958002  3293 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.046414 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.044538 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:22.052407 28756 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:14:22.053786 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 175211us
I20260709 13:14:22.055455 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:22.044633 28080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.009s	sys 0.025s
W20260709 13:14:22.056440 28080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.009s	sys 0.025s
I20260709 13:14:22.057271 28762 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:21.876535  3303 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.210s
I20260709 13:14:22.095541 28152 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:21.767624  3107 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.006608  3272 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.041560  3176 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.058925  3299 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:22.115201 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.046458  3239 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.016s	sys 0.047s
I20260709 13:14:21.875301  3275 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.119877 28152 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:14:22.115157  3131 consensus_meta.cc:287] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.672s	user 0.005s	sys 0.006s
W20260709 13:14:22.122241 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.012s	sys 0.031s
W20260709 13:14:22.128871 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.012s	sys 0.031s
I20260709 13:14:22.143961 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:22.130982 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.135876 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:22.099129  3293 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:22.140540 28158 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 19: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:22.141216 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:22.144101 28161 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 25.
I20260709 13:14:22.119809 28756 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 26.
I20260709 13:14:22.144380 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:22.122179  3239 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.016s	sys 0.047s
I20260709 13:14:22.040050  3065 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.148118 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.159715 28156 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:14:22.152504 28231 raft_consensus.cc:2410] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:14:22.152853 28755 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:22.155696 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:22.155705  3107 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:22.163511 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:22.165069 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:22.165716 28156 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:22.158217  3116 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.158854  3181 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:22.161939  3158 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:22.154385 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:22.179019 28763 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.008s	sys 0.017s
I20260709 13:14:22.169247 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:14:22.179615 28763 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.009s	sys 0.017s
W20260709 13:14:22.173731 28086 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.039s	sys 0.076s
I20260709 13:14:22.176815 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:22.181253 28086 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.040s	sys 0.077s
W20260709 13:14:22.194796 28232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.020s	sys 0.029s
W20260709 13:14:22.195228 28232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.020s	sys 0.029s
I20260709 13:14:22.209575 28154 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:22.162905  3303 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:22.214440 28758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.028s	sys 0.025s
W20260709 13:14:22.217162 28758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.029s	sys 0.025s
I20260709 13:14:22.221128  3273 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.224485  3273 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.227392 28084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66540b865ca24721910c667ccd790b77"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:22.228538 28084 raft_consensus.cc:606] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:22.228999 28084 raft_consensus.cc:614] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:22.229920 28156 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:22.236303 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:22.237317 28755 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:22.175387  3131 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:22.255671 28154 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:22.200038  3193 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:22.214927  3176 raft_consensus.cc:2764] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:14:22.219762  3181 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:14:22.270104 28730 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.000s	sys 0.030s
W20260709 13:14:22.270498 28730 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.000s	sys 0.030s
W20260709 13:14:22.270820 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 582651us
I20260709 13:14:22.287518 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.288673 28156 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:22.289245 28755 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
W20260709 13:14:22.275275  2403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.012s	sys 0.025s
I20260709 13:14:22.303131 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.305708 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.306638 28156 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:22.303380 28077 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2790) took 1132 ms. Trace:
I20260709 13:14:22.262087  3193 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:22.307240 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 489486us
I20260709 13:14:22.310657 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:22.308429  2403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.012s	sys 0.026s
I20260709 13:14:22.313964 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:22.308403 28077 rpcz_store.cc:276] 0709 13:14:21.171109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.179551 (+  8442us) service_pool.cc:224] Handling call
0709 13:14:21.200865 (+ 21314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:21.201135 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:21.805681 (+604546us) log.cc:844] Serialized 14 byte log entry
0709 13:14:22.302705 (+497024us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:22.302768 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:22.302843 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.302944 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.303300 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":916004,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":4823,"thread_start_us":1064299,"threads_started":2}
I20260709 13:14:22.320412 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:22.322779 28126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.009s	sys 0.009s
I20260709 13:14:22.307859  3028 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.018s
I20260709 13:14:22.307843  3181 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:22.312289  3193 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:22.330696 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:22.319059  3165 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.000s	sys 0.014s
W20260709 13:14:22.333467 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.337644 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.320174  2892 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:22.314198  3092 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:22.339464 28085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:22.343851 28085 raft_consensus.cc:606] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:22.344283 28085 raft_consensus.cc:614] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:14:22.335975 28157 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.009s	sys 0.010s
I20260709 13:14:22.321183  3273 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.318113  3312 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:14:22.335742  3165 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.000s	sys 0.015s
W20260709 13:14:22.357528 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.345359 28157 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.009s	sys 0.011s
I20260709 13:14:22.380159 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:22.381114 28759 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
W20260709 13:14:22.378970 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 323563us
I20260709 13:14:22.382515 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:22.382814 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:22.392936 28198 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.011s	sys 0.018s
I20260709 13:14:22.361609  3312 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:22.371877  3302 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.234s	user 0.000s	sys 0.002s
I20260709 13:14:22.373824  3198 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:22.356518  3179 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:22.398057 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:22.354933  3273 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.377909  2892 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:22.396674 28198 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.012s	sys 0.019s
I20260709 13:14:22.426441 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:22.396590  3194 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.004s	sys 0.027s
I20260709 13:14:22.422704 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:22.396559  3131 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:22.471709 28077 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:22.472568 28085 raft_consensus.cc:2368] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
W20260709 13:14:22.423076 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 253935us
W20260709 13:14:22.398247  3307 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.185s	user 0.001s	sys 0.001s
I20260709 13:14:22.475667 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:22.423729  3091 consensus_meta.cc:287] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.000s
I20260709 13:14:22.477504 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:22.413228  3065 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:22.426824  3309 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:14:22.481956 28761 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:22.484545 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:22.478580 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1436) took 2524 ms. Trace:
W20260709 13:14:22.475363  3194 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.004s	sys 0.028s
I20260709 13:14:22.485988 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.503908 28229 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:22.471927  3211 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:22.502975 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 172436us
I20260709 13:14:22.476565  3293 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.506734 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:22.485637  3305 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.232s	user 0.001s	sys 0.001s
I20260709 13:14:22.510020 28229 raft_consensus.cc:1240] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:22.492033 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:22.492224 28157 rpcz_store.cc:276] 0709 13:14:19.953980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.279750 (+325770us) service_pool.cc:224] Handling call
0709 13:14:20.280088 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:20.432347 (+152259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:20.432396 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:20.454030 (+ 21634us) log.cc:844] Serialized 52 byte log entry
0709 13:14:20.609877 (+155847us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:14:20.704721 (+ 94844us) log.cc:844] Serialized 14 byte log entry
0709 13:14:21.016006 (+311285us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b18004458a0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce66f060 00007f59ce681716 00007f59ce692287 00007f59ce692208
0709 13:14:22.371884 (+1355878us) log.cc:844] Serialized 14 byte log entry
0709 13:14:22.478138 (+106254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.478233 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:22.478350 (+   117us) raft_consensus.cc:1685] finished
0709 13:14:22.478369 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.478521 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14867,"prepare.queue_time_us":104638,"prepare.run_cpu_time_us":1737,"prepare.run_wall_time_us":17529,"raft.queue_time_us":391070,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":29844,"spinlock_wait_cycles":5785344,"thread_start_us":1782523,"threads_started":6,"wal-append.queue_time_us":312732,"wal-append.run_cpu_time_us":2935,"wal-append.run_wall_time_us":1060232}
I20260709 13:14:22.511300 28230 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:22.481323  3291 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:22.396612  3181 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.481160  3116 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:22.513988 28234 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:14:22.490769  3164 raft_consensus.cc:3055] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:22.539283 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2793) took 1139 ms. Trace:
I20260709 13:14:22.520176 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3698) took 1310 ms. Trace:
I20260709 13:14:22.505496  3194 raft_consensus.cc:697] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:22.485440  3322 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:14:22.529119 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.548492 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.533993 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.521095  2892 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.413766  3092 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:22.535774  3310 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:14:22.491549  3065 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.485777  3131 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:22.539669 28080 rpcz_store.cc:276] 0709 13:14:21.400000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.400108 (+   108us) service_pool.cc:224] Handling call
0709 13:14:21.443404 (+ 43296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:21.443610 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:21.443638 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:21.475374 (+ 31736us) log.cc:844] Serialized 26 byte log entry
0709 13:14:22.056929 (+581555us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:22.056968 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.057043 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:22.539024 (+481981us) raft_consensus.cc:1685] finished
0709 13:14:22.539045 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.539221 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":581105,"threads_started":1,"wal-append.queue_time_us":829097}
I20260709 13:14:22.487478  3091 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.567581 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1593) took 1531 ms. Trace:
I20260709 13:14:22.491003  3176 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:14:22.549226 28157 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:22.540887 28228 rpcz_store.cc:276] 0709 13:14:21.209786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.209908 (+   122us) service_pool.cc:224] Handling call
0709 13:14:21.210218 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:21.219849 (+  9631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:21.219879 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:21.220346 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:14:21.873161 (+652815us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:21.873223 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.873330 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:22.519965 (+646635us) raft_consensus.cc:1685] finished
0709 13:14:22.519989 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.520120 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9229,"raft.queue_time_us":10180,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3341,"thread_start_us":652457,"threads_started":1,"wal-append.queue_time_us":932888}
W20260709 13:14:22.547502  3315 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:14:22.538617  3164 raft_consensus.cc:740] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:22.573241 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:22.517956  3293 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.573269 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.569921 28082 rpcz_store.cc:276] 0709 13:14:21.036000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.091809 (+ 55809us) service_pool.cc:224] Handling call
0709 13:14:21.092224 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:21.092600 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:21.092628 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:21.093220 (+   592us) log.cc:844] Serialized 52 byte log entry
0709 13:14:21.733430 (+640210us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:21.733469 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.733536 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:22.567328 (+833792us) raft_consensus.cc:1685] finished
0709 13:14:22.567350 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.567522 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":639673,"threads_started":1,"wal-append.queue_time_us":989721}
I20260709 13:14:22.577868 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:22.583588 28157 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:22.584020 28161 raft_consensus.cc:2376] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 25. Re-sending same reply.
I20260709 13:14:22.584925 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.585794 28157 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 15.
I20260709 13:14:22.586992 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.589154 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:22.590009 28763 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:22.515305  3239 consensus_queue.cc:1059] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:14:22.515090  3211 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.585089 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.592226 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:22.518239  3313 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:14:22.588366 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:22.595415 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 105406us
I20260709 13:14:22.538196  3291 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.604811 28152 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 24: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:22.553320  3179 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.606997 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.548535  3319 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:14:22.598084 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:22.624758 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1621) took 1180 ms. Trace:
I20260709 13:14:22.574695  3182 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
W20260709 13:14:22.622118  3314 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 13:14:22.542523  3116 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:22.637343  3332 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:14:22.638211 28227 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:22.629244 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:22.593999  3289 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:22.630340 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57189.5us
I20260709 13:14:22.579778  3339 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:22.627604 28758 rpcz_store.cc:276] 0709 13:14:21.444303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.445061 (+   758us) service_pool.cc:224] Handling call
0709 13:14:21.445330 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:21.466324 (+ 20994us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:21.466358 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:21.466808 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:14:22.217523 (+750715us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:22.220187 (+  2664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.220267 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:22.624509 (+404242us) raft_consensus.cc:1685] finished
0709 13:14:22.624535 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.624680 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23252,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":736286,"threads_started":1,"wal-append.queue_time_us":843454}
I20260709 13:14:22.632577 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:22.578308  3165 consensus_queue.cc:1059] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:14:22.633846  3194 consensus_queue.cc:237] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.634755  3065 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:22.614363  3334 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:22.613178  3324 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:14:22.640767 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.640470 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.641122 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:22.579258  3330 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
W20260709 13:14:22.646907  3327 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:14:22.637755  3116 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.654160 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.655197  3082 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:22.655745 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1643) took 1251 ms. Trace:
W20260709 13:14:22.655807  3342 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:14:22.200987  3299 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.517607  3312 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.657270  3301 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:22.658334  3293 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:22.633445  3239 raft_consensus.cc:3055] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:22.661237  3065 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.662253  3211 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:22.652714  3324 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.662768 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1644) took 1258 ms. Trace:
I20260709 13:14:22.666548 28227 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:22.661100 28124 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:22.738368 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 77353.5us
I20260709 13:14:22.737483 28235 rpcz_store.cc:276] 0709 13:14:21.404638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.404905 (+   267us) service_pool.cc:224] Handling call
0709 13:14:21.405184 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:21.405408 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:14:21.810830 (+405422us) log.cc:844] Serialized 14 byte log entry
0709 13:14:22.662201 (+851371us) spinlock_profiling.cc:243] Waited 9.8 ms on lock 0x7b18003d6fc0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59ce21f4bc 00007f59cb56d48b 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a
0709 13:14:22.662407 (+   206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:22.662461 (+    54us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:22.662512 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.662584 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.662710 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":847725,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":4410,"raft.queue_time_us":72048,"spinlock_wait_cycles":22529280,"thread_start_us":826910,"threads_started":2,"wal-append.queue_time_us":586270}
I20260709 13:14:22.667021 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:22.668138  3240 consensus_meta.cc:287] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.963s	user 0.011s	sys 0.001s
I20260709 13:14:22.669339 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:22.670495 28231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3712) took 1232 ms (client timeout 1600 ms). Trace:
I20260709 13:14:22.732138  3239 raft_consensus.cc:740] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:22.742940 28231 rpcz_store.cc:269] 0709 13:14:21.438101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.448841 (+ 10740us) service_pool.cc:224] Handling call
0709 13:14:22.670230 (+1221389us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b54000e8654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:22.670424 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1608786304}
I20260709 13:14:22.640034  3131 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.744201  3239 consensus_queue.cc:260] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.745726  3239 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:22.729046  3273 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:22.650867  3339 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:22.678236  3107 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.001s
I20260709 13:14:22.683403  3164 consensus_queue.cc:260] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:22.673509  3323 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.196s	user 0.001s	sys 0.001s
I20260709 13:14:22.705487 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1598) took 1174 ms. Trace:
I20260709 13:14:22.739080  3324 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:22.707171 28728 leader_election.cc:336] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.674s (SENT)
W20260709 13:14:22.721616 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 92589.1us
W20260709 13:14:22.794113 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 86977.4us
I20260709 13:14:22.794382 28153 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:22.722575  3193 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:22.723717 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.724164 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 217523us
W20260709 13:14:22.710143 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 65224.1us
I20260709 13:14:22.707068 28232 rpcz_store.cc:276] 0709 13:14:21.404343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.404833 (+   490us) service_pool.cc:224] Handling call
0709 13:14:21.443628 (+ 38795us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:21.443944 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:21.443982 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:21.456521 (+ 12539us) log.cc:844] Serialized 26 byte log entry
0709 13:14:21.618926 (+162405us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b180033dba0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce2170d1 00007f59ce632a0f
0709 13:14:22.195625 (+576699us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:22.195687 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.195755 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:22.651283 (+455528us) raft_consensus.cc:1685] finished
0709 13:14:22.651309 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.655666 (+  4357us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1156,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":62971,"spinlock_wait_cycles":14994688,"thread_start_us":576487,"threads_started":1,"wal-append.queue_time_us":857011}
I20260709 13:14:22.727183 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:22.651635  3334 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:22.704285  3165 raft_consensus.cc:3055] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:22.711113  3082 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:22.725560  3116 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:14:22.673609  3304 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.543s	user 0.005s	sys 0.001s
I20260709 13:14:22.702775  3176 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:14:22.812520  3082 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:22.817636  3330 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:22.818009  3330 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:22.819036  3330 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.820425  3330 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:22.775071 28086 rpcz_store.cc:276] 0709 13:14:21.531068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.560911 (+ 29843us) service_pool.cc:224] Handling call
0709 13:14:21.561397 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:21.601230 (+ 39833us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:21.601263 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:21.601724 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:14:22.181822 (+580098us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:22.181885 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.181981 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:22.705195 (+523214us) raft_consensus.cc:1685] finished
0709 13:14:22.705227 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.705409 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39382,"raft.queue_time_us":63794,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":56829,"thread_start_us":579749,"threads_started":1,"wal-append.queue_time_us":750112}
I20260709 13:14:22.837321 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:22.838199 28227 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:22.839084 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:22.705533 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.841444 28158 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:22.841583 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.842701 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.842890  3182 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:22.843284 28157 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:22.843488  3182 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:22.844193 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.746721  3347 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:22.777215  3227 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.845058  3347 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:22.845273  3182 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.845993  3347 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.846935  3182 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:22.855422  3205 raft_consensus.cc:3055] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:22.855887  3205 raft_consensus.cc:740] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:22.857779  3112 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:22.858332  3112 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:22.812270  3291 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.738926  3211 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:22.785604 28155 raft_consensus.cc:2410] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:14:22.779783 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.750502  3293 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:22.789237 28050 leader_election.cc:336] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.503s (SENT)
I20260709 13:14:22.737365  3065 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.801282 28198 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:22.784904  3164 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:22.805989 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:22.888011 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 86812.6us
I20260709 13:14:22.851553  3082 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.807492  3165 raft_consensus.cc:740] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:22.807406  3354 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:22.786275  3355 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.813974 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.817937 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:22.790215  3091 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.731892  3301 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:22.733314  3322 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:22.821482  3116 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:22.822013 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.822346 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.822677 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.905779 28235 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:22.906801 28234 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:22.907348 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:22.907621 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.823076 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.908478 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:22.909284 28235 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 23: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:22.825187 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.822379 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:22.826288  3337 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:14:22.913713 28235 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:14:22.896549  3331 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:22.832022 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1630) took 1102 ms. Trace:
I20260709 13:14:22.834003  3339 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:22.917296 28235 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:22.835212  3272 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.835783  3193 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.778589 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3355) took 1091 ms. Trace:
I20260709 13:14:22.920447 28159 rpcz_store.cc:276] 0709 13:14:21.687440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.691837 (+  4397us) service_pool.cc:224] Handling call
0709 13:14:21.717638 (+ 25801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:21.717869 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:14:22.010012 (+292143us) log.cc:844] Serialized 14 byte log entry
0709 13:14:22.543244 (+533232us) log.cc:844] Serialized 14 byte log entry
0709 13:14:22.778200 (+234956us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:22.778270 (+    70us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:22.778321 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.778385 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.778521 (+   136us) inbound_call.cc:177] Queueing success response
0709 13:14:22.828953 (+ 50432us) spinlock_profiling.cc:243] Waited 7.89 ms on lock 0x7b18004891c0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb58a3f3 00007f59cb58c7c1 00007f59cb58c779 00007f59cb58c709 00007f59cb58c6d1 00007f59cb58b9cd 000000000038f354 000000000038f199 00007f59cb56d83a
Metrics: {"prepare.queue_time_us":217667,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":308126,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":21135,"spinlock_wait_cycles":22856192,"thread_start_us":747359,"threads_started":3,"wal-append.queue_time_us":512103}
I20260709 13:14:22.922982 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.865206 28157 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:22.866364 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:22.876856 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:14:22.925576 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 136468us
I20260709 13:14:22.925333  3301 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:22.877596 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.926671  3176 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:22.888907 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 83016.1us
I20260709 13:14:22.927613  3176 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:22.928236  3176 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.806581  3348 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:14:22.930953 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.931025  3176 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.889451 28198 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:22.890314  3291 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.932931  3165 consensus_queue.cc:260] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.933352  3291 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.934371  3165 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:22.892802  3357 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:14:22.863708  3158 consensus_meta.cc:287] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.669s	user 0.006s	sys 0.007s
I20260709 13:14:22.897697 28156 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16.
I20260709 13:14:22.944437  3255 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.944959  3255 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.945165 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:22.946880 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:22.947649 28156 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:22.897805 28232 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:22.887418  3293 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.908712  3355 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.896720  3082 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.899135 28233 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:14:22.910830 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:22.911775 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.910840 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.912318 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.915210 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.914997 28757 rpcz_store.cc:276] 0709 13:14:21.729255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.729445 (+   190us) service_pool.cc:224] Handling call
0709 13:14:21.766519 (+ 37074us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:21.965664 (+199145us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:14:21.965704 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:21.966245 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 13:14:22.314220 (+347975us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:14:22.314267 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.831447 (+517180us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54006c0f54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:22.831514 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:22.831622 (+   108us) raft_consensus.cc:1685] finished
0709 13:14:22.831648 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.831942 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6192,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":456776064,"thread_start_us":347028,"threads_started":1,"wal-append.queue_time_us":518431}
I20260709 13:14:22.810485  3334 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:22.882642 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:22.892159  3181 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:22.889928  3359 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:22.956879 28226 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:22.957990 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.926465 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.948719 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:22.904702  3354 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:22.911294  3350 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:22.887153  3211 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.954566 28228 raft_consensus.cc:2393] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 21: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:22.975517  3350 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.976863 28228 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:22.977003  3211 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:22.977715  3350 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.978724 28228 raft_consensus.cc:1240] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:14:22.978924  3359 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:22.979280 27975 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 21 to 22, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 22 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:22.955207 28231 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:22.957211 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 162294us
I20260709 13:14:22.955971 28229 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:22.926540  3301 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:22.992159 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:22.949450  3303 consensus_meta.cc:287] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.729s	user 0.006s	sys 0.004s
I20260709 13:14:22.955713 28234 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:14:22.953581 28233 pending_rounds.cc:85] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
W20260709 13:14:22.960089 28050 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.593s (SENT)
I20260709 13:14:22.938521 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:22.864660  3205 consensus_queue.cc:260] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:22.996816 28050 leader_election.cc:336] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.606s (SENT)
I20260709 13:14:22.997354 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:22.997670  3205 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:22.963958 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:22.964170  3354 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:22.915489  3116 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.944218  3065 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:22.969849  3355 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.970001 28156 raft_consensus.cc:2376] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 16. Re-sending same reply.
I20260709 13:14:22.971190 28232 raft_consensus.cc:2410] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:14:22.973007  3322 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:22.987493  3349 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:22.988371 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:22.989223 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:23.025020 28757 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 24: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:22.991310 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:23.025586 28756 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:23.027230 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.028126 28157 raft_consensus.cc:2410] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:14:22.995574 28233 pending_rounds.cc:107] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:14:23.028942 28233 raft_consensus.cc:2889] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:22.995958 28235 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 26
I20260709 13:14:23.029943 28764 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35892 (request call id 1285) took 1633 ms. Trace:
I20260709 13:14:23.030462 28764 rpcz_store.cc:276] 0709 13:14:21.396301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.396799 (+   498us) service_pool.cc:224] Handling call
0709 13:14:23.029685 (+1632886us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54006db854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:23.029875 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938292608}
I20260709 13:14:23.031665  3293 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:22.944243  3291 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:22.999013 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.000399 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 145348us
W20260709 13:14:23.036065 27977 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 279) took 46633 ms (client timeout 15000 ms). Trace:
I20260709 13:14:23.037030  3348 consensus_queue.cc:1059] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:23.000991 28226 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
W20260709 13:14:23.001281 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 111951us
I20260709 13:14:23.038503  3313 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:23.003223 28755 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
W20260709 13:14:23.006409 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.039868  3330 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:23.040076 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.040329  3330 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:23.017074 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:23.040922 28158 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 25.
W20260709 13:14:23.036569 27977 rpcz_store.cc:269] 0709 13:13:36.403162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.403449 (+   287us) service_pool.cc:224] Handling call
0709 13:14:23.035957 (+46632508us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:14.245326 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:14.246164 (+   838us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:14.246228 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:14:14.246268 (+    40us) tablet.cc:662] Decoding operations
0709 13:14:14.247951 (+  1683us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:14.248026 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:14.248056 (+    30us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:14:14.266368 (+ 18312us) tablet.cc:701] Row locks acquired
0709 13:14:14.266442 (+    74us) write_op.cc:260] PREPARE: finished
0709 13:14:14.266659 (+   217us) write_op.cc:270] Start()
0709 13:14:14.266833 (+   174us) write_op.cc:276] Timestamp: P: 1783602854266581 usec, L: 0
0709 13:14:14.266853 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:14:14.273552 (+  6699us) log.cc:844] Serialized 7130 byte log entry
0709 13:14:15.482442 (+1208890us) op_driver.cc:464] REPLICATION: finished
0709 13:14:15.916890 (+434448us) write_op.cc:301] APPLY: starting
0709 13:14:15.921618 (+  4728us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:16.382644 (+461026us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:16.382759 (+   115us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:16.607387 (+224628us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:16.607859 (+   472us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:14:16.607916 (+    57us) write_op.cc:312] APPLY: finished
0709 13:14:16.608469 (+   553us) log.cc:844] Serialized 164 byte log entry
0709 13:14:16.766919 (+158450us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:16.767116 (+   197us) write_op.cc:454] Released schema lock
0709 13:14:16.767197 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:16.767763 (+   566us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:16.767792 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":164127,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":246004,"apply.run_cpu_time_us":127680,"apply.run_wall_time_us":865444,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79619,"num_ops":18,"prepare.queue_time_us":358990,"prepare.run_cpu_time_us":42426,"prepare.run_wall_time_us":253308,"raft.queue_time_us":287841,"raft.run_cpu_time_us":9622,"raft.run_wall_time_us":241728,"replication_time_us":1196147,"thread_start_us":960943,"threads_started":5,"wal-append.queue_time_us":871561,"wal-append.run_cpu_time_us":49210,"wal-append.run_wall_time_us":2813624}]]}
I20260709 13:14:23.041318  3330 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:23.042421 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.042644 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.043752 28199 leader_election.cc:400] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
I20260709 13:14:23.043856 28159 raft_consensus.cc:2410] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 13:14:23.044061 28199 leader_election.cc:403] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:14:23.046295  3176 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:23.048292  3239 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:23.038791 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:23.041563 28758 raft_consensus.cc:2410] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:14:23.036069 28760 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:23.055577 28760 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1631) took 1287 ms. Trace:
I20260709 13:14:23.057977  3313 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:23.056144 28760 rpcz_store.cc:276] 0709 13:14:21.767720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.767961 (+   241us) service_pool.cc:224] Handling call
0709 13:14:23.035774 (+1267813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:23.054102 (+ 18328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:23.055286 (+  1184us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54006db850. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:23.055488 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2914933504}
I20260709 13:14:23.038223  3348 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:14:22.981314  3359 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:22.981036 27975 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 24. New cstate: current_term: 24 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.075007 27975 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.003805  3181 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:23.072789 28756 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:23.080420 28227 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:23.092892 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.003861  3303 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:23.062188  3255 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:23.005162  3082 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:23.103495  3293 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:23.093964 28227 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:23.005323  3300 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:23.104878  3300 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:23.063159  3330 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:23.105386  3300 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:23.106364  3116 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:22.941579  3331 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:23.100281 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56716.1us
W20260709 13:14:23.101308 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 62598.3us
I20260709 13:14:23.109920  3359 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
I20260709 13:14:23.110445 28198 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:23.099876  2892 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:14:23.111946 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:23.112947  3291 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:23.096093  3223 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.108703 27975 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 18, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 18 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:23.114159  3223 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:23.024271  3349 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:23.125123 28154 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:23.122139 28198 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:23.126127 28235 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 26.
I20260709 13:14:23.123456 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:23.123900 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.128472 28156 raft_consensus.cc:2393] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 19: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:23.078691 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:23.093376  3357 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.133478 28198 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:23.133939 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55331.4us
I20260709 13:14:23.078706  3354 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:23.076395  3324 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:23.141440 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:23.141774 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:23.130851 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:23.145488 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.150738 28156 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:23.126225  3318 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:23.128014  3116 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:23.155287 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:23.152316  3181 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:23.137307  3131 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:23.139722  3182 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:23.156711 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:23.137182  3303 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:23.157953 28231 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:14:23.154595 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:23.123582  3112 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:23.153903  3190 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:23.155725  3318 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:23.161818  3324 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:23.163475 28756 raft_consensus.cc:2393] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 21: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:23.166095 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:23.144359 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:23.166429  3019 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:23.172569  3019 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.172292 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:23.172989  3019 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:23.173497  3291 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:23.173056  3318 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:23.173959 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:23.174806  3318 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:23.175186 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:23.175410  3347 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:23.176151  3347 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:23.178342 28232 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:23.179308  3255 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:23.179975  3347 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:23.179849  3255 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:23.165648  3190 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.180812  3190 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:23.182061  3255 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:23.164578  3303 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:23.183848  3303 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:23.189057 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.194706 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:23.197404 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:23.202282 28234 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:23.114652 27975 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.209184 28156 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:23.238565 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:23.238895 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 72871us
I20260709 13:14:23.208328 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:23.207144 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:23.240082 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:23.147696  3357 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:23.237587 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 43804.1us
I20260709 13:14:23.240329 28079 raft_consensus.cc:3060] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:23.240352 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.244338 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.237989 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 93709.6us
I20260709 13:14:23.246032 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:23.245368 28050 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:23.247524 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:23.247794 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 152744us
I20260709 13:14:23.170531  3319 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:23.248557 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:23.207862 28086 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:14:23.247002 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:23.208086 28127 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:23.202086  3291 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:23.175007  3158 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.066s
I20260709 13:14:23.254958 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:23.258810 28084 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:23.259740 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:23.255740 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 47783.2us
I20260709 13:14:23.259886 28086 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:14:23.243624  3131 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.259804 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:23.272198 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 102349us
I20260709 13:14:23.268297 28127 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:23.236606  3081 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:23.283005 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:23.283632 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.251163  3324 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:23.246541  3313 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:23.254647 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:23.241410  3312 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:23.290035 28078 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:14:23.300987 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:23.257509  3293 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:23.303603 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3364) took 1362 ms. Trace:
W20260709 13:14:23.304914 28728 leader_election.cc:336] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.840s (SENT)
I20260709 13:14:23.329864 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:23.249213  3319 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:23.310281 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3376) took 1047 ms. Trace:
I20260709 13:14:23.296137  3158 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:23.302407  3331 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:23.261835  3322 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:23.314764  3179 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:23.316705 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.262903  3346 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:23.237282  3334 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:23.318545 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 63952.1us
I20260709 13:14:23.319937 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.374083 28159 raft_consensus.cc:2368] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 21. Current term is 22.
I20260709 13:14:23.322242 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.187109  3349 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:23.321002 28124 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:23.253530  3223 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:23.329344 28153 rpcz_store.cc:276] 0709 13:14:21.941078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.979527 (+ 38449us) service_pool.cc:224] Handling call
0709 13:14:23.303412 (+1323885us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b5400053554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:23.303537 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1780759424}
I20260709 13:14:23.331308 28154 rpcz_store.cc:276] 0709 13:14:22.263014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:22.266726 (+  3712us) service_pool.cc:224] Handling call
0709 13:14:23.310088 (+1043362us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b540033e854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:23.310203 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1888131456}
W20260709 13:14:23.375662 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 127188us
I20260709 13:14:23.311237  3291 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:23.297550  3372 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:23.313715 28079 raft_consensus.cc:2468] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:23.314878  3339 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:23.316071  3181 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:23.282312  2792 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:14:23.316474 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.329929  3116 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:23.406493 28232 raft_consensus.cc:1275] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 13:14:23.359362 28158 raft_consensus.cc:2368] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19.
I20260709 13:14:23.406509 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:23.331965  3194 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:14:23.360311 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 55727.1us
I20260709 13:14:23.388182 28157 raft_consensus.cc:2376] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22. Re-sending same reply.
W20260709 13:14:23.412137 28728 leader_election.cc:336] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 1.879s (SENT)
I20260709 13:14:23.412807 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:23.329264  3293 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.372227 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.329418  3355 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:23.331570  3347 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:14:23.347743  3158 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:23.313014  3131 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:23.377444 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 56511.3us
W20260709 13:14:23.380183 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 90005.5us
I20260709 13:14:23.380824 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.443866 28153 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:23.418655  3223 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:23.398324 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:23.398749 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:14:23.457513  3369 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:14:23.407486 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.458969 28156 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:23.407877 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.408627 28232 pending_rounds.cc:85] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
I20260709 13:14:23.461349 28232 pending_rounds.cc:107] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:14:23.461700 28232 raft_consensus.cc:2889] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:23.413165 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:23.415354 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.464134 28157 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:23.464511 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.421188 28161 raft_consensus.cc:2393] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 24: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:23.376688 28050 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:23.359972  3346 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:23.476595  3346 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:23.477160 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.445273  3112 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:23.485064  3158 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:23.486441  3158 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:23.487300  3382 consensus_queue.cc:1059] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:23.426170 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.436786  3349 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:23.423270  3293 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:23.386546  3181 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:23.377444  3372 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:23.424602  3368 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:14:23.377300  3272 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:23.511526 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:23.416843  3194 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.511022  3193 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.541079  3194 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:23.541361  3193 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:23.428932  3365 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:14:23.417704  3091 raft_consensus.cc:2804] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:23.406812  3339 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:23.544952  3339 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:23.546867  3354 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:23.444890 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.445016 28078 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:23.410414  3318 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:23.381881 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.435974  3116 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:23.450448  3019 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:23.451423  3367 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:14:23.371794 27975 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:23.452339  3373 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:14:23.372483  3179 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:23.450495  3366 log.cc:927] Time spent T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:14:23.399411 28230 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:23.465202 28158 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:23.465086 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:23.474112 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.475776  3275 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:23.558988  3091 raft_consensus.cc:697] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:14:23.407503 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 101623us
I20260709 13:14:23.491156 28084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:23.500108 28759 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:14:23.444123  3245 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:23.593385 28226 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:23.510578  3372 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:23.489338  2792 consensus_queue.cc:1243] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:14:23.523438  3205 consensus_meta.cc:287] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.003s
I20260709 13:14:23.510669  3181 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:23.529320  3176 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:23.529843  3334 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:23.529417  3303 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:23.529306 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.620123 28159 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:23.827636 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.551046 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:23.509534 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:23.517933 28156 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:23.839804 28081 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 25.
I20260709 13:14:23.490037 28154 raft_consensus.cc:2368] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18.
I20260709 13:14:23.840854 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.842504 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:23.591027 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.843532 28154 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:23.520656 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:23.845674 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.846433 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.635664 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44311.7us
I20260709 13:14:23.743722 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:23.822502 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:23.521718 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 145117us
I20260709 13:14:23.851799 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:23.853088 28161 raft_consensus.cc:2393] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 25: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:23.541481 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:23.652486 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53899.6us
W20260709 13:14:23.584756 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.587455 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:23.550518 28153 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:23.972509 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:23.972965  3160 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:23.643105  2403 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.115s
I20260709 13:14:23.977240 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:23.978312  2403 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:23.979028  2403 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:23.980176  3239 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:23.677018  3019 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:23.534952  3131 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:23.644797  3164 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.065s
I20260709 13:14:23.984007  3164 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:14:23.696367  3165 consensus_meta.cc:287] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.742s	user 0.009s	sys 0.002s
I20260709 13:14:23.984130  3334 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:23.675477  3116 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:23.664676  3179 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:23.637513  3176 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:23.990025  3176 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:23.625465  3205 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:23.702327  3190 consensus_meta.cc:287] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.521s	user 0.005s	sys 0.003s
I20260709 13:14:23.580040  3193 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:23.995076  3160 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:23.532891  3272 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.005264  3190 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.006039  3272 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:23.762972  3291 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:14:23.712363  3301 consensus_meta.cc:287] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.000s
I20260709 13:14:24.007371  3205 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:23.666502  3303 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.008194  3190 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:23.661953  3300 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.003s
I20260709 13:14:24.009357 28228 raft_consensus.cc:2368] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19.
I20260709 13:14:24.010160  3290 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:24.009719  3300 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.011075  3290 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.012586 28232 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:24.013887 28230 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:14:24.014824 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:23.503584  3313 raft_consensus.cc:3055] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:24.017215  3313 raft_consensus.cc:740] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:24.019268  3313 consensus_queue.cc:260] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.021090  3299 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:24.021785  3299 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.024999 28758 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:14:23.680399  3329 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:24.025663 28759 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:14:24.026059  3329 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:24.027644 28762 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:14:24.027578  3329 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:23.559358  3318 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:23.998621  3131 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:24.031330  3318 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:23.689544  3319 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:23.535137  3322 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:24.033561 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:24.033742 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:24.034358 28759 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
W20260709 13:14:23.704389  3330 consensus_meta.cc:287] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.599s	user 0.008s	sys 0.004s
I20260709 13:14:23.498577  3355 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.035377 28235 raft_consensus.cc:1240] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:23.586580  3357 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:23.845851 27975 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.037225 28157 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:14:24.037521 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:24.038601 28156 raft_consensus.cc:2410] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:14:23.841213 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 290075us
I20260709 13:14:24.039381  3300 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:23.896893 28158 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:23.897275 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:24.036414  3355 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.041262 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1451) took 1141 ms. Trace:
I20260709 13:14:24.041734 28159 rpcz_store.cc:276] 0709 13:14:22.899710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.896747 (+997037us) service_pool.cc:224] Handling call
0709 13:14:24.041191 (+144444us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:24.041822 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:24.043004  3355 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:24.043656 28159 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:24.044361 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3382) took 1686 ms. Trace:
I20260709 13:14:23.902994 28152 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 19. (term mismatch)
I20260709 13:14:24.044878 28152 pending_rounds.cc:85] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 18
I20260709 13:14:24.045116 28152 pending_rounds.cc:107] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 27.19
I20260709 13:14:24.045385 28152 raft_consensus.cc:2889] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:24.046319 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3380) took 1711 ms. Trace:
I20260709 13:14:24.046617 28152 rpcz_store.cc:276] 0709 13:14:22.335375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.899830 (+1564455us) service_pool.cc:224] Handling call
0709 13:14:23.900712 (+   882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.046055 (+145343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.046246 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:23.774266  3293 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:23.762200  3091 consensus_queue.cc:237] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:23.918157  3359 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.923965  3358 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:23.931113  3372 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.036618 28155 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:23.993392  3164 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:23.937988 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:23.939199 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:23.966436  3379 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:23.967401 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 383819us
I20260709 13:14:23.982017 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:24.034554 28755 raft_consensus.cc:2368] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 25. Current term is 26.
I20260709 13:14:23.654594  3346 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.038465 27975 catalog_manager.cc:5697] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 19, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 19 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.880520 28082 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:24.048676 28152 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:14:24.044668 28159 rpcz_store.cc:276] 0709 13:14:22.357837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.042900 (+1685063us) service_pool.cc:224] Handling call
0709 13:14:24.043419 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.044059 (+   640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.044273 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:23.937125 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 85446.1us
I20260709 13:14:24.035347  3290 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.005117  3385 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.024531  3299 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.076583 28154 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:24.070112  3318 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.092381  3303 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:24.093489 28153 raft_consensus.cc:2368] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 23. Current term is 24.
I20260709 13:14:24.095419  3293 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:24.095958 28226 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:24.096933  3379 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.097522 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43184 (request call id 1455) took 1197 ms. Trace:
I20260709 13:14:24.099306 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3405) took 1352 ms. Trace:
I20260709 13:14:24.151866 28156 rpcz_store.cc:276] 0709 13:14:22.746547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.840112 (+1093565us) service_pool.cc:224] Handling call
0709 13:14:24.099030 (+258918us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54003935d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:24.099234 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451841920}
I20260709 13:14:24.102766  3160 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:24.152872  3348 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.151095 28157 rpcz_store.cc:276] 0709 13:14:22.900335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.464833 (+564498us) service_pool.cc:224] Handling call
0709 13:14:24.097265 (+632432us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b5400237654. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:24.097443 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1048712064}
I20260709 13:14:24.153533  3348 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.158903  3346 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:24.159731  3379 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.161703 28084 raft_consensus.cc:606] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:24.163313 27975 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 24, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 24 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.167295 27975 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.169370  3164 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.171660  3164 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:24.172132 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:24.091490  3205 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:24.103408  3399 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:14:24.106685 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:24.107059 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:24.108018 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:24.108202 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:24.109221 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:24.108979  3359 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.108606 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:24.111948 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:24.086404  3329 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:24.122020  3401 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:24.123493  3313 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:24.123185  3330 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:24.125150 28234 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:24.170435  3379 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:14:24.130205 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:24.124226  3382 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:24.029723  3131 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.131378 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:24.146504 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2955) took 1617 ms. Trace:
I20260709 13:14:24.145162 28079 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:14:24.201869 28152 rpcz_store.cc:276] 0709 13:14:22.524375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.047821 (+1523446us) service_pool.cc:224] Handling call
0709 13:14:24.048298 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:24.133084 (+ 84786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.141995 (+  8911us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:24.180117 28161 raft_consensus.cc:2393] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:24.181062  3179 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:24.181578 28755 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 19: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:14:24.181728  3275 consensus_meta.cc:287] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.007s
I20260709 13:14:24.181821  3160 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.173684  3190 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:24.182703  3385 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:24.187237 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 148779us
I20260709 13:14:24.198698 28234 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:24.124938 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:24.204712 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2969) took 1041 ms. Trace:
I20260709 13:14:24.210821 28153 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:24.212097  3190 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.213372 28759 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 19: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:24.214120  3160 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.130452  2403 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.217077 28158 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 25.
I20260709 13:14:24.212097  3318 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.218395  2403 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:24.218874 28158 raft_consensus.cc:1240] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:24.219476  3182 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.219784  3318 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.220207  3182 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.211483  3303 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:24.221175  3160 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.221436  3303 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.222630  3182 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.219542 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1449) took 1320 ms. Trace:
W20260709 13:14:24.226157 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 163469us
I20260709 13:14:24.224681 28158 rpcz_store.cc:276] 0709 13:14:22.899165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.218214 (+1319049us) service_pool.cc:224] Handling call
0709 13:14:24.218705 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.219301 (+   596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.219492 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:24.209839 28161 rpcz_store.cc:276] 0709 13:14:23.163062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.888796 (+725734us) service_pool.cc:224] Handling call
0709 13:14:24.204655 (+315859us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:24.230808 28229 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 13:14:24.232713 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.234500 28758 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:14:24.231894 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1442) took 1333 ms. Trace:
I20260709 13:14:24.235963 28153 rpcz_store.cc:276] 0709 13:14:22.891790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.191654 (+1299864us) service_pool.cc:224] Handling call
0709 13:14:24.203991 (+ 12337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.224970 (+ 20979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.225246 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:24.251250 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 79236.2us
I20260709 13:14:24.217180  3359 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:24.234422 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:24.252763 28728 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Timed out: RequestConsensusVote RPC to 127.26.83.194:38757 timed out after 2.111s (SENT)
I20260709 13:14:24.271157 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:24.272352 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 2958) took 1475 ms. Trace:
I20260709 13:14:24.272864 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:24.274423 28155 rpcz_store.cc:276] 0709 13:14:22.796918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.476907 (+679989us) service_pool.cc:224] Handling call
0709 13:14:24.267848 (+790941us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b5400237154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:24.272282 (+  4434us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284003584}
I20260709 13:14:24.282470  3080 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.284786  3080 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.287626 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:24.288691 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.291108  3322 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:24.292079  3322 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:24.292824  3334 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:24.294054 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 91219.3us
I20260709 13:14:24.294909 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:24.296985  3358 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:24.298184  3019 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:24.299052  3318 consensus_queue.cc:1059] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:24.299602 28234 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.300838  3318 raft_consensus.cc:3055] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:24.301200 28234 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:24.301402  3318 raft_consensus.cc:740] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:24.302704 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.305109 28082 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:24.296519 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:24.314854  3359 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:24.315573  3359 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.315786  3385 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.316200  3359 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:24.317382 28157 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:24.312057  3303 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.267354  3275 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.319687 28156 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:24.321368  3275 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.322490 28156 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:24.322319  3318 consensus_queue.cc:260] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.323947  3312 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:24.324250  3318 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:24.316692 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:24.325414 28160 raft_consensus.cc:2410] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:14:24.336619 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1448) took 1440 ms. Trace:
I20260709 13:14:24.250137 27975 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 13 to 17, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 17 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:24.324496 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:24.299177  3181 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.665s	user 0.013s	sys 0.011s
I20260709 13:14:24.337170 28152 rpcz_store.cc:276] 0709 13:14:22.896423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.206794 (+1310371us) service_pool.cc:224] Handling call
0709 13:14:24.257575 (+ 50781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.310269 (+ 52694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.310304 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:24.317217 (+  6913us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:24.336357 (+ 19140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.336432 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:24.336553 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:24.330996 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:24.331197 28198 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:24.323846  3275 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.332890  3223 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:24.324615  3312 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:24.359309 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 120409us
W20260709 13:14:24.361673  3194 consensus_meta.cc:287] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.820s	user 0.019s	sys 0.004s
I20260709 13:14:24.362239  3194 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.362938 28728 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:24.365124  3245 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:24.365629  3080 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.366215 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:24.365705 27975 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 11 to 16, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 16 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.367022 28755 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:24.368944 28762 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:14:24.369206  3400 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.156s	user 0.000s	sys 0.002s
I20260709 13:14:24.371062 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:24.353525  3181 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:24.373600 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.373555 28086 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:24.374452 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:24.376201  3181 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.376420  3165 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:24.378722  3239 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:24.379129  3239 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:24.379233  3354 consensus_meta.cc:287] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.734s	user 0.012s	sys 0.013s
I20260709 13:14:24.381474 28161 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:14:24.383008 28153 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:14:24.385228 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 205763us
I20260709 13:14:24.386600 28156 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:14:24.388082 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1454) took 1487 ms. Trace:
I20260709 13:14:24.388916 28158 rpcz_store.cc:276] 0709 13:14:22.900184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.241198 (+1341014us) service_pool.cc:224] Handling call
0709 13:14:24.311936 (+ 70738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.379988 (+ 68052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.380024 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:24.380075 (+    51us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:24.387831 (+  7756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.387901 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:24.388015 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:24.385921 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2964) took 1272 ms. Trace:
I20260709 13:14:24.391346 28161 rpcz_store.cc:276] 0709 13:14:23.113611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.235516 (+1121905us) service_pool.cc:224] Handling call
0709 13:14:24.311079 (+ 75563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.382764 (+ 71685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.385866 (+  3102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:24.381129  3312 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:24.391469 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:24.382947  3299 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:24.387995  3346 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.384569  3293 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:24.368052 27975 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 24. New cstate: current_term: 24 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.412159 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.407541  3290 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:24.395938  3080 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:24.406587 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:24.425424 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.427259  3390 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.427989  3390 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.431921 27975 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 19 to 22, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 22 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:24.441198 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:24.442633 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:24.442842 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:24.443737 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:24.415026  3293 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:24.458930 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 162524us
I20260709 13:14:24.443764 28226 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
W20260709 13:14:24.446610 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53088us
I20260709 13:14:24.418269  3322 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.409829  3303 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:24.418846  3290 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.473081  3290 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:24.474080 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2959) took 1670 ms. Trace:
I20260709 13:14:24.474610 28154 rpcz_store.cc:276] 0709 13:14:22.803298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.150581 (+1347283us) service_pool.cc:224] Handling call
0709 13:14:24.155846 (+  5265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.156435 (+   589us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:24.393586 (+237151us) log.cc:844] Serialized 14 byte log entry
0709 13:14:24.473611 (+ 80025us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:24.473675 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:24.473732 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.473804 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:24.473962 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":308430,"raft.queue_time_us":106,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":259997,"threads_started":2,"wal-append.queue_time_us":53845}
W20260709 13:14:24.481737 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 88813.3us
I20260709 13:14:24.483637 28227 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:24.484144 28084 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:24.463497  3293 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:24.423625  3080 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:24.487519 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:24.423988  3299 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:24.491600 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 245566us
I20260709 13:14:24.394791  3239 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.490340 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:24.479605  3205 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:24.485483  3300 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:24.501430 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:24.500300 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:24.499019 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 105323us
I20260709 13:14:24.499800 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:24.504508 28157 raft_consensus.cc:2410] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:14:24.511673 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.512974  3300 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:24.513398 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.514130  3300 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.514940 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.505666  3303 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.517230  3194 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.517894  3312 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.518455  3303 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.519200 28085 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 25. Current term is 26. Ops: []
I20260709 13:14:24.523347  3312 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.523694 28050 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:24.525842  3312 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:24.527915 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.530406 28161 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:24.529995 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.531698 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:24.532471  3239 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:24.532899 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:24.533768 28152 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:24.534190 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:24.535808 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:24.491331  3322 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:24.538115  3413 log.cc:927] Time spent T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.080s	user 0.001s	sys 0.001s
I20260709 13:14:24.538388  3348 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.538575 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.539619 28084 raft_consensus.cc:2393] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 22: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:24.542393 28231 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:24.482446  3346 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.497623  3382 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:24.544628 28235 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:24.545207  3382 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.545764  3382 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:24.544647 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:24.516263 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.544844  3357 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.557860  3357 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.559017 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.560230  3357 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:24.560598 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.561445 28084 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:24.563086  3194 consensus_queue.cc:1048] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:24.565490 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:24.566514 28233 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:24.566646 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:24.568585 28077 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:24.568925  3299 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.569248  3080 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.569973  3255 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:24.570408  3255 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.570766  3255 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:24.571120 28127 leader_election.cc:400] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19.
I20260709 13:14:24.496711  3390 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.573151 28198 leader_election.cc:400] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 25. Current term is 26.
I20260709 13:14:24.573496 28198 leader_election.cc:403] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:24.532498 28161 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 17: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:24.573860 28154 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:24.424827 28081 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
I20260709 13:14:24.581601 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:24.584920 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 103391us
I20260709 13:14:24.532907  3312 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:24.582846 28156 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:24.588798 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:24.592543 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:24.595463 28234 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:14:24.570425  3131 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:24.597505 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 100970us
I20260709 13:14:24.434872 27975 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 18, now has a pending config: VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193), VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194), VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195), VOTER 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 18 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } } pending_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:24.596099 28234 pending_rounds.cc:85] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 15
I20260709 13:14:24.612169 28234 pending_rounds.cc:107] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:14:24.600965  3116 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 25. Current term is 26.
I20260709 13:14:24.612542 28234 raft_consensus.cc:2889] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:24.602690  3176 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:14:24.614439 28160 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3360) took 2844 ms (client timeout 1667 ms). Trace:
I20260709 13:14:24.614799  3176 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:24.615321  3176 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:24.605007 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.606910  3223 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:24.617285 28159 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:14:24.602277 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 1606) took 1225 ms. Trace:
I20260709 13:14:24.607867  3190 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.616914  3372 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.617379  3223 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.606549 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:24.614943 28160 rpcz_store.cc:269] 0709 13:14:21.769699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.769867 (+   168us) service_pool.cc:224] Handling call
0709 13:14:24.614141 (+2844274us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b5400343fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:24.614336 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5746558336}
I20260709 13:14:24.612493 27975 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 20, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 20 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:24.617813 28159 pending_rounds.cc:85] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 12
I20260709 13:14:24.624080 28159 pending_rounds.cc:107] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:14:24.624365 28159 raft_consensus.cc:2889] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:24.617946 28079 rpcz_store.cc:276] 0709 13:14:23.380691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.406226 (+ 25535us) service_pool.cc:224] Handling call
0709 13:14:24.602027 (+1195801us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b540017c354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:24.602197 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651918208}
I20260709 13:14:24.622004 28080 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:24.630622 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:24.619226  3357 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:24.636392 28152 raft_consensus.cc:2468] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:24.620081  3390 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:24.623313  3223 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:24.645110 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:24.650873 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 58980.7us
I20260709 13:14:24.657249 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1609) took 1232 ms. Trace:
I20260709 13:14:24.657722 28086 rpcz_store.cc:276] 0709 13:14:23.425637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.425771 (+   134us) service_pool.cc:224] Handling call
0709 13:14:23.426054 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.374822 (+948768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.656892 (+282070us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b5400209254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:24.657159 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2178115456}
I20260709 13:14:24.659101 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:24.649735 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.649462 28198 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:24.670835  3312 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.673801  3346 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.675889  3275 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:24.677300  3372 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.678125  3372 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.685150 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:24.686189  3385 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:14:24.686995 28728 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:24.688094  3385 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:24.688553  3385 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.689055  3385 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:24.689656  3372 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.691514 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:24.692379 28227 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:24.616460  3176 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.693457  3176 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:24.693965 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:24.695485  3372 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:24.698356  3385 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.695916  3372 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.699985  3372 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:24.700280 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.701818 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:24.703456 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.643237  3390 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.668555  3322 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.688383 28728 leader_election.cc:400] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 23. Current term is 24.
I20260709 13:14:24.640075  3164 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.113s
I20260709 13:14:24.704697 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.754866 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3402) took 2009 ms. Trace:
W20260709 13:14:24.692386 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.778389 28159 rpcz_store.cc:276] 0709 13:14:22.745648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.150418 (+1404770us) service_pool.cc:224] Handling call
0709 13:14:24.616889 (+466471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.701462 (+ 84573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.754523 (+ 53061us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b5400343fd0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:24.754716 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066674560}
I20260709 13:14:24.715469 28081 raft_consensus.cc:606] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:24.727214 28728 leader_election.cc:403] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:14:24.780303 28081 raft_consensus.cc:614] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:14:24.723802 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.726943 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:24.709362 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 67365.9us
W20260709 13:14:24.783824 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.786898 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:24.788785 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:24.726893  2403 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.508s	user 0.001s	sys 0.011s
W20260709 13:14:24.715559  3313 consensus_meta.cc:287] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.003s
I20260709 13:14:24.793926 28079 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:14:24.797906 28086 raft_consensus.cc:2410] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:14:24.714265  3329 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.004s
W20260709 13:14:24.723090  3418 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:14:24.805253 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.712169  3116 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:24.807166 28082 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:24.808372 28125 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:24.709714  3390 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:24.814782 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.813700 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:24.708666  3357 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:24.705116  3385 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.709758  3181 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:24.816998 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.710409  3131 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:24.818229 28158 raft_consensus.cc:2410] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 13:14:24.713762  3205 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:24.709527  3359 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.825354 28125 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:24.826731 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.795872  3299 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.826952 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:24.827191  3275 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:24.843248 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:24.810063  3194 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.712348  3355 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.112s
I20260709 13:14:24.787611  3293 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.723491  3190 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:14:24.800498  3334 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:24.856391 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 144845us
I20260709 13:14:24.857547 28125 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:24.755697  3346 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:14:24.860782 28235 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:24.840844  3205 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:24.861281 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 68883.7us
I20260709 13:14:24.723726  3290 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.873955 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:24.874703 28152 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:24.871466 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:24.879477 28232 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:24.856338  3181 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.877557 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:24.864147  3329 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:14:24.897346 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 209170us
I20260709 13:14:24.866468  3275 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:14:24.903966 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.888823 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.901901 28155 raft_consensus.cc:2376] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 18. Re-sending same reply.
I20260709 13:14:24.871456  3080 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:24.940061 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.881678 28124 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:24.860314  3299 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:24.946058  3422 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:14:24.946346 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:24.946594  3293 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.864327  3131 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.946878 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:24.947274  3313 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:24.965185  3313 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:24.965561  3313 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:24.966454  3313 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.967545  3313 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:24.947420  3080 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.680707 27975 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 11 to 19, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 19 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.875211  3334 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.868801  3300 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:14:24.953214  2403 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:24.864104  3194 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.973182  2403 consensus_queue.cc:1243] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:14:24.968461  3382 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.973886  2403 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:24.974395  2403 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:24.975217  3160 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:14:24.975180  2403 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.975759  3160 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:24.886291  3290 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.977274 28084 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 19: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
W20260709 13:14:24.977555 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 96024.3us
W20260709 13:14:24.978600 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.979000  3301 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:14:24.981050  3372 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:24.981412 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.981956 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.982254  3194 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.982829 28086 raft_consensus.cc:2410] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:14:24.983239  3372 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.984800 28084 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:14:24.986373 28084 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:24.987212 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.987763 28084 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:24.937589  3329 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.989054  3329 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:24.969859  3080 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.981051 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.991204 28079 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:24.950907 28152 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:24.892915  3091 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:24.992834  3160 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.993772  3091 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.881939  3303 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 13:14:24.992589 28079 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:14:24.995903  3116 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:24.996214  3334 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:24.941810  3275 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:24.962395  3428 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:24.964144 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.995707 27975 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 14 to 16, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 16 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:24.997380 28082 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:24.997907 28756 raft_consensus.cc:2368] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18.
I20260709 13:14:24.994499  3091 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:24.976547  2403 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:24.961429  3255 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:24.970248  3131 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:24.999670 28082 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:24.990687  3181 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:24.994477 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 145067us
I20260709 13:14:24.996850  3275 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:24.997079  3428 consensus_queue.cc:1243] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:14:25.004942 28084 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:14:25.019134 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:25.005806 28083 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:24.884909  3388 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 23. Current term is 24.
W20260709 13:14:25.019202 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 60233.2us
I20260709 13:14:25.022099 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:25.015728 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:24.886319  3359 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:25.024355 28199 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:25.034304 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:25.041347 28125 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:25.029908 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:25.046731 28232 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:25.053236 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:25.050150 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:25.062613 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:25.064873 28158 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:14:25.065032  3160 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:25.069229 28160 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:14:24.998236 27975 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 19, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 19 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:25.070762 28161 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:14:25.057381  3359 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:25.070509 27975 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 15 to 17, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 17 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:25.071769 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.073825  3299 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:25.074970  3382 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:25.075500  3194 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:25.075986  3194 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.076359  3194 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:25.078126 28159 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:14:25.082320  3205 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.085217  3255 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:25.089218 28077 raft_consensus.cc:2410] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:14:25.091133 28156 raft_consensus.cc:1240] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:25.092512 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.093062 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:25.093204 28161 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:25.095371 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.096021 28158 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:25.097002 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.097606 28154 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:25.072324 27975 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:25.099710 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.099643 28227 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:25.059767 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.104303 28082 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:25.107208 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:24.896951  3357 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.045336  3181 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:25.049000  3322 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:25.102630 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:25.110728 28235 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:25.108094 28227 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:25.110314 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:25.118117 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:25.118348 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 68806.5us
I20260709 13:14:25.105114  3190 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:25.130018 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:25.136838 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:25.137239 28229 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:25.138636 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:25.132786 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.141165 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.120923  3176 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:25.145869 28227 raft_consensus.cc:1240] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:25.050174  3165 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.072s
I20260709 13:14:25.148129 28160 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:25.113253  3322 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:25.149041  3176 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.158507 28050 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:25.096670  3223 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.165005 28226 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:14:25.173182 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:25.138284  3205 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:25.180949 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:25.181674 28231 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:25.182284 28234 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:25.186211 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:25.186599 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:25.187106 28761 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:25.187011 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:25.187889 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:25.188297 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:25.189502 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:25.190809 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:25.191994  3431 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:25.193177 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 134685us
I20260709 13:14:25.194088 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:25.198331 28232 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:25.195302 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:25.202216 28233 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 25.
I20260709 13:14:25.197989 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:25.213928 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46911.8us
I20260709 13:14:25.214960 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.157660  3322 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:25.218417 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 46787.6us
I20260709 13:14:25.161224  3176 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:25.222690 28049 leader_election.cc:400] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18.
I20260709 13:14:25.223057 28049 leader_election.cc:403] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:14:25.142388  3181 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.225013 28758 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:25.157721  3318 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.063s
I20260709 13:14:25.216403  3239 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:25.239684 28159 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:25.247099  3312 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:25.242476 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 75237.2us
I20260709 13:14:25.222818  3359 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.248958 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53512.9us
I20260709 13:14:25.259611 28154 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
W20260709 13:14:25.266477 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 148093us
I20260709 13:14:25.235090  3091 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.275913  3131 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.294325 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:25.286471  3312 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18.
I20260709 13:14:25.293509  3181 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:25.290444  3194 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.236590  3313 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 13:14:25.308108  3329 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:25.309700  3430 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:14:25.313225 28052 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:14:25.431174 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 118081us
I20260709 13:14:25.336035 28231 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:25.347357 28235 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:25.297479 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:25.352510 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
W20260709 13:14:25.319415  3419 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:14:25.363224 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:25.366263 28080 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:25.318843  3385 raft_consensus.cc:3055] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:25.384574 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:25.317631  3223 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:25.318183  3301 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:14:25.288938  3359 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.315812  3275 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.415218 28233 raft_consensus.cc:2393] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 25: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:25.388536  3255 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
W20260709 13:14:25.419741 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.436859 28755 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 25.
W20260709 13:14:25.445628 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 148294us
W20260709 13:14:25.454725 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 70276.1us
I20260709 13:14:25.437894 28052 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:25.458360 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:25.458557  3401 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:25.461444 28158 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 22: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:14:25.462320 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 75460.6us
I20260709 13:14:25.457675 28199 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:25.469092 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:25.438831  3382 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:25.439491  3190 raft_consensus.cc:2764] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:14:25.445350  3385 raft_consensus.cc:740] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:25.454228  3303 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.469527  3390 consensus_meta.cc:287] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.002s
W20260709 13:14:25.497746 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.466950  3357 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:25.491381  3401 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:25.513653 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.513960  3401 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.515244 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.518069  3318 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:25.518687  3318 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.519376  3318 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:25.520030 28728 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:25.520958  3433 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 13:14:25.518803  3275 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:25.436252  3181 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:14:25.478668  3293 raft_consensus.cc:2804] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:25.525704 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:25.505934  3431 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:25.547685 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.466456  3131 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.553028 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.553670 28154 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:14:25.553941 28159 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:25.523885  3357 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.556125 28154 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:14:25.560716 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:25.553606 28158 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:25.529995  3428 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:25.543061  3239 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:25.528919  3441 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:14:25.567232 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47326us
W20260709 13:14:25.574918 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 116596us
I20260709 13:14:25.569432 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:25.549008  3431 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:25.581128  3431 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:25.582265  3451 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:25.524080  3390 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:25.585063 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 96727.1us
I20260709 13:14:25.586712  3390 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:25.586807 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:25.587749 28078 raft_consensus.cc:2368] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
I20260709 13:14:25.567939  3275 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:14:25.591836  3275 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.593375  3445 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:14:25.576983 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:25.551887  3293 raft_consensus.cc:697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:25.578506 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.547715  3313 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 13:14:25.581805  3385 consensus_queue.cc:260] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:25.559839 28199 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.593s (SENT) [suppressed 27 similar messages]
I20260709 13:14:25.584373  3131 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:25.575357 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:25.581171 28152 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:25.525178  3303 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:25.637010 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50288.2us
I20260709 13:14:25.550110  3160 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:14:25.561436  3357 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:25.581363 28159 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:25.651180 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 75953.1us
I20260709 13:14:25.577527  3441 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.651957 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:25.650556 28730 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:25.656720 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:25.627345  3164 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:25.573726  3239 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:25.594785  3275 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.658486  3454 consensus_queue.cc:1059] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.005s
I20260709 13:14:25.663946  3164 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:25.664882  3293 consensus_queue.cc:237] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.667338  3181 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:25.668754  3450 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:25.668974 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:25.670744 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:25.691322  3181 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.697111 28157 raft_consensus.cc:3060] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:25.697266  3181 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:25.701812  3176 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.702818  3275 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:25.704272  3176 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:25.704435 28155 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:14:25.675328  3432 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.476s	user 0.002s	sys 0.003s
I20260709 13:14:25.677181  3194 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:25.678056  3359 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:25.638736  3080 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.710573  3080 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.712146  3456 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:25.678663 28082 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 25. Current term is 26. Ops: []
I20260709 13:14:25.681478 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:25.712770  3456 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.713231 28153 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:25.714676 28156 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:14:25.688530  3164 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:25.715911  3164 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:25.682915 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:25.722702  3322 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.648828  3354 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:25.692849 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:25.660071  3441 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:25.683559 28756 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 26.
I20260709 13:14:25.728829 28086 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:25.740386 28152 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:25.742033 28085 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:25.690099  3334 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:25.574193  3428 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.744205 28157 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:25.685245  3239 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:25.774087  3385 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:14:25.757618 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 297167us
I20260709 13:14:25.713255  3456 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:25.748777  3354 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:25.755616 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 234235us
I20260709 13:14:25.755658 28153 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 26.
W20260709 13:14:25.806735 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 150101us
I20260709 13:14:25.807535 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:14:25.809302 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 158865us
I20260709 13:14:25.786198  3390 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:25.800573  3354 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:25.808456  3255 consensus_queue.cc:1243] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:14:25.806450  3382 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:25.800498  3275 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:25.815476 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:25.827924 28079 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:25.840142 28082 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:25.837013 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 277184us
I20260709 13:14:25.837550 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:25.842748 28077 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:25.845355 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:25.848187  3091 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:25.848887  3131 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:25.849577  3131 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.853812  3322 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:25.854130 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:25.854875 28232 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:25.855774 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:25.856462 28231 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:14:25.856877  3448 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.328s	user 0.003s	sys 0.000s
I20260709 13:14:25.859232  3223 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:25.859717  3223 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:25.866362  3080 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:25.868039 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2962) took 1178 ms. Trace:
I20260709 13:14:25.815745  3190 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:25.839946  3160 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 13:14:25.876969  3299 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:25.877424  3299 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:25.877843  3299 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:25.879612  3329 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:25.879967  3080 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:25.880589  3080 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.881090  3359 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:25.882833  3080 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:25.883677  3080 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:25.884215  3080 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.876226  3190 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.885843  3428 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:25.830951  3318 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.886725  3080 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:25.886960  3190 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:25.840448  3334 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:14:25.888033  3318 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:25.888303  3334 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:25.888789  3334 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:25.833164 28050 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:25.868464 28078 rpcz_store.cc:276] 0709 13:14:24.689191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.819510 (+130319us) service_pool.cc:224] Handling call
0709 13:14:25.867802 (+1048292us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54002ad354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:25.867971 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1712788352}
I20260709 13:14:25.903550 28085 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 26.
W20260709 13:14:25.905073 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 219175us
W20260709 13:14:25.909060 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 52056us
I20260709 13:14:25.917956 28232 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:25.917141 28231 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:25.921155 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:25.922114 28079 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 26: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:25.841444  3440 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.926690 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 93596.9us
I20260709 13:14:25.928301 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:25.929081 28085 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:14:25.832314  3464 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:25.876201  3431 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.895524  3131 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:25.935142 28049 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:25.936511 28085 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:25.937914 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:25.945032 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:25.924271 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 114294us
W20260709 13:14:25.936892 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 90246us
I20260709 13:14:25.948058 28758 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:25.942979 28082 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:25.953189 28086 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 24: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:25.946830 28052 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:25.954489 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:25.955090 28157 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:25.951987 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:25.955613 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:25.956236 28227 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 21: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:25.959781 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:25.959910 28125 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:25.960345 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.960566 28154 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:25.961529 28125 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:25.965922 28756 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:25.966355 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.933615  3176 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.968554  3313 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:14:25.971109 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:25.971804 28152 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:14:25.972231 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:25.966970 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:25.899731  3164 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.974968 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:25.909260  3275 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:25.980506 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:25.985875 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:25.957504  3450 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.986696 28231 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:25.934121  3329 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:14:25.932711  3181 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.999728 28157 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:25.966962  3080 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:26.006080 28755 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
W20260709 13:14:26.018208 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51303.2us
I20260709 13:14:26.007782  3164 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:26.011324 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.060161 28231 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:26.012172 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.061020 28232 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:25.945870  3440 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:25.985739  3160 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.033272 28154 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
W20260709 13:14:26.018990 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 70099.9us
I20260709 13:14:26.015709  3313 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:25.975397  3464 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:25.986263  3455 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:25.974498  3465 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:25.934516  3334 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.040477 28077 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:25.974040  3176 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.077339 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.098881 28231 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:26.045014  3390 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:26.058825  3080 raft_consensus.cc:697] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:26.102946 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.101205 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.099274 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:26.106462 28082 raft_consensus.cc:2410] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:14:26.103935 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.107224 28226 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:26.111267 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:25.987191  3382 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.117569 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.113924 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.119257 28086 raft_consensus.cc:2393] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:26.119352 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:26.120631 28080 raft_consensus.cc:2410] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:14:26.122320 28226 raft_consensus.cc:2468] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:26.092012  3464 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:26.094895  3318 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.040848  3450 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.098208  3455 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:26.076776  3160 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.144572  3453 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.145052  3080 consensus_queue.cc:237] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.146239  3457 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.146775  3457 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.149175  3275 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.150364  3160 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:26.119807  3390 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:26.151057  3313 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.152632  3390 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.140146  3318 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.156091  3455 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.156975  3318 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.158620  3176 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.158804  3450 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.145515  3453 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.168383  3464 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.169687  3181 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.135802  3329 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.171291 28229 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:26.137526  3462 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:26.181068  3456 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.192878  3462 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.197438 28232 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:26.197208 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.198882 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.180877  3440 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.209491 28226 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:26.209163 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.190589  3385 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.212491 28228 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 25.
I20260709 13:14:26.214233 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.215221 28232 raft_consensus.cc:2393] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 21: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:26.216473 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 87838.4us
I20260709 13:14:26.220410 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.221312 28154 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:26.224166 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:26.187603  3329 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:26.220082  3357 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 13:14:26.221441 28232 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:14:26.226982 28756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8"
dest_uuid: "70996712c3c942468a6006602de6477f"
mode: GRACEFUL
new_leader_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:47466
I20260709 13:14:26.250169 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.263672 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.263458 28759 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:26.262876 28756 raft_consensus.cc:606] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Received request to transfer leadership to TS 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:26.273309 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.273638 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.274062 28228 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:26.276364 28235 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:26.224623  3431 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.281152 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.286608 28762 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 24.
I20260709 13:14:26.284233 28198 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:26.275827 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.273379 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.288120 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:26.296624 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.300138 28228 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:26.303478 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.310542 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.311993 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.312551 28230 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:26.313241 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.314514 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.317610 28756 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:26.320092 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.319877 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.316772 28198 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:26.223488  3385 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.282861  3382 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.329417 28235 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 16: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:26.324975 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:26.337054 28078 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:14:26.302057  3359 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:14:26.340063  3483 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:14:26.325575  3181 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:26.345391  3454 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:26.345438 28230 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
W20260709 13:14:26.348222  3498 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 13:14:26.353890  3479 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.224s	user 0.002s	sys 0.000s
I20260709 13:14:26.356128 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.358254 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:26.334620  3239 consensus_meta.cc:287] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.549s	user 0.005s	sys 0.005s
I20260709 13:14:26.271217  3357 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.397620 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.400141  3205 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.333151  3478 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:14:26.405215 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.302615  3313 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 13:14:26.423326 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.370697  3359 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.428383 28763 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:14:26.442958  3453 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.417724  3303 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.399396  3454 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:26.492033 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.492846  3454 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.458626 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.461546 28078 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:14:26.461997 28762 raft_consensus.cc:1275] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:26.499101 28230 raft_consensus.cc:2376] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20. Re-sending same reply.
I20260709 13:14:26.424608  3440 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:26.467187 28756 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 25.
I20260709 13:14:26.467864 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:26.475198 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:26.475566  3490 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:14:26.477933 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.476974 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.476073 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.479626  3239 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.485343 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.481190  3452 raft_consensus.cc:3055] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:26.486315  3205 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.509651  3303 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:26.413616  3468 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:26.510829  3385 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.511219  3468 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.511637  3468 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:26.511914  3357 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:26.513832 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:26.515765  3492 log.cc:927] Time spent T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:14:26.517016 28153 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:26.516242 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.518267 28230 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:26.483675  3478 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:26.495428 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.519146  3478 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:26.519362 28226 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:26.359328  3382 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.525561  3239 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.526801 28226 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 26.
W20260709 13:14:26.527294  3441 consensus_meta.cc:287] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.746s	user 0.008s	sys 0.004s
I20260709 13:14:26.486408 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:26.527810  3441 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.504544 28084 raft_consensus.cc:2376] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17. Re-sending same reply.
I20260709 13:14:26.484432  3456 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.501039 28086 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:26.530609 28158 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 22. Current term is 23. Ops: [20.12-22.13]
I20260709 13:14:26.535386 28760 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:26.536434 28079 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 24: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:26.538127 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.538925 28762 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 26: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:26.538833 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.539565 28154 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 16.
I20260709 13:14:26.540068 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.540817 28760 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:26.541384 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.541718 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.541567  3468 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.542135  3457 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.542713 28080 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:26.543265 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.541417 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.543628  3478 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.544358 28154 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
W20260709 13:14:26.544793  3354 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.704s	user 0.010s	sys 0.002s
I20260709 13:14:26.545185  3354 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.545908  3478 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.490387  3476 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.548122  3476 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.551849  3476 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.554234  3223 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.555172 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:26.555725  3470 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.504s	user 0.001s	sys 0.000s
I20260709 13:14:26.555862  3223 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:26.555966 28234 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:26.490648  3359 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:26.561338  3440 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.564154  3440 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.565012 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.566035  3454 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:26.566767  3329 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.567298  3329 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:26.567808  3329 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.569061  3354 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.569882  3434 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.570111 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.579362 28079 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 21: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:26.571311  3322 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:26.572167 28127 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:26.572795  3439 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:26.576761 28125 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:26.578055 28153 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:26.582773 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:26.583968 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.584601 28077 raft_consensus.cc:2410] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:14:26.587329 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.585995 28728 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:26.572394  3275 consensus_queue.cc:1059] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:14:26.508047  3452 raft_consensus.cc:740] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:26.587154 28125 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:26.590725  3485 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:26.589035  3453 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:26.591264  3485 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.588632 28730 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:26.578692  3465 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.574543  3441 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.583546  3489 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 13:14:26.597337  3453 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:26.580442  3165 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:26.600214 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.581579  3462 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.603082 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.603549  3439 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:26.585604  3080 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:26.604456 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.601639 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:26.605953 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:26.607584 28226 raft_consensus.cc:1240] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:26.608064  3388 raft_consensus.cc:2804] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:26.608326  3165 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.609810 28234 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:14:26.593765  3440 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:26.638289 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.639110 28227 raft_consensus.cc:2393] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 24: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:26.640439 28763 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:26.636899  3164 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:26.645304 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 2996) took 1067 ms. Trace:
I20260709 13:14:26.645682 28158 rpcz_store.cc:276] 0709 13:14:25.577935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.579375 (+  1440us) service_pool.cc:224] Handling call
0709 13:14:25.579926 (+   551us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:26.531053 (+951127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:26.645015 (+113962us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b54003935d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:26.645209 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2185470208}
I20260709 13:14:26.589599  3434 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.648142  3434 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.604383  3465 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.650588 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:26.615051  3439 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:26.614157  3441 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:26.616873  3489 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:26.612098  3080 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.618780  3477 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:26.619779  3334 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:26.619871  3359 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.621035  3388 raft_consensus.cc:697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:26.623100 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:26.662433 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.662959 28730 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:26.663415 28234 raft_consensus.cc:2393] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 22: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:26.625855  3382 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.598543  3485 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:26.626379 28728 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:26.637465 28199 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:26.613363 28760 raft_consensus.cc:2368] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 25. Current term is 26.
I20260709 13:14:26.650962 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.652370 28085 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:26.658833  2403 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:26.661113 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:26.669402 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:26.671517 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:26.672109 28756 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:14:26.656406  3489 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:26.658998  3477 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.685701 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:26.677531 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:26.659142  3080 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:26.677587 28227 raft_consensus.cc:2368] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 25. Current term is 26.
W20260709 13:14:26.679649 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 53365.4us
I20260709 13:14:26.680893 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.690055 28231 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:26.690116 28728 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:26.692160 28080 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:14:26.692792 28085 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 23.
I20260709 13:14:26.695575 28078 raft_consensus.cc:1240] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:14:26.668562  3205 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.669435  3164 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:26.676450  3478 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:26.673820  3472 raft_consensus.cc:2804] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:14:26.661495  3354 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:14:26.705216 28052 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:26.658432  3441 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.679950  3455 raft_consensus.cc:3055] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:14:26.699828  3299 consensus_meta.cc:287] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.822s	user 0.009s	sys 0.005s
I20260709 13:14:26.707350 28156 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:26.727723 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.711758 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.712067 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.708648 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:26.715224 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:26.728637 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.728967 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.678774  3357 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:26.696713  3382 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:26.707209  3164 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.690050  3477 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:26.723549 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.689838  3255 raft_consensus.cc:2804] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:26.726071 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:26.727820 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.716331 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.728925 28159 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:26.727037 28052 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:26.737514 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:26.730445 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.706607  3223 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:14:26.737785 28759 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 19: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:26.748011 28086 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:26.748956 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.750662 28085 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:26.758316 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.758942 28077 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:26.757552 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.759801 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:26.760229 28083 raft_consensus.cc:3060] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:26.747812 28161 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:26.758606 28049 leader_election.cc:400] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 25. Current term is 26.
I20260709 13:14:26.762892 28049 leader_election.cc:403] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:14:26.689365  3466 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:14:26.655694  3452 consensus_queue.cc:260] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.747607  3439 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.767469 28728 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:26.770123 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.772621 28085 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:26.703387  3457 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:14:26.747082  3164 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:26.746447  3401 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:26.721745  3455 raft_consensus.cc:740] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:14:26.776116 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49153.1us
I20260709 13:14:26.789069 28052 leader_election.cc:400] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 25. Current term is 26.
I20260709 13:14:26.777359  2403 consensus_queue.cc:1059] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:26.737850  3472 raft_consensus.cc:697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 26 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:26.737301  3176 consensus_queue.cc:1059] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:14:26.775538  3440 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:26.783749 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:26.792484 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.798287 28234 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:26.766134  3255 raft_consensus.cc:697] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:26.772244  3329 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.802510 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.781565  3357 raft_consensus.cc:2764] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:14:26.806362  2403 consensus_queue.cc:1243] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:14:26.805725 28052 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:26.747117  3299 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.725831  3390 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:14:26.790375  3312 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:26.762820  3478 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
W20260709 13:14:26.815358 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 137879us
I20260709 13:14:26.792639  3466 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:26.837709 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:26.845075 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:26.821010 28083 raft_consensus.cc:2468] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 26.
I20260709 13:14:26.845320 28082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757"
dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
mode: GRACEFUL
new_leader_uuid: "11f10c12c03e47dcac88b050d4c1431b"
 from {username='slave'} at 127.0.0.1:43764
W20260709 13:14:26.847057 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 121095us
I20260709 13:14:26.847635 28082 raft_consensus.cc:606] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Received request to transfer leadership to TS 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:26.848579 28228 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:26.848073 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:26.814973  3476 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:26.849989 28762 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:26.769958  3388 consensus_queue.cc:237] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:26.845096  3501 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:14:26.853495 28231 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:14:26.856591 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 89215.5us
I20260709 13:14:26.797317  3453 consensus_queue.cc:1059] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:14:26.852689 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:26.823262  3312 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.842828  3464 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:26.843000  3466 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:26.844702  3165 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.863016 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.864012  3299 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:26.879819 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.906374 28160 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:26.869740 28227 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:14:26.803350  3506 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
W20260709 13:14:26.908393 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 60397.5us
I20260709 13:14:26.849287  3462 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 25. Current term is 26.
I20260709 13:14:26.872253  3359 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.852619  3465 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.902684  3464 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.905346 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:26.923959 28159 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:26.906778 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 62690.2us
I20260709 13:14:26.900187 28728 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:26.909292 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:26.898551  3223 raft_consensus.cc:697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 26 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:26.900805 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:26.824509  3503 consensus_queue.cc:1048] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.092s
I20260709 13:14:26.899399  3312 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:26.927126 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 75020.2us
I20260709 13:14:26.925925 28125 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:26.921607  3453 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:26.823417  3472 consensus_queue.cc:237] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.904359  3390 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:14:26.934267 28124 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:26.901499  3357 consensus_queue.cc:1059] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:14:26.931295 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:26.936403 26959 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:26.936683  3484 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.937281  3484 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.938848 26959 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:26.939570  3484 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:26.939908 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:26.940279  3512 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:14:26.941193  3466 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.941994  3299 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:14:26.942503  3466 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:26.911108  3255 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:26.945237  3502 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.120s	user 0.001s	sys 0.001s
I20260709 13:14:26.945963 28756 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:14:26.940718 28153 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 19: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:26.942027 28759 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:14:26.948135 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:26.927623  3464 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:26.956204  3517 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:26.956683  3517 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.957185  3517 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:26.960814  3510 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:26.964108 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:26.965611 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:26.966233 28761 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:26.964897  3299 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 13:14:26.968062  3453 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:26.969178  3453 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.969797  3303 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:26.970216  3303 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:26.971043  3303 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:26.972383  3303 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:26.972669 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:26.973964 28124 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:26.934859  3506 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:26.949626  3441 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:26.949872  3359 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:26.981568  3484 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:26.981943  3434 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:26.993311  3519 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:26.952986  3275 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:26.994241 28159 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:26.917222  3440 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:26.997102 28761 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:27.003659 28161 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
W20260709 13:14:27.005023 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49943.4us
I20260709 13:14:27.010732 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:27.019294 28079 raft_consensus.cc:2376] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 18. Re-sending same reply.
W20260709 13:14:27.025715 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 77639.2us
I20260709 13:14:26.936700  3223 consensus_queue.cc:237] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.006294  3434 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.035868  3516 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:27.036827  3516 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:27.037753  3511 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:14:27.039868  3516 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.040584  3515 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:27.041241  3515 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.041697  3515 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:27.006217  3509 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.187s
I20260709 13:14:27.044656  3518 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:27.032372  3434 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:27.023149  3299 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.051571  3080 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.040602  3477 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.057219  3312 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.059494  3517 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.024749  3359 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:27.031728  3275 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:27.075888  2403 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.100533  2403 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.102664  2403 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:27.066701  3164 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.104137 28764 raft_consensus.cc:2368] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 24. Current term is 25.
I20260709 13:14:27.105028  3164 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.080029  3312 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.083863  3434 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.090745 28758 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:14:27.097483 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.107487  3434 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:27.113306 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:27.126632 28755 raft_consensus.cc:2368] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 24. Current term is 25.
W20260709 13:14:27.134963 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 102013us
I20260709 13:14:27.122277  3164 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.141769 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:27.069317  3299 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.149405 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:27.151144 28050 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:27.152158 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:27.154121  2403 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:27.154649  2403 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:27.155411  2403 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.148434  3527 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:27.163291  3275 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.164364  3527 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.166047  3477 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.166162  3458 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:27.166836  3458 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:27.167868  3299 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:27.167706  3458 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.157068  3312 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:27.168460  3533 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:27.199150  3312 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:27.199237  3533 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.199936  3533 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:27.200273  3312 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.171861  3468 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.172287  3442 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:27.205269  3442 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.172798  3531 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:27.173237  3525 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.208879  3525 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.209821  3334 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:27.210522  3334 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:27.174207  3521 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:14:27.168453  3359 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:27.213121  3507 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.400s	user 0.004s	sys 0.000s
W20260709 13:14:27.217654 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.175001  3455 consensus_queue.cc:260] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.178422  3461 raft_consensus.cc:3055] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:27.146004  3164 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.236151 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:27.181490  3484 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:27.182046 28198 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:27.237396 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 88378.4us
W20260709 13:14:27.238345 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 56456.8us
I20260709 13:14:27.241870 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:27.243851 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:27.244855 28234 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:27.248641 28198 leader_election.cc:400] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 24. Current term is 25.
I20260709 13:14:27.249351 28198 leader_election.cc:403] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:14:27.251611 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:27.217417  3510 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:27.145243  3509 consensus_queue.cc:1059] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:14:27.230538 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:27.198489  3458 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:27.252966  3334 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:27.264384 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:27.265957 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:27.267019 28228 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:27.264079 28759 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:27.234151  3461 raft_consensus.cc:740] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:27.268707 28226 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:27.271749 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:27.257815  3455 raft_consensus.cc:3060] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:27.256233  3531 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:27.285144 28728 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:27.328828 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.331220 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:27.332391 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:27.333900 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:27.335140 28234 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 25.
I20260709 13:14:27.341449 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:27.317998  3509 consensus_queue.cc:1243] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 611ec64ec58244fe82058d6130e4e6f3 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:14:27.360577 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:27.361557 28229 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:27.348609  3506 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:27.337425  3510 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 24. Current term is 25.
I20260709 13:14:27.345300  3454 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:27.368683 28226 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 25.
I20260709 13:14:27.375581 28153 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:14:27.376662 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:27.381734  3468 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:27.382277  3468 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:27.382593  3532 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:14:27.382795  3468 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:27.382467  3529 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:14:27.381656  3538 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:27.372572  3255 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:27.373418  3453 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:27.373724  3528 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:27.318328  3489 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.347577  3461 consensus_queue.cc:260] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:27.405884 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 120808us
I20260709 13:14:27.392067  3357 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:14:27.414147 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:27.398547  3255 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:27.415460 28160 raft_consensus.cc:2410] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
W20260709 13:14:27.424360 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 152564us
I20260709 13:14:27.405920  3538 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:14:27.419273  3255 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:27.407735  3453 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.410523  3466 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:14:27.423038  3461 raft_consensus.cc:3060] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:27.438091 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:27.441449 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:27.410912  3528 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.449030  3515 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.449878  3509 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:27.451581  3515 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.431993  3164 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.479552 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:27.471001  3329 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:27.475692  3164 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:14:27.484431  3542 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.140s	user 0.002s	sys 0.000s
I20260709 13:14:27.496609 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:27.500978 28229 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:27.503417 28199 leader_election.cc:400] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
I20260709 13:14:27.503959 28199 leader_election.cc:403] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:27.494220  3538 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:27.471668  3464 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.521286 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:27.503227  3539 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:14:27.522843  3517 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.542619 28730 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:27.528013  3545 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:14:27.496850  3329 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.533578  3359 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:27.544328 28229 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:27.545434 28232 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:14:27.505107  3164 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 25 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:27.546433  3558 raft_consensus.cc:2804] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:27.571815  3558 raft_consensus.cc:697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 25 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:27.547202  3544 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:14:27.548091  3468 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.549683  3453 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:27.552965 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55689.3us
I20260709 13:14:27.575781 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:27.576929 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:27.577744 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:27.578352 28229 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 25: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:27.578279 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:27.580233 28755 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:14:27.580432 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:27.581070 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:27.553746 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:27.559561 28231 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:14:27.560384  3452 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:27.562856  3359 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.564064  3528 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:27.509326  3549 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20.
I20260709 13:14:27.543365  3538 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:27.495843  3533 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.554973  3546 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:27.566268  3329 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:27.568104  3464 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:27.568943  3519 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.003s
I20260709 13:14:27.570430  3164 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:27.554219  3541 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:14:27.598030  3553 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:14:27.572815  3558 consensus_queue.cc:237] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.581957 28226 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:27.601078 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:27.583102 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:27.586534  3557 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:27.586905 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:27.603715 28761 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:27.587574  3357 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:27.591704 28762 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:27.584934  3466 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:27.580407 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:27.587970  3556 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 13:14:27.601845  3519 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:14:27.597441 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 75158.1us
I20260709 13:14:27.611879  3299 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:27.612330  3299 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:27.606281 28160 raft_consensus.cc:1275] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:14:27.612907 28160 pending_rounds.cc:85] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 15
I20260709 13:14:27.613200 28160 pending_rounds.cc:107] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:14:27.613694 28160 raft_consensus.cc:2889] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:27.610646  3538 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.605271 28080 raft_consensus.cc:2410] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:14:27.615334  3359 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:27.615595  3329 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:27.594249  3546 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.616514  3546 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:27.620234  3489 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:27.609076  3520 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:27.621073  3520 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:27.622186  3520 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.623593  3520 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:27.626026  3468 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:27.626861  3468 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:27.606346 28124 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:27.628701 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:27.630934 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:27.634635 28227 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:27.629022  3518 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:27.644050 28761 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 26.
I20260709 13:14:27.646220 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:27.644097 28226 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:27.630564  3299 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.631862  3442 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.653079 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:27.617136  3554 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.670883  3513 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:27.671618  3513 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.669018  3455 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:27.673162  3455 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:27.673561  3455 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:27.674412  3560 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:14:27.676095  3546 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.651396  3518 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:27.663138  3442 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:27.642870  3472 consensus_queue.cc:1048] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:14:27.690063  3546 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.696094 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:27.697003  3476 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:27.698712 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:27.699899  3239 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.700918  3476 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:27.700454  3239 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:27.703732 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.704074 28232 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:14:27.705986  3313 raft_consensus.cc:3055] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:27.706431  3313 raft_consensus.cc:740] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:27.707604  3334 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:27.704138  3239 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.710487 28080 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:27.719161 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:27.719898 28161 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:27.720634 28159 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:14:27.720942 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:27.721531 28158 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:27.685155  3299 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:27.685369  3478 raft_consensus.cc:3055] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:27.725131  3478 raft_consensus.cc:740] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:27.726099  3478 consensus_queue.cc:260] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.727391 28049 leader_election.cc:400] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 24. Current term is 25.
I20260709 13:14:27.727991 28049 leader_election.cc:403] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:27.727785  3401 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:27.730144  3239 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:14:27.730727  3401 raft_consensus.cc:3060] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:14:27.706372 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:27.732692 28232 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:27.734411 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:27.686878  3442 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.735699  3442 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:27.736029 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.712549  3525 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.700610  3458 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:27.738077  3472 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:14:27.738719  3080 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:27.738178  3458 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.707561  3313 consensus_queue.cc:260] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.740839  2403 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:27.741381  2403 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.741708  2403 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:27.741726 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.743193 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:27.759740 28759 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:27.759516 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:27.762277 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:27.762636 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:27.763501 28759 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:27.759449 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:27.766618 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:27.767328 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:27.768075 28078 raft_consensus.cc:2368] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21.
I20260709 13:14:27.767333 28161 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:27.769913 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:27.771292 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:27.771445 28156 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:27.772104 28082 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 25: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:27.772503 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:27.774271 28158 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:27.773473 28227 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:27.765139  3478 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:27.764462  3461 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:14:27.779680 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 201430us
I20260709 13:14:27.787142 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:27.787927 28728 leader_election.cc:400] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21.
I20260709 13:14:27.788301 28728 leader_election.cc:403] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:14:27.684074  3456 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.873s	user 0.014s	sys 0.008s
I20260709 13:14:27.677492  3554 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.782918 28081 raft_consensus.cc:1275] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:14:27.761215  3313 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:27.779515 28052 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:27.794164 28160 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:14:27.770825  3181 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.794623 28160 pending_rounds.cc:85] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 9
I20260709 13:14:27.795717 28160 pending_rounds.cc:107] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:14:27.772794  3510 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:27.796037 28160 raft_consensus.cc:2889] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:27.799396 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:27.801412 28761 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:14:27.798178  2403 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.796790 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:27.804483 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:27.805356 28084 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 21: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:27.806567 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:27.793867  3456 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.015s	sys 0.009s
I20260709 13:14:27.808943 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:27.809139 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:27.812222 28758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1393) took 1141 ms. Trace:
I20260709 13:14:27.812546 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:27.813575  3359 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:27.826592 28758 rpcz_store.cc:276] 0709 13:14:26.670876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.671005 (+   129us) service_pool.cc:224] Handling call
0709 13:14:27.090298 (+419293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.106250 (+ 15952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.811946 (+705696us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b5400686fd0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:27.812110 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919206144}
I20260709 13:14:27.839900 28084 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:27.840402 28227 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:27.833564 28124 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:27.815397  3527 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.820823 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:27.821334  3559 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:14:27.766860  3334 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:14:27.821945  3255 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.823935  3533 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:27.824312 28086 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:14:27.781119  3080 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:27.827793 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:27.831700 28050 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:27.835783 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:27.836303 28152 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:14:27.875552 28161 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:27.837576 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:27.841961  2403 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.786796  3461 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.857236 28761 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:27.796345  3312 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.859292 28077 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 24: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:27.863547 28157 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:27.806079  3468 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:27.803588  3510 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.871881 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:27.914260 28081 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:27.873309 28127 leader_election.cc:336] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.548s (SENT)
I20260709 13:14:27.878262 28155 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:27.917820 28757 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:27.845772  3520 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:27.805966  3181 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:27.906564 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:27.861090  3275 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:14:27.912699 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 84979us
I20260709 13:14:27.928224 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:27.929944 28227 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:14:27.941807 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:27.942540 28235 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:27.859019  3239 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:14:27.859422  3519 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.071s
I20260709 13:14:27.901604  3461 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:27.918947  3510 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:27.908334  3223 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21.
W20260709 13:14:27.927223  3531 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.001s	sys 0.012s
I20260709 13:14:27.954021 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:27.917032  3431 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:14:27.971829 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:27.929627  3557 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:27.979270 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.975272 28161 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:27.976534 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:27.916533  3472 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.050s
I20260709 13:14:27.938355  3458 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:27.978082 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:27.981366 28728 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:27.987201 28086 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 15.
I20260709 13:14:27.981844 28227 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
W20260709 13:14:27.982368 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.983517 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:27.939112  3513 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:27.966377  3538 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:27.977916  3565 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 13:14:28.006106 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.993192 26959 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:27.917898  3080 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:27.919550  3312 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:27.981519  3531 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.001s	sys 0.012s
I20260709 13:14:28.027218  3531 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:28.027676 28086 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:27.994202 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:27.955780  3275 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 26 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:14:28.031306 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47863.3us
I20260709 13:14:28.007988  3181 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:28.008955  3577 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:27.970110  3457 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:14:27.982713  3464 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:27.989895  3519 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:27.995214  3575 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:27.950285  3533 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 13:14:28.017772 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:28.023026 28083 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:14:28.018421  3452 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.992376  3520 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:14:28.024887 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:28.025594  3554 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:28.033470 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:28.034487 28078 raft_consensus.cc:1275] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:14:28.035037 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:28.041767  3255 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:28.042613  3566 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:14:27.995440  3491 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:28.045552  3275 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.047798  3518 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:28.028249  3531 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.051393  3442 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.020862  3516 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.059535 28049 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:28.077144 28086 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:28.077919 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:28.101287 28155 raft_consensus.cc:2410] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:14:28.084152 28159 raft_consensus.cc:1240] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:28.095207 28081 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:28.076761 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:28.111377 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:28.085456  3516 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.112298 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:28.074545  3520 consensus_queue.cc:1243] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:28.113054 28226 raft_consensus.cc:2368] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:28.113870 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:28.114737 28158 raft_consensus.cc:2393] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 21: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:28.118777 28078 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:14:28.118796 28125 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:28.119517 28758 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:28.127800 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:28.077316  3176 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:28.077451  3453 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:28.071705  3457 consensus_queue.cc:1243] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:14:28.066620  3080 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:28.135246  3503 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:28.135794  3313 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.136543  3313 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:28.136871  3506 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:28.119092  3575 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:28.138386  3575 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.138904  3313 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:28.142267 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 82964.7us
I20260709 13:14:28.143199 28086 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
W20260709 13:14:28.145355 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46424.1us
I20260709 13:14:28.147562 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:28.148551 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:28.149739 28229 raft_consensus.cc:2410] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:14:28.151278 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:28.152321 28235 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 22: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:28.158044 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.071419  3577 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:28.058413  3312 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:28.072223  3519 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:28.078950  3491 raft_consensus.cc:697] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:28.201589 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54106us
I20260709 13:14:28.214931 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:28.206503 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:28.084125  3464 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:28.224813 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:28.225767 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:28.225692 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 79440us
I20260709 13:14:28.211284 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:28.227166 28227 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 22: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:28.212971 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 69958.1us
I20260709 13:14:28.219388 28158 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:28.228417 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:28.217911 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:28.213507 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:28.140599  3575 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:28.229575 28156 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 22: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:14:28.246060 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.074249  3452 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:28.148478  3468 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:28.125089  3513 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:28.207389  3329 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:28.078053  3476 consensus_queue.cc:1048] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.125s
I20260709 13:14:28.243515 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:28.250181 28235 raft_consensus.cc:1275] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:14:28.251513 28156 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:14:28.056661  3455 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.140605  3526 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 13:14:28.215534  3453 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:14:28.261869  3576 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:28.203454  3577 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:28.206256  3176 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:28.275569  3401 consensus_meta.cc:287] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.005s
I20260709 13:14:28.273726 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:28.234534  3503 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:28.276042  3472 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:28.247102  3456 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:14:28.249676  3313 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:28.297801 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:28.285498 28226 raft_consensus.cc:1240] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:14:28.300575 28155 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: [21.17-21.17]
W20260709 13:14:28.237221 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 405575us
I20260709 13:14:28.288937 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:28.304806 28227 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 22: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:28.244024  3458 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.291316 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:28.247689  3489 consensus_queue.cc:1048] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 13:14:28.310791 28228 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17. Ops: []
W20260709 13:14:28.311339 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.317674 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:28.321329 28235 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 24: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:28.320289 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:28.294481  3503 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:28.329331 28157 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 13:14:28.329772 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:28.338367 28155 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:28.327324 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:28.297108  3401 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 24. Current term is 25.
W20260709 13:14:28.328063  3568 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
I20260709 13:14:28.362881 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:28.286899  3519 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.363862 28157 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:28.342103  3503 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:28.373127 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:28.374104 28156 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 17.
I20260709 13:14:28.307350  3464 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:28.387912 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:28.318481  3458 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.394470 28127 leader_election.cc:400] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17.
I20260709 13:14:28.400700 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:28.321677  3468 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.408124 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:28.383551  3580 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.412829 28157 raft_consensus.cc:2410] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:14:28.331323  3491 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.414170 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:28.415591 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:28.404178  3275 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:28.417258  3455 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:28.417407  3452 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:28.420221 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:28.405933  3578 log.cc:927] Time spent T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:14:28.399739  3519 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:28.437407 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.440369 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:28.440661 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.448828 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:28.364712  3176 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:28.451659 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:28.454850 28155 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:28.426820  3526 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:14:28.441555  3572 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:14:28.457134 28079 raft_consensus.cc:2468] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 25.
I20260709 13:14:28.427301  3275 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:28.458189 28155 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:14:28.462157 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 217662us
I20260709 13:14:28.462831 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:28.463315 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:28.464129 28155 raft_consensus.cc:2368] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 23. Current term is 24.
I20260709 13:14:28.464844 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:28.474632 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:28.476666 28084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:28.476382 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:28.483081 28077 raft_consensus.cc:2368] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 24. Current term is 25.
I20260709 13:14:28.457423  3452 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:28.491501 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:28.492381 28083 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 22: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:28.493144 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:28.465695  3508 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:28.494349 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:28.494644 28231 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:14:28.497701 28231 pending_rounds.cc:85] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 12
I20260709 13:14:28.498091 28231 pending_rounds.cc:107] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:14:28.495205 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:28.499171 28231 raft_consensus.cc:2889] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:28.499857 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:28.501048 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:28.439189  3510 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.428148  3580 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.502775 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:28.501436 28077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:28.503635 28232 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 22: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:28.471567  3583 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.506575 28199 leader_election.cc:400] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 24. Current term is 25.
I20260709 13:14:28.477936  3526 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:28.489203  3312 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.519598  3531 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:28.524952  3583 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.526620  3580 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:28.533447  3582 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:14:28.534626 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.516749 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:28.537730  3587 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:14:28.462296  3453 consensus_queue.cc:1048] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:14:28.468211  3322 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:14:28.537611 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:28.509753 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:28.507351  3461 consensus_meta.cc:287] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.000s
I20260709 13:14:28.519698 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:28.542439 28234 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:28.547222 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:28.558333 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:28.559497 28083 raft_consensus.cc:2368] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22.
I20260709 13:14:28.559193 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:28.560576 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:28.558182 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:28.564586 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:28.565734 28154 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: [21.11-21.11]
W20260709 13:14:28.567512 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.567942 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:28.568629 28161 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:28.565140 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:28.572445  3452 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:28.573150 28082 raft_consensus.cc:1240] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: [18.11-20.12]
I20260709 13:14:28.574154  3452 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:28.549417  3312 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:28.574882  3452 consensus_queue.cc:1243] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:14:28.562458  3531 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:14:28.562660  3461 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:28.580065 28084 raft_consensus.cc:2368] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19.
W20260709 13:14:28.557165  3586 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:14:28.569895 28125 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:28.586398 28085 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:14:28.587211 28154 raft_consensus.cc:1240] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:28.589936 28125 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:28.592237 28083 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 22. Current term is 23. Ops: [20.12-22.13]
I20260709 13:14:28.588457 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:28.557623  3299 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:28.595219 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:28.596872 28125 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:28.607668 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:28.585099  3592 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:28.582306  3576 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:28.607668 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:28.633985 28154 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:28.593480  3322 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:28.590102  3520 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:14:28.564121  3575 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 13:14:28.588866  3312 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:28.603891  3518 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:28.633109  3592 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.610337  3461 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:28.627125  3525 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:28.631014  3513 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:28.725112  3575 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:28.730168  3575 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:28.731042  3575 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.732406  3575 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:28.601722  3452 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 13:14:28.671679  3080 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.732434  3299 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.722209  3518 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:28.721637  3592 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:28.758985  3581 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.460s	user 0.000s	sys 0.002s
I20260709 13:14:28.778294 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3450) took 1825 ms. Trace:
I20260709 13:14:28.778932 28153 rpcz_store.cc:276] 0709 13:14:26.952719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.965935 (+ 13216us) service_pool.cc:224] Handling call
0709 13:14:26.980216 (+ 14281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.378125 (+397909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.777906 (+1399781us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b5400051c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:28.778198 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":908362112}
W20260709 13:14:28.782140 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 223021us
I20260709 13:14:28.635349  3576 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.786352 28235 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:14:28.786949 28235 pending_rounds.cc:85] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 9
I20260709 13:14:28.787351 28235 pending_rounds.cc:107] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:14:28.787671 28235 raft_consensus.cc:2889] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:28.601027  3580 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.117s
I20260709 13:14:28.774390  3299 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.795405  3518 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:28.799969  3461 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:14:28.765632 28158 raft_consensus.cc:2410] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 22 because replica has last-logged OpId of term: 21 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 17.
I20260709 13:14:28.766521  3503 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.773526  3080 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.790652  3576 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:14:28.852603 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66545.2us
I20260709 13:14:28.853912 28199 leader_election.cc:400] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 23. Current term is 24.
I20260709 13:14:28.854236 28199 leader_election.cc:403] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:28.829856  3461 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.816851  3510 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:28.836894  3579 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:28.862762 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.840397  3401 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:28.873188 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.873188 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:28.885593 28161 raft_consensus.cc:2410] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:14:28.806366  3478 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 13:14:28.902307 28154 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:28.905609 28157 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: [17.12-18.13]
I20260709 13:14:28.908439 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:28.870645  3385 raft_consensus.cc:993] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: : Instructing follower 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:14:28.917243 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:28.930658 28154 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:28.865481  3602 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 23. Current term is 24.
I20260709 13:14:28.937402 28154 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:14:28.863871  3461 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:28.870790  3603 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.073s
W20260709 13:14:28.953487  3535 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 1.589s	user 0.005s	sys 0.000s
W20260709 13:14:28.954496 28234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 1720) took 1184 ms (client timeout 1553 ms). Trace:
I20260709 13:14:28.957851  3080 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:28.958065  3516 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:28.959391 28226 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 13:14:28.989656 28226 pending_rounds.cc:85] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
I20260709 13:14:28.990020 28226 pending_rounds.cc:107] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 13:14:28.990307 28226 raft_consensus.cc:2889] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:28.933331  3478 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:28.871218  3579 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:28.909157  3520 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.903857  3453 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:14:28.936259  3451 raft_consensus.cc:1081] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Signalling peer 611ec64ec58244fe82058d6130e4e6f3 to start an election
I20260709 13:14:28.965992 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3457) took 1202 ms. Trace:
I20260709 13:14:29.000878 28083 raft_consensus.cc:1275] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:14:29.000674 28159 rpcz_store.cc:276] 0709 13:14:27.763620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.772924 (+  9304us) service_pool.cc:224] Handling call
0709 13:14:27.779790 (+  6866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.102207 (+322417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.965606 (+863399us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b5400343fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:28.965905 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699428224}
I20260709 13:14:28.974193 28757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35896 (request call id 1399) took 1478 ms. Trace:
W20260709 13:14:28.984306 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:28.984012 28234 rpcz_store.cc:269] 0709 13:14:27.769560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.769666 (+   106us) service_pool.cc:224] Handling call
0709 13:14:28.954169 (+1184503us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54003c9e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:28.954388 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1774731520}
I20260709 13:14:28.986478 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:28.987170 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:28.988149 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 79798.1us
I20260709 13:14:28.989423 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:28.894448  3401 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:28.966003  3503 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:28.933601  3299 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:28.994692 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.006096 28050 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:29.006888 28199 leader_election.cc:400] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22.
I20260709 13:14:29.006654 28764 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:29.002610 28757 rpcz_store.cc:276] 0709 13:14:27.495760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.495924 (+   164us) service_pool.cc:224] Handling call
0709 13:14:27.496151 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.928014 (+431863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.973957 (+1045943us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b5400472cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:28.974134 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954885888}
I20260709 13:14:29.021171 28199 leader_election.cc:403] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:29.021168 28761 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:29.026978 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:29.040973 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:29.043432  3478 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:29.044106 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:29.045466  3478 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:29.046123  3478 consensus_queue.cc:1059] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:29.046911  3478 raft_consensus.cc:3055] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:29.047230  3478 raft_consensus.cc:740] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:29.034683  3299 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587) with higher term. Message: Invalid argument: T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22.
I20260709 13:14:29.049053  3478 consensus_queue.cc:260] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.050292  3478 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:29.054286 28125 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:29.059077 28235 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:29.059507 28233 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:14:29.062404 28234 pending_rounds.cc:85] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 11
I20260709 13:14:29.062753 28234 pending_rounds.cc:107] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:14:29.063019 28234 raft_consensus.cc:2889] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:29.069388  2403 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:29.070185  2403 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:29.071509  2403 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.079257 28199 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:29.011143  3520 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.039084  3503 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:29.094020 28235 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:14:29.067390  3525 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:29.077622  3519 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:29.123818 28228 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:29.123574  3385 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.126453  3385 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.118535  3603 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:14:29.130036  3442 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.096939  3503 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.131115  3503 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:29.131386  3603 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:29.131525  3519 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.132059  3603 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.132893 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:29.134150  3519 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.134557  3603 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:29.136751  3458 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.137709 28078 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:14:29.137807 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:29.138675 28761 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:29.134601 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 1647) took 1375 ms. Trace:
I20260709 13:14:29.141350 28079 rpcz_store.cc:276] 0709 13:14:27.759102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.759248 (+   146us) service_pool.cc:224] Handling call
0709 13:14:29.134403 (+1375155us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b54000c99d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:29.134531 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1382264064}
W20260709 13:14:29.097340  3577 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.785s	user 0.019s	sys 0.007s
I20260709 13:14:29.144052 28154 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:14:29.155890 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:29.157234 28759 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:29.159065 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.087109  3299 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:29.168694  3612 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:14:29.186172  3520 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.087270  3575 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:14:29.188299  3575 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:29.188760  3575 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:29.189713  3575 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.191044  3575 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:29.193559  3579 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:29.196504  3461 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:29.162624  3478 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:29.200495  3478 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.202862  3478 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.206126  3329 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:29.206782  3329 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:29.191072  3583 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.209089  3520 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:29.209689  3520 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.212924  3385 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:29.220089 28761 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:29.229916 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.231487 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:29.232595 28759 raft_consensus.cc:2393] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 20: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:29.235368 28757 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:14:29.162724  3299 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.239650 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:29.245442 28761 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:14:29.244314 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.247334 28759 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:29.208011  3329 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:29.260000 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 253944us
W20260709 13:14:29.255172 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 96847.9us
I20260709 13:14:29.256319 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.166674  3577 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.274333 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.243921  3299 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:29.236506  3592 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:29.300282 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.273296  3612 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:29.284461  3583 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.255321  3478 consensus_queue.cc:1059] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.042s
I20260709 13:14:29.289554  3603 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:29.312713 28049 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:29.303917  3329 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:29.309640  3579 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.311480  3584 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:29.310993  3531 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:29.370154  3334 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:29.371042  3334 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.376259  3617 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:29.377175  3617 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.377616  3617 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:29.376415  3334 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:29.350299  3513 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:29.385505  3513 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.387763  3513 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:29.389604  3619 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:14:29.391475  3520 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.319720  3612 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.395380  3612 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.320611  3603 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.397310 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:29.397965 28159 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:29.398497  3603 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:29.400223  3577 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:29.403124  3573 log.cc:927] Time spent T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 1.149s	user 0.007s	sys 0.002s
I20260709 13:14:29.409189  3164 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:29.409595  3592 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:29.313110  3583 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.417645  3583 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.419306  3584 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.421646  3584 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.319458  3576 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.424309  3576 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.425338 28080 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 25: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:29.319567  3614 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:29.426928  3614 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.429260  3614 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.429910  3531 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.432149  3531 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.433988 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:29.434597 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:29.434928 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:29.435797 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:29.448184 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:29.449597 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:29.450222 28078 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 25.
I20260709 13:14:29.451017 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:29.452003 28078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b"
I20260709 13:14:29.441687  3520 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.444080  3612 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:29.454645  3576 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.452039  3457 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.466331  3457 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.353514  3579 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.468609  3579 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:29.469022  3520 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.471506  3520 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:29.459409  3456 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:29.505402 28229 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 26.
I20260709 13:14:29.506263 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2846) took 1294 ms. Trace:
W20260709 13:14:29.507524 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 242353us
I20260709 13:14:29.506666 28229 rpcz_store.cc:276] 0709 13:14:28.214426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.225343 (+ 10917us) service_pool.cc:224] Handling call
0709 13:14:29.433829 (+1208486us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b58001c0598. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:14:29.506184 (+ 72355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9964800}
I20260709 13:14:29.468662  3457 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.480592  3612 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:29.636965 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 392635us
I20260709 13:14:29.602830  3442 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.603331  3519 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.622982  3478 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:29.638691  3503 consensus_meta.cc:287] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.000s
I20260709 13:14:29.513053  3576 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.603441  3579 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:29.637048  3612 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:29.517868  3456 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.677436 28083 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 3004) took 1175 ms. Trace:
I20260709 13:14:29.674628 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40992 (request call id 2987) took 1198 ms. Trace:
I20260709 13:14:29.675925 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3451) took 1957 ms. Trace:
I20260709 13:14:29.691730 28160 rpcz_store.cc:276] 0709 13:14:27.718236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.719515 (+  1279us) service_pool.cc:224] Handling call
0709 13:14:27.719799 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.801203 (+ 81404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.675660 (+1874457us) spinlock_profiling.cc:243] Waited 73.9 ms on lock 0x7b54000807d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:29.675861 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170078464}
I20260709 13:14:29.677342 28085 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:14:29.626688  3205 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:14:29.681658  3564 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:29.687805 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 252952us
W20260709 13:14:29.673063  3624 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:14:29.627745  3080 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:14:29.636102  3508 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.053s
I20260709 13:14:29.690747 28084 rpcz_store.cc:276] 0709 13:14:28.476154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.476325 (+   171us) service_pool.cc:224] Handling call
0709 13:14:29.674419 (+1198094us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b54001ad454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:29.674567 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223540992}
W20260709 13:14:29.704071 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.705097 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:29.693976 28083 rpcz_store.cc:276] 0709 13:14:28.504327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.930800 (+426473us) service_pool.cc:224] Handling call
0709 13:14:28.931106 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.001754 (+ 70648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.677079 (+675325us) spinlock_profiling.cc:243] Waited 66.3 ms on lock 0x7b54003aa7d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:29.677351 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152488320}
I20260709 13:14:29.707149  3603 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:29.707723  3603 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.709209 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.710265  3603 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:29.724437 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:29.728540 28235 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:14:29.729660 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.731101 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.671801  3516 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:29.677398  3531 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:29.737371 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:29.759160 28226 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:29.740404  3576 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:29.746079  3533 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:29.746877  3456 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:29.698132  3579 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.729719  3564 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:29.750367  3607 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.707s	user 0.004s	sys 0.004s
I20260709 13:14:29.711810  3452 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:29.752382  3547 raft_consensus.cc:3055] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:14:29.755960 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 151001us
I20260709 13:14:29.668952  3442 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.776494 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.671378  3519 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.670500  3478 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:29.686681  3503 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.750926 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:29.786768  3575 consensus_meta.cc:287] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.001s
W20260709 13:14:29.786463 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.786966 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:29.790181 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:29.773284  3533 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:29.796440 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:29.800277  3452 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.821875  3602 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:29.822463 28756 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:14:29.791674 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61540.4us
I20260709 13:14:29.793408  3547 raft_consensus.cc:740] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:29.770242  3516 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.774928  3531 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.933029 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:29.852443 28159 raft_consensus.cc:2410] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 13:14:29.965090 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.001670 28082 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
W20260709 13:14:30.072245 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 246897us
W20260709 13:14:30.065402 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 424782us
I20260709 13:14:30.073316 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:29.924163 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.001965 28077 raft_consensus.cc:1240] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:30.064671 28049 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:30.076011 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:29.937100 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 165109us
I20260709 13:14:29.905958  2403 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.078333  2403 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:29.841045 28759 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:30.066605 28124 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:30.089740 28083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:30.093027 28077 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:14:30.098129 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:30.090813 28083 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:30.041471  3575 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:30.061995  3601 log.cc:927] Time spent T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:14:30.106719 28761 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:14:30.107525 28124 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:30.003093  3525 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.069187  3602 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:30.021824  3634 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:29.866439  3452 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.075527  3503 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:29.968652  3519 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:30.109313  3520 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:30.100466  3508 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:29.927457  3275 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:29.969624  3322 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:30.111649  3575 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.214097  3322 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.216112  3322 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.009299  3533 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:30.216162  3575 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.183377  3634 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.089856  3639 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:29.907317  3617 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.126967  3547 consensus_queue.cc:260] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.111294  3080 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:30.169344 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.169514 28756 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 25.
I20260709 13:14:30.234803 28161 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:30.198764  3478 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:29.865370  3612 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.190797  3255 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.191172 28127 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:30.192011 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 3006) took 1919 ms. Trace:
I20260709 13:14:30.277217 28158 rpcz_store.cc:276] 0709 13:14:28.272826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.273480 (+   654us) service_pool.cc:224] Handling call
0709 13:14:30.191772 (+1918292us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b5400376ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:30.191929 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085368064}
W20260709 13:14:30.277665 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 86565.1us
I20260709 13:14:30.278601 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.192831 28157 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 18. (term mismatch)
W20260709 13:14:30.193110  3585 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.747s	user 0.010s	sys 0.000s
I20260709 13:14:30.279556 28157 pending_rounds.cc:85] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 17
I20260709 13:14:30.192087  3602 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:30.185753  3299 consensus_meta.cc:287] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.886s	user 0.012s	sys 0.001s
I20260709 13:14:30.176842  3401 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.190996  3579 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:30.180666  3464 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:30.196264  3628 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:30.279927  3547 raft_consensus.cc:3060] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:30.171212  3525 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.212884  3275 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.232120  2403 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.279129  3205 raft_consensus.cc:2749] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:14:30.259989 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.283906  3080 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:30.306527  3531 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.289728  3526 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:30.289130 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:30.313531 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.293344 28226 raft_consensus.cc:2393] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 19: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:30.182406  3527 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:30.301744 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.299177  3516 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:30.176616 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 69146.9us
I20260709 13:14:30.222455  3520 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:30.280457  3255 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.307022  3602 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:30.314488 28079 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.012s	sys 0.021s
I20260709 13:14:30.233798  3639 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.279861 28157 pending_rounds.cc:107] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 21.18
W20260709 13:14:30.326923 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 226103us
I20260709 13:14:30.339507 28157 raft_consensus.cc:2889] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:30.326167 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 261535us
W20260709 13:14:30.336526 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.285701  3299 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.340713 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:14:30.343104 28199 leader_election.cc:336] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.623s (SENT)
I20260709 13:14:30.346740  3080 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.291877  3322 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:14:30.348426 28084 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2882) took 1037 ms. Trace:
I20260709 13:14:30.315382  3526 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:30.349112 28084 rpcz_store.cc:276] 0709 13:14:29.310861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.702794 (+391933us) service_pool.cc:224] Handling call
0709 13:14:29.703150 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:29.703550 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:29.703580 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:29.704143 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:14:29.935800 (+231657us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:29.935858 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.935956 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:30.348104 (+412148us) raft_consensus.cc:1685] finished
0709 13:14:30.348143 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.348343 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2663,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"thread_start_us":231171,"threads_started":1,"wal-append.queue_time_us":571978}
I20260709 13:14:30.351044  3526 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:30.352010  3299 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.352259  3526 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:30.344352  3531 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.354776  3531 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:30.360162  3255 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:30.361388  3650 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:14:30.381089 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.376372  3575 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:30.426982 28159 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:30.361991  3464 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:30.377297  3617 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:30.368892  3642 log.cc:927] Time spent T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:14:30.382333 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.451345 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.383219  3478 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.343241  3452 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.400631 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 3001) took 1909 ms. Trace:
I20260709 13:14:30.365475 28049 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:30.424938 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1496) took 2545 ms. Trace:
W20260709 13:14:30.370857  3609 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.330s	user 0.001s	sys 0.001s
I20260709 13:14:30.468930 28085 rpcz_store.cc:276] 0709 13:14:28.490942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.598911 (+107969us) service_pool.cc:224] Handling call
0709 13:14:29.677023 (+1078112us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:29.693285 (+ 16262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.392555 (+699270us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001ad450. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:30.400525 (+  7970us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2478267136}
I20260709 13:14:30.383488  3205 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 13:14:30.428546  3299 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.376475  3564 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:30.433162 28082 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40992 (request call id 2999) took 1942 ms. Trace:
I20260709 13:14:30.438190 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2886) took 1126 ms. Trace:
I20260709 13:14:30.444008 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.446556 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.478644 28086 rpcz_store.cc:276] 0709 13:14:29.312181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.754397 (+442216us) service_pool.cc:224] Handling call
0709 13:14:29.754892 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.755173 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:14:30.221951 (+466778us) log.cc:844] Serialized 14 byte log entry
0709 13:14:30.437702 (+215751us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.437789 (+    87us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:30.437855 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.437944 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.438108 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":541292,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":504,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":21193,"spinlock_wait_cycles":4480,"thread_start_us":681036,"threads_started":2}
W20260709 13:14:30.480163 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 114742us
I20260709 13:14:30.478415 28232 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:30.458367  3519 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:30.486976 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.454509  3503 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:30.489893 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.489966 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:30.464389 28186 heartbeater.cc:646] Failed to heartbeat to 127.26.83.254:42913 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.26.83.254:42913 timed out after 15.000s (SENT)
W20260709 13:14:30.360616 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 287361us
I20260709 13:14:30.471679 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.470323 28152 rpcz_store.cc:276] 0709 13:14:27.838438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.876058 (+ 37620us) service_pool.cc:224] Handling call
0709 13:14:27.876534 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:27.877074 (+   540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.877120 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:27.877746 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:14:28.292363 (+414617us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:28.292406 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.292476 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:30.192505 (+1900029us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:30.383165 (+190660us) raft_consensus.cc:1685] finished
0709 13:14:30.383188 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.383375 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8200,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":10265,"spinlock_wait_cycles":8147200,"thread_start_us":348862,"threads_started":1,"wal-append.queue_time_us":483032}
I20260709 13:14:30.492709 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:30.493145 28230 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:30.493350 28757 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:30.482757 28082 rpcz_store.cc:276] 0709 13:14:28.490393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.589121 (+ 98728us) service_pool.cc:224] Handling call
0709 13:14:28.589950 (+   829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.072408 (+1482458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.432818 (+360410us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54001ae5d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:30.433068 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3246200192}
I20260709 13:14:30.488453 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:30.494449 28231 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:30.459986  3508 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:30.493782 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.498059 28152 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:30.499794 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.500649 28152 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:30.506039 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:30.431975  3575 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:30.478344  3546 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.434s	user 0.003s	sys 0.011s
I20260709 13:14:30.490684 28232 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:30.508386 28759 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:14:30.488221  3617 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:30.511279 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:30.483906  3612 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.498668  3640 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.478724  3452 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.504103  3628 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:30.514907  2403 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:30.516191 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
W20260709 13:14:30.518489  3653 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:14:30.505261  3531 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:14:30.538553 28229 raft_consensus.cc:2410] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:14:30.539170 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3477) took 1735 ms. Trace:
I20260709 13:14:30.554409 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.555476 28229 raft_consensus.cc:2410] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:14:30.539387 28049 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:30.554833 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.515669 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.475360  3478 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.481199  3299 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.561969 28229 pending_rounds.cc:85] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 11
I20260709 13:14:30.562372 28229 pending_rounds.cc:107] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:14:30.562635 28229 raft_consensus.cc:2889] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:30.511195 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.563515 28152 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:30.507865  3508 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:30.515900  3546 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.471s	user 0.003s	sys 0.012s
I20260709 13:14:30.543201  2403 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.546890 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.570011  3508 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:30.570271 28081 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41006 (request call id 2877) took 2058 ms. Trace:
I20260709 13:14:30.570761 28081 rpcz_store.cc:276] 0709 13:14:28.511529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.957064 (+445535us) service_pool.cc:224] Handling call
0709 13:14:28.957405 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:29.057675 (+100270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:29.057708 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:30.215900 (+1158192us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b2c000014c8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:14:30.216466 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:14:30.454635 (+238169us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:30.454691 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.454800 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:30.569951 (+115151us) raft_consensus.cc:1685] finished
0709 13:14:30.569970 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.570183 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7178,"raft.queue_time_us":105898,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":11981,"spinlock_wait_cycles":6180736,"thread_start_us":330300,"threads_started":2,"wal-append.queue_time_us":334920}
I20260709 13:14:30.570787  3656 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:30.572983  3656 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.573375  3656 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:30.575870  3464 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:30.549350 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.577276 28758 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:30.550086 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.550194 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.553951 28153 rpcz_store.cc:276] 0709 13:14:28.803472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.803686 (+   214us) service_pool.cc:224] Handling call
0709 13:14:28.837699 (+ 34013us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:28.846046 (+  8347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:28.846100 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:28.846893 (+   793us) log.cc:844] Serialized 26 byte log entry
0709 13:14:28.944910 (+ 98017us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:14:28.944954 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.945032 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:30.370020 (+1424988us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:30.535080 (+165060us) raft_consensus.cc:1685] finished
0709 13:14:30.535105 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.539074 (+  3969us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2743,"raft.queue_time_us":10831,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":10261,"spinlock_wait_cycles":51206528,"thread_start_us":56434,"threads_started":1,"wal-append.queue_time_us":139169}
I20260709 13:14:30.483572  3527 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:30.579101 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:30.579576  3527 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.580088  3478 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.552026  3628 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.558560 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.544589  3658 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.583628  3658 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:30.504571  3503 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.560113  3634 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.556831 28232 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:30.497658  3519 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.539875  3275 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.567692 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:30.567912 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.601907 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.601996 28756 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:30.576615 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:30.603158 28154 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:30.544023  3640 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:30.559195  3639 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.591499 28231 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:30.552320  3633 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.544909  3322 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:30.552120  3617 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:30.608264  2403 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.617014 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.595769 28757 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:30.609530 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.611812 28159 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 28.
I20260709 13:14:30.636830 28049 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:30.638309  2403 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:30.597649  3525 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.649820 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.651058 28152 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:30.657821  2403 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:30.658020 28756 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:30.602787  3464 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.659904 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.658749  2403 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.664436 28154 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
W20260709 13:14:30.665612 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 347388us
I20260709 13:14:30.668846 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.675519 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.677094 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:30.677767 28756 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:30.684703 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.685568 28758 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:30.618561  3658 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:30.685892 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.657330  3639 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.670930  3508 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.695254  3508 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:30.720019 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.619168  3322 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.671304  3525 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:30.671996  3520 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:30.731416 28231 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:14:30.720583 28756 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:30.739339  3575 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.620553  3633 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:30.739995  3665 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:14:30.742327  3322 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.742944  3525 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:30.744621  3663 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:14:30.745548  3664 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:14:30.746025  3508 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.749640  3628 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:30.751091  3640 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:30.751698  3617 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.754261 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.755275 28759 raft_consensus.cc:2410] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 22 because replica has last-logged OpId of term: 20 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 17.
I20260709 13:14:30.756073 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.757094 28756 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 24.
I20260709 13:14:30.671453  3464 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:30.763056 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.761893  3520 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:30.772832 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:30.757719  3575 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:30.770625  3633 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:30.697913  3639 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:30.765424  3520 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.792820  3656 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.833490 28229 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:14:30.864718 28229 pending_rounds.cc:85] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 8
I20260709 13:14:30.865093 28229 pending_rounds.cc:107] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:14:30.865399 28229 raft_consensus.cc:2889] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:30.865797 28229 pending_rounds.cc:107] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:14:30.866092 28229 raft_consensus.cc:2889] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:30.842249 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.809237 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 3011) took 2438 ms. Trace:
I20260709 13:14:30.860843 28082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:30.839874 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:30.836577 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:30.838320 28730 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.627s (SENT) [suppressed 19 similar messages]
I20260709 13:14:30.838930 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39698 (request call id 2874) took 1067 ms. Trace:
I20260709 13:14:30.874542  3510 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.874902 28226 rpcz_store.cc:276] 0709 13:14:29.771186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.771775 (+   589us) service_pool.cc:224] Handling call
0709 13:14:30.838641 (+1066866us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b5400305554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:30.838852 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071979776}
I20260709 13:14:30.875257  3510 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:30.878439 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.879637 28086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:30.816180 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:30.867328 28125 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:30.816972  3668 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:14:30.902951 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:30.835336  3547 consensus_meta.cc:287] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.000s
I20260709 13:14:30.909821 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.906598 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:30.902658 28081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:30.851970  3525 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.907222 28085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:30.867789 28155 rpcz_store.cc:276] 0709 13:14:28.370556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.465264 (+ 94708us) service_pool.cc:224] Handling call
0709 13:14:28.465624 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.465866 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:28.581168 (+115302us) log.cc:844] Serialized 14 byte log entry
0709 13:14:30.585257 (+2004089us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b180010aec0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59d206f3c7 00007f59ce6819a7 00007f59ce6910bf 00007f59ce691038
0709 13:14:30.585460 (+   203us) log.cc:844] Serialized 14 byte log entry
0709 13:14:30.805006 (+219546us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.805084 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:30.805154 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.808981 (+  3827us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.809162 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121817,"prepare.queue_time_us":1915974,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":2644,"raft.queue_time_us":19148,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":20030,"spinlock_wait_cycles":27209088,"thread_start_us":433011,"threads_started":3,"wal-append.queue_time_us":198559,"wal-append.run_cpu_time_us":1150,"wal-append.run_wall_time_us":1014450}
I20260709 13:14:30.851878  3503 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.833426  3519 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.826545  3639 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:30.913713  3667 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:30.913962  3656 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:30.914106 28080 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36768 (request call id 1660) took 1814 ms (client timeout 2092 ms). Trace:
I20260709 13:14:30.914641 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.831964  3464 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:30.953924 28226 raft_consensus.cc:2410] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 13:14:30.915343 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.954442 28229 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:30.894485  3640 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:30.955348 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:30.916610 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:30.952886 28080 rpcz_store.cc:269] 0709 13:14:29.100129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.132544 (+ 32415us) service_pool.cc:224] Handling call
0709 13:14:30.913907 (+1781363us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400209254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:30.914050 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670574464}
I20260709 13:14:30.917407 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:30.919296  3520 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:30.922677 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3483) took 1127 ms. Trace:
I20260709 13:14:30.924918  3442 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:30.912027  3322 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:14:30.805656  3455 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:30.963752  3608 log.cc:927] Time spent T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 1.921s	user 0.005s	sys 0.000s
I20260709 13:14:30.910607 28227 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:30.929245 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.934645 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:30.936825 28086 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:30.913419 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 3020) took 2036 ms. Trace:
I20260709 13:14:30.791340  3575 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:30.817601  3518 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.958235 28232 raft_consensus.cc:2410] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:14:30.952634  3667 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.958076 28161 rpcz_store.cc:276] 0709 13:14:29.795688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.921896 (+1126208us) service_pool.cc:224] Handling call
0709 13:14:30.922201 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.922378 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:30.922402 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.922442 (+    40us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:30.922472 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.922524 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.922620 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.910171  3672 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:30.903371  3080 consensus_queue.cc:1059] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:14:31.931306 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:30.967867 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:31.043255 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:31.934103 28226 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:14:31.868178 28079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:30.955946 28229 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:31.935912 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3813) took 1020 ms. Trace:
I20260709 13:14:31.936224 28229 rpcz_store.cc:276] 0709 13:14:30.915260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.955091 (+ 39831us) service_pool.cc:224] Handling call
0709 13:14:31.935854 (+980763us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:30.992031 28231 raft_consensus.cc:2468] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
W20260709 13:14:31.289099 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 353222us
W20260709 13:14:31.282400 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 369118us
I20260709 13:14:31.219610 28078 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1658) took 2133 ms. Trace:
I20260709 13:14:31.016332 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:31.160439 28761 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:31.938123 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:31.938908 28160 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
W20260709 13:14:31.272588 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.939409 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3492) took 1016 ms. Trace:
W20260709 13:14:31.939877 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.939746 28160 rpcz_store.cc:276] 0709 13:14:30.923337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.015940 (+ 92603us) service_pool.cc:224] Handling call
0709 13:14:31.939354 (+923414us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.210417 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:31.212472 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3481) took 1417 ms. Trace:
I20260709 13:14:31.940542 28154 rpcz_store.cc:276] 0709 13:14:29.795026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.671716 (+876690us) service_pool.cc:224] Handling call
0709 13:14:30.672040 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.749123 (+ 77083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:30.749153 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.749203 (+    50us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:14:30.749234 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.212134 (+462900us) spinlock_profiling.cc:243] Waited 76.8 ms on lock 0x7b54000a3cd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:31.212200 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.212393 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176711936}
I20260709 13:14:31.211386 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:31.160625 28764 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:14:31.924998 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:31.921190 28157 rpcz_store.cc:276] 0709 13:14:28.877077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.913856 (+ 36779us) service_pool.cc:224] Handling call
0709 13:14:30.192443 (+1278587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.340094 (+147651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.913189 (+573095us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400376ed0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:30.913358 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2895854976}
I20260709 13:14:31.923695 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3809) took 1009 ms. Trace:
I20260709 13:14:31.244484 28228 raft_consensus.cc:2468] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:31.431988 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3479) took 1637 ms. Trace:
I20260709 13:14:31.942952 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3810) took 1028 ms. Trace:
I20260709 13:14:31.947796 28161 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:31.947235 28153 rpcz_store.cc:276] 0709 13:14:29.794509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.579973 (+785464us) service_pool.cc:224] Handling call
0709 13:14:30.931830 (+351857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.932037 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:30.932065 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.932111 (+    46us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:14:30.932154 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.932220 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.431691 (+499471us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b5400238550. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:31.431897 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808431360}
I20260709 13:14:31.949209 28230 raft_consensus.cc:2393] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 18: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:31.929538 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:31.950430 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.949678 28227 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3811) took 1035 ms. Trace:
W20260709 13:14:31.950876 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.929845 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:31.946909 28079 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2904) took 1038 ms. Trace:
I20260709 13:14:31.951769 28079 rpcz_store.cc:276] 0709 13:14:30.908218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.908400 (+   182us) service_pool.cc:224] Handling call
0709 13:14:31.946855 (+1038455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.948021 28232 rpcz_store.cc:276] 0709 13:14:30.913962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.917133 (+  3171us) service_pool.cc:224] Handling call
0709 13:14:31.923581 (+1006448us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.953639 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:31.955091 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:31.955730 28757 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:31.955531 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:31.949785 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3812) took 1034 ms. Trace:
I20260709 13:14:31.956689 28229 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:31.949909 28158 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:31.959338 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1502) took 2141 ms. Trace:
I20260709 13:14:31.938043 28078 rpcz_store.cc:276] 0709 13:14:29.086416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.454343 (+367927us) service_pool.cc:224] Handling call
0709 13:14:29.455533 (+  1190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:29.486660 (+ 31127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:29.486688 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:29.519380 (+ 32692us) log.cc:844] Serialized 26 byte log entry
0709 13:14:29.938729 (+419349us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:14:29.938799 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.191674 (+1252875us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b540017c854. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:31.191740 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:31.191810 (+    70us) raft_consensus.cc:1685] finished
0709 13:14:31.191830 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.219498 (+ 27668us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":499,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":9205,"spinlock_wait_cycles":70647168,"thread_start_us":277274,"threads_started":1,"wal-append.queue_time_us":644446,"wal-append.run_cpu_time_us":2567,"wal-append.run_wall_time_us":1140885}
I20260709 13:14:31.960134 28155 rpcz_store.cc:276] 0709 13:14:29.817624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.952532 (+2134908us) service_pool.cc:224] Handling call
0709 13:14:31.957373 (+  4841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.959013 (+  1640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:31.959040 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:31.959081 (+    41us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:31.959126 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.959190 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.959289 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.947656 28233 rpcz_store.cc:276] 0709 13:14:30.914357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.967556 (+ 53199us) service_pool.cc:224] Handling call
0709 13:14:31.942867 (+975311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.931800  3223 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:31.962666 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1508) took 1840 ms. Trace:
I20260709 13:14:31.962370  3223 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:31.963027 28153 rpcz_store.cc:276] 0709 13:14:30.122278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.961856 (+1839578us) service_pool.cc:224] Handling call
0709 13:14:31.962190 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.962353 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:31.962377 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:31.962418 (+    41us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:31.962460 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.962515 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.962615 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.945256  3672 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:31.964380 28153 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:31.965619 28157 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:31.061447  3674 log.cc:927] Time spent T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 13:14:31.071813  3676 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:14:31.959278 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1504) took 2136 ms. Trace:
I20260709 13:14:31.966480 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1507) took 1866 ms. Trace:
I20260709 13:14:31.070446  3455 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:31.019285  3681 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:14:31.059525  3299 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:31.282488  3510 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:31.956924 28230 rpcz_store.cc:276] 0709 13:14:30.914982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.210657 (+295675us) service_pool.cc:224] Handling call
0709 13:14:31.949727 (+739070us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.968710 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1511) took 1778 ms. Trace:
I20260709 13:14:31.985245 28153 rpcz_store.cc:276] 0709 13:14:30.189941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.963921 (+1773980us) service_pool.cc:224] Handling call
0709 13:14:31.964218 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.968473 (+  4255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.968657 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.226817  3538 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:31.910720  3513 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:31.302285  3452 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:31.968151 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1509) took 1781 ms. Trace:
I20260709 13:14:31.973173 28157 rpcz_store.cc:276] 0709 13:14:30.100572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.965177 (+1864605us) service_pool.cc:224] Handling call
0709 13:14:31.965454 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.966236 (+   782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.966423 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.910874  3533 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:31.995692 28155 rpcz_store.cc:276] 0709 13:14:30.187149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.967327 (+1780178us) service_pool.cc:224] Handling call
0709 13:14:31.967597 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.967780 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:31.967805 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:31.967842 (+    37us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:31.967870 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.967979 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.968101 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.997370  3538 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:31.951241 28199 connection.cc:256] RPC call timeout handler was delayed by -1.3902452960001028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 13:14:32.004464 28199 connection.cc:256] RPC call timeout handler was delayed by -1.3085387480000463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2871s for a response
W20260709 13:14:32.004884 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2056920390000414s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
W20260709 13:14:32.005187 28199 connection.cc:256] RPC call timeout handler was delayed by -1.0739005780001207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 13:14:32.006072 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:31.972421 28160 rpcz_store.cc:276] 0709 13:14:29.822697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.958172 (+2135475us) service_pool.cc:224] Handling call
0709 13:14:31.958580 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.958797 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:31.958835 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:31.958894 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:31.958942 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.959014 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.959180 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.979555 28161 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 25.
I20260709 13:14:32.008899 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3493) took 1070 ms. Trace:
I20260709 13:14:32.009251 28161 rpcz_store.cc:276] 0709 13:14:30.938798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.921717 (+982919us) service_pool.cc:224] Handling call
0709 13:14:32.008837 (+ 87120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.234345  3627 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:31.057596  3620 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 1.730s	user 0.006s	sys 0.000s
I20260709 13:14:31.296422  3453 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:31.936288  3667 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:31.993007 28158 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:32.019786 28161 raft_consensus.cc:1240] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:31.950996 28227 rpcz_store.cc:276] 0709 13:14:30.914667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.965251 (+ 50584us) service_pool.cc:224] Handling call
0709 13:14:31.949535 (+984284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.997397 28155 raft_consensus.cc:1240] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: [20.18-20.18]
I20260709 13:14:32.011754 28230 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:14:32.007186 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:32.026909 28230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3818) took 1110 ms. Trace:
I20260709 13:14:32.029558 28230 rpcz_store.cc:276] 0709 13:14:30.974277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.994931 (+1020654us) service_pool.cc:224] Handling call
0709 13:14:32.007859 (+ 12928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.026616 (+ 18757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.026839 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:31.983726 28154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8"
dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
 from {username='slave'} at 127.0.0.1:43184
I20260709 13:14:32.022461 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 3037) took 1524 ms. Trace:
I20260709 13:14:32.031154 28154 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:32.031323 28160 rpcz_store.cc:276] 0709 13:14:30.497903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.010652 (+1512749us) service_pool.cc:224] Handling call
0709 13:14:32.021974 (+ 11322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.022151 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:32.022177 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:32.022218 (+    41us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:32.022250 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.022301 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:32.022407 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.032207 28154 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:32.023124 28226 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:31.961105  3518 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.020876 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3491) took 1097 ms. Trace:
I20260709 13:14:32.034188 28158 rpcz_store.cc:276] 0709 13:14:30.923058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.210020 (+286962us) service_pool.cc:224] Handling call
0709 13:14:32.020814 (+810794us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.035744 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 3033) took 1539 ms. Trace:
I20260709 13:14:32.036199 28155 rpcz_store.cc:276] 0709 13:14:30.496493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.996805 (+1500312us) service_pool.cc:224] Handling call
0709 13:14:31.997130 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:32.033763 (+ 36633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.035689 (+  1926us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.037930 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3486) took 1116 ms. Trace:
I20260709 13:14:32.038406 28155 rpcz_store.cc:276] 0709 13:14:30.921663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.037166 (+1115503us) service_pool.cc:224] Handling call
0709 13:14:32.037439 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.037611 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:32.037636 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:32.037691 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:32.037722 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.037774 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:32.037877 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.040174 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3487) took 1118 ms. Trace:
I20260709 13:14:32.033264 28226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3816) took 1117 ms. Trace:
I20260709 13:14:32.041803 28226 rpcz_store.cc:276] 0709 13:14:30.916140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.967542 (+ 51402us) service_pool.cc:224] Handling call
0709 13:14:32.033205 (+1065663us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.041441 28158 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:14:32.040593 28155 rpcz_store.cc:276] 0709 13:14:30.921950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.039416 (+1117466us) service_pool.cc:224] Handling call
0709 13:14:32.039687 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.039856 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:32.039888 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:32.039934 (+    46us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:32.039962 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.040013 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:32.040124 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.012144  3510 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:32.046185 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3480) took 2251 ms. Trace:
I20260709 13:14:32.046643 28155 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:14:32.047619 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3488) took 1125 ms. Trace:
I20260709 13:14:32.048355 28155 rpcz_store.cc:276] 0709 13:14:30.922225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.043732 (+1121507us) service_pool.cc:224] Handling call
0709 13:14:32.044018 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.047369 (+  3351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.047565 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.050190 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3494) took 1110 ms. Trace:
I20260709 13:14:32.046800 28152 rpcz_store.cc:276] 0709 13:14:29.794774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.667648 (+872874us) service_pool.cc:224] Handling call
0709 13:14:30.668005 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:30.668394 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:30.685614 (+ 17220us) log.cc:844] Serialized 14 byte log entry
0709 13:14:30.759591 (+ 73977us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:30.771706 (+ 12115us) log.cc:844] Serialized 26 byte log entry
0709 13:14:30.771907 (+   201us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:30.771942 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.772027 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:32.045881 (+1273854us) raft_consensus.cc:1685] finished
0709 13:14:32.045910 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:32.046097 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24610,"prepare.queue_time_us":17367,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":1251,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":7112,"thread_start_us":65100,"threads_started":1,"wal-append.queue_time_us":122917}
I20260709 13:14:32.050697 28155 rpcz_store.cc:276] 0709 13:14:30.939906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.049292 (+1109386us) service_pool.cc:224] Handling call
0709 13:14:32.049585 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.049883 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:32.049907 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:32.049949 (+    42us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:32.049987 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.050041 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:32.050140 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.075398 28231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39712 (request call id 3805) took 1242 ms. Trace:
I20260709 13:14:32.075872 28231 rpcz_store.cc:276] 0709 13:14:30.832814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.836205 (+  3391us) service_pool.cc:224] Handling call
0709 13:14:32.075213 (+1239008us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000e8b54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:32.075335 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273040128}
I20260709 13:14:31.995555  3513 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:32.077744 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39698 (request call id 2864) took 3030 ms. Trace:
I20260709 13:14:32.078388 28234 rpcz_store.cc:276] 0709 13:14:29.047431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.061876 (+ 14445us) service_pool.cc:224] Handling call
0709 13:14:29.062190 (+   314us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:29.078893 (+ 16703us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:29.078924 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:29.079497 (+   573us) log.cc:844] Serialized 52 byte log entry
0709 13:14:29.236445 (+156948us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:29.236492 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.236564 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:30.965415 (+1728851us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:32.077237 (+1111822us) raft_consensus.cc:1685] finished
0709 13:14:32.077282 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:32.077629 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15176,"raft.queue_time_us":15363,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":4262,"spinlock_wait_cycles":43759360,"thread_start_us":119375,"threads_started":1,"wal-append.queue_time_us":208709}
I20260709 13:14:32.021485  3627 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.021121  3453 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:32.086159 28154 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.090690 28160 raft_consensus.cc:1240] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:32.097534 28154 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:32.098630 28157 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:14:32.052990  3510 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.112079 28086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2900) took 1260 ms. Trace:
I20260709 13:14:32.114856 28154 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43184 (request call id 1505) took 2017 ms. Trace:
I20260709 13:14:32.115425  3223 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:32.119933  3176 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:32.122143 28158 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1516) took 1515 ms. Trace:
I20260709 13:14:32.122578 28158 rpcz_store.cc:276] 0709 13:14:30.606611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.035058 (+1428447us) service_pool.cc:224] Handling call
0709 13:14:32.035347 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.042472 (+  7125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.121792 (+ 79320us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b5400237154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:32.122058 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13261184}
W20260709 13:14:32.124334 28228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44070 (request call id 1725) took 2888 ms (client timeout 1627 ms). Trace:
W20260709 13:14:32.124775 28228 rpcz_store.cc:269] 0709 13:14:29.236389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.237512 (+  1123us) service_pool.cc:224] Handling call
0709 13:14:30.782351 (+1544839us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b54001f5d28. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d4454924 00007f59d363d227 00007f59d363cab8 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:32.124118 (+1341767us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54001f5ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:32.124236 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348150016}
I20260709 13:14:32.126128  3575 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:32.112545 28086 rpcz_store.cc:276] 0709 13:14:30.851957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.852182 (+   225us) service_pool.cc:224] Handling call
0709 13:14:32.111864 (+1259682us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b540027c9d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:32.112016 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70511360}
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 6
I20260709 13:14:32.115439 28154 rpcz_store.cc:276] 0709 13:14:30.081161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.954497 (+1873336us) service_pool.cc:224] Handling call
0709 13:14:32.098716 (+144219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.141580  3538 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:14:32.120501  3176 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:32.150177  3683 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.124s	user 0.002s	sys 0.000s
I20260709 13:14:32.144433  3538 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:32.239133 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1513) took 1653 ms. Trace:
I20260709 13:14:32.227391 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 3034) took 1730 ms. Trace:
I20260709 13:14:32.241187 28160 rpcz_store.cc:276] 0709 13:14:30.585981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.032844 (+1446863us) service_pool.cc:224] Handling call
0709 13:14:32.045928 (+ 13084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.091832 (+ 45904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.238744 (+146912us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b54001978d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:32.239024 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102506752}
I20260709 13:14:32.241489 28161 rpcz_store.cc:276] 0709 13:14:30.496644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.009896 (+1513252us) service_pool.cc:224] Handling call
0709 13:14:32.019596 (+  9700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.020191 (+   595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.227125 (+206934us) spinlock_profiling.cc:243] Waited 9.38 ms on lock 0x7b54003764d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:32.227327 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21582592}
I20260709 13:14:32.203320  3685 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:32.203790 26959 tablet_server.cc:179] TabletServer@127.26.83.193:0 shutting down...
I20260709 13:14:32.201264  3538 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.199558  3672 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:32.217008  3688 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.225728  3640 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:14:32.228135  3684 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:14:32.215510  3627 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:32.229566  3510 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:32.290778  3510 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.291231  3510 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:32.223522  3518 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:32.280151  3538 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:32.283628  3640 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.287580  3688 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:32.288594 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 358831us
I20260709 13:14:32.275197  3223 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:32.298470 28124 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:32.302462 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:32.279474 28730 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.406s	user 0.006s	sys 0.009s
W20260709 13:14:32.304672 28730 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.431s	user 0.007s	sys 0.009s
W20260709 13:14:32.305086 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 1.46668e+06us
I20260709 13:14:32.289656  3672 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.293624  3685 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.298446  3640 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:32.320941 28085 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:14:32.338944 28085 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36768 (request call id 1661) took 1321 ms. Trace:
I20260709 13:14:32.339502 28085 rpcz_store.cc:276] 0709 13:14:31.017353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.320264 (+1302911us) service_pool.cc:224] Handling call
0709 13:14:32.320587 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.338577 (+ 17990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.338855 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.345237  3453 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:32.345996  3453 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:32.346377  3453 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:32.346755  3453 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:14:32.350106  3686 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.342s	user 0.000s	sys 0.002s
I20260709 13:14:32.352702 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 3032) took 1856 ms. Trace:
I20260709 13:14:32.353366 28157 rpcz_store.cc:276] 0709 13:14:30.496308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.990138 (+1493830us) service_pool.cc:224] Handling call
0709 13:14:32.004355 (+ 14217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:32.101869 (+ 97514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.352293 (+250424us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b5400255154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:32.352609 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215925632}
I20260709 13:14:32.358525  3513 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:32.359349  3513 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.360080  3627 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:32.360328 28728 leader_election.cc:336] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.863s (SENT)
I20260709 13:14:32.362149 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:32.363535 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:32.364454 28764 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:32.367048 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:32.370772 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:32.371176 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.371534 28764 raft_consensus.cc:3055] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:32.372696 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:32.371951 28764 raft_consensus.cc:740] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:32.318143  3518 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:32.358575  3453 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.397822 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:32.397982 28080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0faaace2b36c4bcebf2fa6835235e92b" is_pre_election: true
I20260709 13:14:32.398191 28764 consensus_queue.cc:260] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.398768 28080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41006 (request call id 2905) took 1490 ms. Trace:
I20260709 13:14:32.399225 28764 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:32.399551 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:32.399581  3453 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:32.392618  3489 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:32.401233  3701 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:32.359902  3513 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:32.401863  3701 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:32.403558  3513 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:32.403932  3701 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:32.399091 28080 rpcz_store.cc:276] 0709 13:14:30.975209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.360598 (+1385389us) service_pool.cc:224] Handling call
0709 13:14:32.398709 (+ 38111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:32.404203 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:32.404630  3692 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:14:32.405241 28758 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:32.405578  3692 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:32.406008  3692 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.405728  3455 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:32.407482  3455 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:32.407840  3455 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:32.410030  3688 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:32.411417  3682 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.400s	user 0.003s	sys 0.003s
I20260709 13:14:32.413051 28156 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3467) took 4072 ms. Trace:
I20260709 13:14:32.413391 28156 rpcz_store.cc:276] 0709 13:14:28.340446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.375049 (+ 34603us) service_pool.cc:224] Handling call
0709 13:14:28.409399 (+ 34350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:28.454018 (+ 44619us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:28.464438 (+ 10420us) log.cc:844] Serialized 14 byte log entry
0709 13:14:30.817727 (+2353289us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b18003a6420. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb56ce07 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59ce21120b 00007f59ce216d23 00007f59ce21723a 00007f59ce681a2a
0709 13:14:30.954445 (+136718us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:31.019005 (+ 64560us) log.cc:844] Serialized 26 byte log entry
0709 13:14:31.019203 (+   198us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:31.019246 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.019343 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:32.404619 (+1385276us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:32.412842 (+  8223us) raft_consensus.cc:1685] finished
0709 13:14:32.412863 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:32.412995 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25549,"prepare.queue_time_us":216,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":34127,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":9467,"spinlock_wait_cycles":35441280,"thread_start_us":135218,"threads_started":1,"wal-append.queue_time_us":3492383}
I20260709 13:14:32.415414  3455 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.416749  3455 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:32.401554  3489 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.419121  3489 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:32.406390  3692 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:32.319764  3672 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.428081 28764 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:32.366422  3176 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:32.318390  3223 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.395118  3627 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.341477  3685 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.429381 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:32.485037 28758 raft_consensus.cc:2468] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 25.
I20260709 13:14:32.451757 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:32.679626 28764 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:32.464337 27975 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 195) took 16993 ms (client timeout 15000 ms). Trace:
I20260709 13:14:32.682219  3455 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:32.682854  3455 raft_consensus.cc:697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:32.684077  3455 consensus_queue.cc:237] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:32.685499 28127 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.017s	sys 0.046s
W20260709 13:14:32.686124 28127 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.017s	sys 0.046s
W20260709 13:14:32.546975 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.686614 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 821167us
W20260709 13:14:32.685510 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 256230us
W20260709 13:14:32.688808 28198 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.021s	sys 0.043s
W20260709 13:14:32.689145 28198 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.021s	sys 0.043s
I20260709 13:14:32.691725 28757 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 24. Current term is 25. Ops: []
W20260709 13:14:32.689426 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 824047us
W20260709 13:14:32.693454 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.686189 28755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.021s	sys 0.037s
W20260709 13:14:32.694609 28755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.021s	sys 0.038s
I20260709 13:14:32.694519 28198 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:32.695758 28755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 1351) took 1446 ms. Trace:
W20260709 13:14:32.710028 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 154847us
W20260709 13:14:32.711162 28050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.002s	sys 0.015s
I20260709 13:14:32.711472 28124 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:32.392935  3255 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:32.711517 28050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.002s	sys 0.015s
I20260709 13:14:32.726006 28198 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:32.727725 28229 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.009s	sys 0.009s
W20260709 13:14:32.728408 28229 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.009s	sys 0.010s
I20260709 13:14:32.712291 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:32.680466  3223 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:32.732575 28199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.004s	sys 0.015s
W20260709 13:14:32.732990 28199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.004s	sys 0.015s
W20260709 13:14:32.733538 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 726358us
I20260709 13:14:32.734372 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:32.730188 28201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.005s	sys 0.013s
W20260709 13:14:32.534819  3533 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.003s
W20260709 13:14:32.709561  3080 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.006s	sys 0.010s
I20260709 13:14:32.425074  3518 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:32.696234 28755 rpcz_store.cc:276] 0709 13:14:31.249195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.931003 (+681808us) service_pool.cc:224] Handling call
0709 13:14:31.931984 (+   981us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:31.932715 (+   731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:31.932774 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:31.943130 (+ 10356us) log.cc:844] Serialized 26 byte log entry
0709 13:14:32.695091 (+751961us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:32.695146 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.695450 (+   304us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:32.695515 (+    65us) raft_consensus.cc:1685] finished
0709 13:14:32.695532 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:32.695687 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":22277,"thread_start_us":729880,"threads_started":1,"wal-append.queue_time_us":717994}
W20260709 13:14:32.736899 28201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.006s	sys 0.014s
I20260709 13:14:32.491062  3672 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:32.751965 28235 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:32.739609 28764 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:32.488783 27982 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:32.465886  3510 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:32.763113 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.709724  3688 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:32.727957  3701 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:14:32.749562  3080 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.006s	sys 0.010s
I20260709 13:14:32.764604 28198 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:14:32.774169 28127 connection.cc:256] RPC call timeout handler was delayed by -1.1616465699999026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 13:14:32.777745 28127 connection.cc:256] RPC call timeout handler was delayed by -1.1130465700000514s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
W20260709 13:14:32.778273 28127 connection.cc:256] RPC call timeout handler was delayed by -1.0329465699999218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 13:14:32.775660 28052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.006s	sys 0.018s
I20260709 13:14:32.731305  3689 raft_consensus.cc:2804] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:32.773093 27982 catalog_manager.cc:5697] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 12 to 19, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 19 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:32.526187  3667 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:32.586874  3640 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.778393 28198 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:32.636345  3538 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.762924  3223 raft_consensus.cc:2764] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:14:32.762063  3299 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:32.782541 28052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.006s	sys 0.019s
I20260709 13:14:32.770932  3533 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:32.782792 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:32.762177  3697 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:32.787189 27982 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 14 to 20, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 20 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:32.741983  3255 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:32.791546 28125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.021s	sys 0.045s
W20260709 13:14:32.791879 28125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.021s	sys 0.045s
W20260709 13:14:32.792420 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 838772us
I20260709 13:14:32.801240 28235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3788) took 2372 ms. Trace:
W20260709 13:14:32.782943  3527 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.050s	sys 0.073s
I20260709 13:14:32.775079  3176 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:32.803367 28153 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.013s	sys 0.017s
I20260709 13:14:32.777180  3672 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:32.814260 28153 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.013s	sys 0.018s
I20260709 13:14:32.816193 28159 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:32.801765 28235 rpcz_store.cc:276] 0709 13:14:30.428477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.428647 (+   170us) service_pool.cc:224] Handling call
0709 13:14:30.429021 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.754062 (+2325041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.800913 (+ 46851us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b54003bf6d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:32.801152 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5341485824}
I20260709 13:14:32.820832  3176 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.797071  3299 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:32.789798  3689 raft_consensus.cc:697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:32.798784  3701 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:32.811560  3080 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:32.777181  3688 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.821604  3533 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:32.809685  3527 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.050s	sys 0.073s
I20260709 13:14:32.776865  3510 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 13:14:32.852038 27982 catalog_manager.cc:5697] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 12 to 22. New cstate: current_term: 22 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:32.608450  3692 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.852675  3667 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:32.854568  3080 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:32.875298  3538 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:32.875648  3688 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:32.871789 27982 catalog_manager.cc:5697] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:32.821074  3672 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:32.868777  3701 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:32.944650 28233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.006s	sys 0.015s
W20260709 13:14:32.953353 28233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.006s	sys 0.016s
W20260709 13:14:32.946606 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 155425us
I20260709 13:14:32.956079 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:32.956390 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:32.956672 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:32.958282 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:32.959146 28161 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 28.
I20260709 13:14:32.960659 28160 raft_consensus.cc:2393] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 25: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:32.961465 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:32.877447  3080 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:32.964999 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 252790us
I20260709 13:14:32.830147  3518 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:32.968372 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:32.943153  3538 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:32.979988 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.001580 28233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3817) took 2085 ms. Trace:
I20260709 13:14:33.002434 28160 raft_consensus.cc:2410] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:14:32.981755 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.002616 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.003298 28161 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:32.984308 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.003773 28154 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:32.984380 28158 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:14:33.002715 28727 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.005s	sys 0.013s
W20260709 13:14:33.005159 28727 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.006s	sys 0.013s
I20260709 13:14:32.986205 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.005934 28155 raft_consensus.cc:2393] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 16: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:32.869642  3533 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:33.008863 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 215782us
W20260709 13:14:33.011121 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:33.012401 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.002038 28233 rpcz_store.cc:276] 0709 13:14:30.916434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.961874 (+1045440us) service_pool.cc:224] Handling call
0709 13:14:31.963291 (+  1417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.963564 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:32.392752 (+429188us) log.cc:844] Serialized 14 byte log entry
0709 13:14:33.001260 (+608508us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:33.001310 (+    50us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:33.001352 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.001431 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:33.001528 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47190,"prepare.queue_time_us":823644,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":4359,"thread_start_us":989596,"threads_started":2}
I20260709 13:14:33.021004 27982 catalog_manager.cc:5697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 8 to 17. New cstate: current_term: 17 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:33.027451 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:33.054713 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 320412us
W20260709 13:14:33.042129 27968 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.049s	sys 0.066s
I20260709 13:14:33.057725 28156 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:14:33.058267 27968 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.049s	sys 0.066s
I20260709 13:14:33.063205 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3495) took 2122 ms. Trace:
I20260709 13:14:33.072661 27982 catalog_manager.cc:5697] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 13 to 18, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 18 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:33.029158  3627 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:33.070536 28159 rpcz_store.cc:276] 0709 13:14:30.940204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.930997 (+990793us) service_pool.cc:224] Handling call
0709 13:14:33.031289 (+1100292us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54001afed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:33.062832 (+ 31543us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025766656}
W20260709 13:14:33.070488 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 292167us
W20260709 13:14:33.069000 28232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.040s	sys 0.062s
W20260709 13:14:33.095999 28232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.040s	sys 0.063s
I20260709 13:14:33.067613  3685 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:33.112299 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.149354  3538 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:33.153990  3710 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:33.154666  3710 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.126920  3692 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:33.128870  3533 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:33.129050  3712 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.806250  3697 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.157399  3712 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.159809  3697 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:33.129962 28198 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:33.133078  3711 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:33.097626  3640 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:33.161090 28157 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 26.
I20260709 13:14:33.057530  3705 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:33.161623  3711 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:33.162364  3706 log.cc:927] Time spent T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:14:33.162091  3705 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.164474  3705 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:33.164633  3711 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:33.165119  3705 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:33.165692  3705 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.168037  3705 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:33.151829  3538 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.169544 28232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3814) took 2254 ms. Trace:
I20260709 13:14:33.169986 28232 rpcz_store.cc:276] 0709 13:14:30.915543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.964800 (+1049257us) service_pool.cc:224] Handling call
0709 13:14:31.965094 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.965302 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:14:32.366775 (+401473us) log.cc:844] Serialized 14 byte log entry
0709 13:14:33.169130 (+802355us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:33.169185 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:33.169237 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.169323 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:33.169454 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72650,"prepare.queue_time_us":853649,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"thread_start_us":1126865,"threads_started":2}
I20260709 13:14:33.171721  3688 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:33.171979  3703 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:33.172927  3533 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:33.173344 28229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3815) took 2257 ms. Trace:
I20260709 13:14:33.173641  3533 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.173827 28229 rpcz_store.cc:276] 0709 13:14:30.915847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.967617 (+1051770us) service_pool.cc:224] Handling call
0709 13:14:31.967872 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.968044 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:32.729061 (+761017us) log.cc:844] Serialized 14 byte log entry
0709 13:14:33.172969 (+443908us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:33.173025 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:33.173076 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.173148 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:33.173276 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73175,"prepare.queue_time_us":859665,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"thread_start_us":1104924,"threads_started":2}
W20260709 13:14:33.171782 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.176718  3510 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:33.176839  3703 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.177441  3510 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.179883  3510 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:33.179949  3703 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:33.172246  3688 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.180979  3688 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:33.079381  3489 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
W20260709 13:14:33.184130 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 126048us
I20260709 13:14:33.104990  3627 consensus_queue.cc:1059] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:14:33.184932  3715 consensus_queue.cc:1048] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:33.186383  3672 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:32.681867 27975 rpcz_store.cc:269] 0709 13:14:15.481158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.527932 (+ 46774us) service_pool.cc:224] Handling call
0709 13:14:32.464171 (+16936239us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:26.140576 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:26.151202 (+ 10626us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:26.151254 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:14:26.151293 (+    39us) tablet.cc:662] Decoding operations
0709 13:14:26.532764 (+381471us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:26.532849 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:26.532882 (+    33us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:14:26.534742 (+  1860us) tablet.cc:701] Row locks acquired
0709 13:14:26.534823 (+    81us) write_op.cc:260] PREPARE: finished
0709 13:14:26.535028 (+   205us) write_op.cc:270] Start()
0709 13:14:26.535195 (+   167us) write_op.cc:276] Timestamp: P: 1783602866534962 usec, L: 0
0709 13:14:26.535219 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:14:26.583017 (+ 47798us) log.cc:844] Serialized 8957 byte log entry
0709 13:14:27.315598 (+732581us) op_driver.cc:464] REPLICATION: finished
0709 13:14:27.470943 (+155345us) write_op.cc:301] APPLY: starting
0709 13:14:27.476666 (+  5723us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:28.530509 (+1053843us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:28.530706 (+   197us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:28.776115 (+245409us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67432a 00007f59ce5f9f14 00007f59ce5f9ec3 00007f59ce5f9e38 00007f59ce5f9df3 00007f59ce5f90dc 00007f59ce622e16 00007f59ce60acbb 00007f59ce5f539d 00007f59ce5f8d75
0709 13:14:29.198585 (+422470us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:29.199050 (+   465us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:14:29.199097 (+    47us) write_op.cc:312] APPLY: finished
0709 13:14:29.199637 (+   540us) log.cc:844] Serialized 196 byte log entry
0709 13:14:29.328628 (+128991us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:29.509921 (+181293us) write_op.cc:454] Released schema lock
0709 13:14:29.602725 (+ 92804us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:29.668079 (+ 65354us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:29.670901 (+  2822us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":243,"mutex_wait_us":1173003,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":102359,"apply.run_cpu_time_us":60742,"apply.run_wall_time_us":2191891,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96395,"num_ops":22,"prepare.queue_time_us":203632,"prepare.run_cpu_time_us":34628,"prepare.run_wall_time_us":597249,"raft.queue_time_us":290635,"raft.run_cpu_time_us":23224,"raft.run_wall_time_us":1525023,"replication_time_us":734568,"spinlock_wait_cycles":9854720,"thread_start_us":392091,"threads_started":5,"wal-append.queue_time_us":421303,"wal-append.run_cpu_time_us":46403,"wal-append.run_wall_time_us":2538311}]]}
I20260709 13:14:33.007638  3689 consensus_queue.cc:237] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.180511  3510 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:33.216995 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 197041us
I20260709 13:14:33.220171 28125 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:33.183434 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 53165.3us
I20260709 13:14:33.183363  3575 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:33.176548  3533 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:33.225010 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1512) took 3034 ms. Trace:
I20260709 13:14:33.198418  3715 consensus_queue.cc:1243] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer 0faaace2b36c4bcebf2fa6835235e92b log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:14:33.221068 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 164059us
I20260709 13:14:33.191771 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:33.233590 27982 catalog_manager.cc:5697] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:33.240475 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.241098 28235 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 25. Current term is 26. Ops: []
I20260709 13:14:33.242460 28228 raft_consensus.cc:1275] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:14:33.235296 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:33.222641 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:33.228906 28153 rpcz_store.cc:276] 0709 13:14:30.190267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.986071 (+1795804us) service_pool.cc:224] Handling call
0709 13:14:31.999474 (+ 13403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.999687 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 13:14:32.814788 (+815101us) log.cc:844] Serialized 14 byte log entry
0709 13:14:33.224604 (+409816us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:33.224663 (+    59us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:14:33.224714 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.224811 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:33.224944 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1813,"prepare.queue_time_us":1138847,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":4941,"thread_start_us":1206973,"threads_started":2}
I20260709 13:14:33.160431  3712 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:33.255640 28232 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:33.244493 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:33.259892 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:33.265174 28228 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:14:33.225793  3452 raft_consensus.cc:3055] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:33.265867  3452 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:33.266301 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.267021 28235 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 26.
I20260709 13:14:33.267280  3452 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.268075 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.276422 28228 raft_consensus.cc:2410] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 13:14:33.268813 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:33.277359 28758 raft_consensus.cc:2468] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:33.269510 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:33.233629  3710 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:33.227727  3672 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:33.275687 27982 catalog_manager.cc:5697] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 16 to 21, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 21 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:33.270349 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:33.286243 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:33.252007  3533 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:33.286932 28757 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:33.287632 28755 raft_consensus.cc:2393] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:33.290133 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.290607 28232 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:14:33.252411  3640 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:33.224581  3510 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.296793 28234 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:33.298951 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:33.307849 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:33.308219 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.310864 28124 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:33.308537 28755 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 26.
I20260709 13:14:33.277845  3703 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:14:33.322809 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:33.312633 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.316313 28232 raft_consensus.cc:2410] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 22 because replica has last-logged OpId of term: 20 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 17.
I20260709 13:14:33.304105  3710 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:33.330349 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.330976 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.331323 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.331736 28227 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:14:33.275277  3716 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:33.334834 27982 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:33.336509 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.292603  3452 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:33.340022 28232 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:33.295179  3533 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:33.342152 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:33.278844  3712 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.059s
I20260709 13:14:33.348580 28232 raft_consensus.cc:2376] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 70996712c3c942468a6006602de6477f in term 21. Re-sending same reply.
I20260709 13:14:33.372700 28757 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 23.
I20260709 13:14:33.373566 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.332964  3627 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:14:33.376256 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.382936 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.383739 28227 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 16.
I20260709 13:14:33.339035  3716 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.384614 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.385372 28227 raft_consensus.cc:2468] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:33.341814  3710 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.386503 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.387689 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.387794  3710 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:33.368789  3533 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:33.395031  3510 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:33.399416 28227 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:33.400224  3176 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:33.400554 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.401907 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.417385 28232 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:33.385169  3716 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:33.419459 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.378954 27982 catalog_manager.cc:5697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 13 to 17, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 17 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:33.388484  3710 raft_consensus.cc:2804] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:14:33.422776 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 111176us
I20260709 13:14:33.423185 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:33.380497  3627 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:33.461774 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.462255 28050 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:33.463454 28127 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.466311 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.416188  3510 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:33.425820  3710 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:33.462671 28232 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:33.502223 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.477276  3223 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:33.486670  3080 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.463994  3627 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:14:33.490751 27982 catalog_manager.cc:5697] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:33.524282 27982 catalog_manager.cc:5697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:33.492942  3255 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:33.493177  3700 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.493840  3299 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:33.493978  3489 raft_consensus.cc:3055] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:33.496560  3719 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:33.518977 28235 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 26.
I20260709 13:14:33.415307  3723 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 13:14:33.533713  3489 raft_consensus.cc:740] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:33.539670  3717 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.525694 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.540453  3727 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:14:33.541402 28232 raft_consensus.cc:2410] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 13:14:33.541957  3739 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:33.542372 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.526953 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.543315  3723 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:33.543897  3723 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:33.537006  3719 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:33.544904  3723 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.545796  3719 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.547081  3719 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:33.536326 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.500638  3718 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.501660  3710 raft_consensus.cc:3060] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:33.549556  3718 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.536161  3700 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.551944  3718 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:33.553332 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:33.554627 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.555266 28232 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 24.
W20260709 13:14:33.555573  3713 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.175s	user 0.000s	sys 0.003s
W20260709 13:14:33.558069 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.509853  3510 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.559417  3710 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.556339 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:33.540347  3717 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.563087 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.585594 28156 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:33.564940 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.565128 28230 raft_consensus.cc:2410] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:14:33.587080 28153 raft_consensus.cc:2410] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:14:33.568976 27982 catalog_manager.cc:5697] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:33.575719 28229 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:33.575697 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:33.593768 28235 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
W20260709 13:14:33.578115 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 237029us
I20260709 13:14:33.575129 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:33.619485 28728 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:33.620045 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:33.560695  3510 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:33.617620 28049 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:33.576658  3733 log.cc:927] Time spent T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:14:33.568470  3725 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:33.623327 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48287.2us
I20260709 13:14:33.642279 28127 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:14:33.574085  3732 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.081s	user 0.001s	sys 0.001s
I20260709 13:14:33.527882  3223 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.552582  3700 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:33.629289 28728 leader_election.cc:336] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:33.652029 28728 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:33.652985 28728 leader_election.cc:336] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:33.654912 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:33.655915 28124 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:33.656965  3726 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:14:33.639688 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.561497  3080 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:33.647142 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:33.657027  3685 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:33.658651 28762 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:14:33.616180 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 351065us
I20260709 13:14:33.661074 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:33.648192 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 187119us
I20260709 13:14:33.666138 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:33.666141 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:33.624771  3640 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:33.669678 28760 raft_consensus.cc:1275] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:14:33.670970 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:33.675379 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:33.659341  3740 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:14:33.683416  3729 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:14:33.684353  3223 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:33.643065  3703 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:33.685676  3703 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.689466 28759 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:14:33.657505  3726 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:33.694227 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:33.660873  3692 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:33.540505  3489 consensus_queue.cc:260] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.622815  3533 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.695478  3727 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:14:33.699810  3724 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:14:33.700294 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.701732  3717 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:33.659979  3725 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.727510 28757 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:14:33.728433 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:33.728776  3725 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:33.719069  3533 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:33.696085  3728 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:33.702858  3627 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.705137  3710 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:33.733172  3489 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:33.733946 28758 raft_consensus.cc:1240] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 28. Current term is 29. Ops: []
I20260709 13:14:33.692629  3685 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:33.673408  3719 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:33.707659  3726 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:14:33.716171 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51753.1us
I20260709 13:14:33.713219  3692 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:33.720371 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:33.672921  3718 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:33.722173  3640 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:33.704824  3080 consensus_queue.cc:1048] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:33.736346  3627 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:33.706663  3744 consensus_queue.cc:1059] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 13:14:33.716806  3727 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:33.748330 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:33.754637 28760 raft_consensus.cc:2393] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:33.758533  3689 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:33.715464 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:33.732440  3728 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:33.704907  3527 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:14:33.719601  3724 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:33.806667 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.807374 28160 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:33.759624  3692 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:33.781471 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.763505  3640 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:33.785467 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:33.786677 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 71306.4us
I20260709 13:14:33.717703  3716 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:33.787207 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:33.787382 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.789038 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.901316 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.902235 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:33.868419 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.902868 28158 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 24.
I20260709 13:14:33.791816 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:33.901330 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 85654.6us
I20260709 13:14:33.792840 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:33.797732  3730 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.241s	user 0.001s	sys 0.004s
I20260709 13:14:33.798355 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:33.905339 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 443158us
I20260709 13:14:33.785254 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.906033 28155 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 17.
I20260709 13:14:33.906255 28159 raft_consensus.cc:2410] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:14:33.901393 28152 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:33.781929  3727 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:33.893929 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.776985  3452 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:33.759804  3223 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 13:14:33.804823  3275 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:33.813300 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.928937 28156 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:33.907650 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.782109  3718 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:33.790297  3717 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:33.895357  3640 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:33.911005 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.750737  3725 consensus_queue.cc:1059] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.036s
W20260709 13:14:33.913269 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 126130us
I20260709 13:14:33.897173 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:33.927126 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:33.933799 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.936887 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.937251 28232 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:33.938653 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 187888us
W20260709 13:14:33.908463 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.812057  3724 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:33.764499  3750 consensus_queue.cc:1048] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.178s
I20260709 13:14:33.947144 28730 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:33.976142 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 71676.8us
I20260709 13:14:33.978255 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:33.978924 28159 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
W20260709 13:14:33.983340 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 86273us
W20260709 13:14:33.993436 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 46370.8us
W20260709 13:14:33.993639 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49573.6us
I20260709 13:14:33.994272 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:33.935456  3731 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:33.928635  3275 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.992322 28127 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:33.793814  3689 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:33.943562  3080 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.007009 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:34.004343 28761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 1356) took 1592 ms. Trace:
I20260709 13:14:34.020640 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:34.011104 28761 rpcz_store.cc:276] 0709 13:14:32.412104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.412297 (+   193us) service_pool.cc:224] Handling call
0709 13:14:32.412644 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:33.005895 (+593251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:33.005932 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:33.018289 (+ 12357us) log.cc:844] Serialized 26 byte log entry
0709 13:14:33.288651 (+270362us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:14:33.288694 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.496003 (+207309us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b5400686fd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:33.496062 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:33.797090 (+301028us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:34.004003 (+206913us) raft_consensus.cc:1685] finished
0709 13:14:34.004041 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:34.004254 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29447,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":86,"spinlock_wait_cycles":916710144,"thread_start_us":438792,"threads_started":2,"wal-append.queue_time_us":465359}
W20260709 13:14:34.026310 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 81069.3us
I20260709 13:14:34.030977 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:33.909353  3739 consensus_queue.cc:1059] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.116s
I20260709 13:14:34.063367 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:34.064574 28153 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
W20260709 13:14:34.065842 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 71374.5us
I20260709 13:14:33.942937  3718 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.009403  3275 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:34.002315  3627 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:34.003429  3691 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:34.009107  3740 raft_consensus.cc:2804] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:34.015482  3697 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:34.075691 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 98566.4us
W20260709 13:14:34.036374  3688 consensus_meta.cc:287] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.855s	user 0.015s	sys 0.000s
I20260709 13:14:33.927657  3176 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.137s
I20260709 13:14:34.079766  3688 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:34.080987  3758 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:14:34.082437  3688 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:34.004056  3728 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.083523 28227 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 23: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:34.084697  3752 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:14:34.039973  3080 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.086853  3080 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:33.935317  3452 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.088380  3770 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:34.089797  3698 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:34.090492  3452 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:34.088397 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57570.8us
I20260709 13:14:33.906136  3685 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.092569 28159 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:34.040337  3720 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:34.079450  3697 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.003675  3711 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.072s
W20260709 13:14:34.132349 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.092840  3691 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.101266  3720 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:34.142386 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 144154us
I20260709 13:14:34.144558 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:34.101814  3740 raft_consensus.cc:697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:34.146032 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:34.095814  3757 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 13:14:34.143605  3691 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:34.145571  3720 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:34.148995  3725 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:14:34.166778  3754 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 13:14:34.168409  3749 log.cc:927] Time spent T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
W20260709 13:14:34.169559  3763 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:14:34.174052  3725 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:34.174799  3725 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:34.174844  3716 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:34.149955  3627 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:14:34.178440  3740 consensus_queue.cc:237] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.18, Last appended by leader: 18, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.188434  3718 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.094931 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:34.092537 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:34.233235 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 224125us
W20260709 13:14:34.231105 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 138440us
I20260709 13:14:34.251111  3757 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:34.252266  3757 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:34.253301  3774 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:34.253114  3757 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:34.258143  3692 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:34.256332 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 161333us
W20260709 13:14:34.272692 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 483852us
W20260709 13:14:34.276417  3726 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.514s	user 0.015s	sys 0.012s
I20260709 13:14:34.279697 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:34.279839  3757 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:34.275592 28728 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:34.309238  3715 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:34.310397  3715 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:34.311089 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 300646us
I20260709 13:14:34.313437 28160 raft_consensus.cc:2393] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 17: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:34.182134  3627 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:34.317727  3627 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:14:34.312489 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:34.327158  3756 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.245s	user 0.003s	sys 0.000s
W20260709 13:14:34.322912 28728 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:34.331543 28728 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:34.332859 28728 leader_election.cc:336] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:34.336059 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:34.337968 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:34.338464  3255 raft_consensus.cc:3055] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:14:34.338778 28050 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:34.339080  3255 raft_consensus.cc:740] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:34.338027  3717 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:14:34.341249  3705 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:34.342257  3255 consensus_queue.cc:260] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.345325  3255 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:34.336539 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:34.348687  3726 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.357832  3715 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:34.367828  3747 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:34.368450  3747 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.368503  3744 raft_consensus.cc:3055] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 LEADER]: Stepping down as leader of term 28
I20260709 13:14:34.369078  3744 raft_consensus.cc:740] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:34.373517  3762 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:34.369043  3747 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:34.376474  3744 consensus_queue.cc:260] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.377565  3757 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.341264  3728 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:34.379354  3744 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:14:34.360692  3726 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:34.387722  3697 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:34.393883 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.394456  3710 raft_consensus.cc:3055] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:34.394959  3710 raft_consensus.cc:740] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:34.393087 28762 consensus_queue.cc:237] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.409224 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:34.396397  3710 consensus_queue.cc:260] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.417671  3710 raft_consensus.cc:3060] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:34.425433 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:34.431917  3757 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:34.437355  3689 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:34.438159 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.442718 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:34.445442 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 107625us
W20260709 13:14:34.446659 28728 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:34.447434 28728 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:34.452250  3762 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:34.462675  3726 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:34.465906 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:34.467324 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:34.468016 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:34.469236 28755 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:14:34.470296 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:34.467370 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:34.471128 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:34.467357 28756 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 13:14:34.473838  3640 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.475081  3715 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:14:34.475905 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 51965.6us
W20260709 13:14:34.481180 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 82104.3us
I20260709 13:14:34.479935 28757 raft_consensus.cc:2376] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23. Re-sending same reply.
I20260709 13:14:34.484014 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:34.480274  3762 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:34.429667  3692 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:34.500563  3740 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:34.500921 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:34.501884 28232 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:34.502454 28157 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 17: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:14:34.502485 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55106.4us
W20260709 13:14:34.503855 28728 leader_election.cc:336] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:34.504457 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:34.504499 28763 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:34.505551 28156 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:34.512862 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:34.515182 28052 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:34.515439 28153 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:34.519182  3715 consensus_queue.cc:1059] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:34.520921  3701 raft_consensus.cc:3055] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:34.521430  3701 raft_consensus.cc:740] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:34.521760 28728 leader_election.cc:336] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:34.523404 28728 leader_election.cc:336] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:34.522821  3701 consensus_queue.cc:260] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.524307  3740 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:34.525019  3772 log.cc:927] Time spent T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.329s	user 0.000s	sys 0.002s
I20260709 13:14:34.517720  3275 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.526889 28161 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 19: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:34.529335  3726 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.541167 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:34.552778 28232 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
W20260709 13:14:34.547223  3489 consensus_meta.cc:287] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.813s	user 0.009s	sys 0.009s
I20260709 13:14:34.548322  3701 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:34.548923  3705 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:34.539551  3753 raft_consensus.cc:2804] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:34.557837  3753 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.558328  3753 raft_consensus.cc:3060] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:34.558697  3724 consensus_meta.cc:287] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.009s
I20260709 13:14:34.533933  3747 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.566171  3275 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.567555  3747 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.567870  3720 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.559239  3724 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.573606 28158 raft_consensus.cc:2393] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 22: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:34.572423  3753 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.570607  3720 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.581609  3753 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:34.548331  3728 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:34.586598 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:34.582605  3724 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.611991 28158 raft_consensus.cc:1240] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:34.618181 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 103107us
I20260709 13:14:34.621039 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:34.622035 28161 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:14:34.625984  3701 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:34.626598  3701 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.627048  3701 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:34.632087  3640 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:34.642845  3726 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:34.643415  3762 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.658200 28052 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:34.662912  3727 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.011s
I20260709 13:14:34.676440  3728 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:34.677030  3728 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.677481  3728 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:34.681001 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:34.676468  3727 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.682278  3715 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:34.682865  3715 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:34.683280 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:34.683372  3715 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:34.684527  3688 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:34.685111  3688 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:34.685530  3688 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:34.688899 28159 consensus_meta.cc:287] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.565s	user 0.000s	sys 0.019s
I20260709 13:14:34.689451 28159 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:34.691010  3762 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.695495  3715 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.701748 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:34.648706  3725 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.703061  3783 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:34.675629 28052 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:34.704712 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:34.708158 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.683862  3727 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:34.704530  3725 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.710279 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:34.703513 28728 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:34.705078  3784 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.729789 28161 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 29.
W20260709 13:14:34.736128 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 51561.3us
I20260709 13:14:34.745046  3703 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:34.745656  3703 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.746109  3703 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:34.746433  3715 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:34.744505  3710 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:34.748951 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73425.4us
I20260709 13:14:34.750387  3710 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:34.751524 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:34.752688  3710 raft_consensus.cc:2804] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:34.753068  3710 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:34.753317  3710 raft_consensus.cc:3060] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:34.743696 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:34.762215 28759 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 26. Current term is 27. Ops: []
I20260709 13:14:34.770047 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:34.774250  3725 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:34.774979  3725 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:34.775456  3725 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:34.763587  3727 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:34.774297 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:34.777428 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 74033.9us
W20260709 13:14:34.781947 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.783993 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:34.793378  3762 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:34.793923  3762 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.794341  3762 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:34.790979 28127 leader_election.cc:336] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.633s (SENT)
I20260709 13:14:34.798954 28198 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:34.802325 28235 raft_consensus.cc:1240] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 13:14:34.815771  3627 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.821930 28761 raft_consensus.cc:2410] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 13:14:34.823318 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:34.769032  3784 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.825627 28728 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:34.826166  3784 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.829423  3223 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:34.829876  3223 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:34.830224  3223 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:34.830355  3716 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:34.831056  3716 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.833232 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:34.833925 28228 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:34.834741 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:34.835829 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51321.8us
I20260709 13:14:34.837504  3726 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:34.831390  3775 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:14:34.842000 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 58125.3us
I20260709 13:14:34.839121  3223 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.788681  3715 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
W20260709 13:14:34.851511 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 56034.3us
I20260709 13:14:34.851042 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:34.839692  3753 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:14:34.853449 28759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56420 (request call id 1730) took 1073 ms. Trace:
W20260709 13:14:34.868110 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.864776 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:34.875200  3533 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:34.875921  3533 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:34.876281  3691 consensus_meta.cc:287] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.638s	user 0.005s	sys 0.009s
I20260709 13:14:34.876951  3691 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.848502  3688 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.876379  3533 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:34.888049 28156 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:14:34.890262 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:34.890691  3691 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.891420 28227 raft_consensus.cc:2393] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 17: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:34.892331  3701 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:34.900429 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.865535 28759 rpcz_store.cc:276] 0709 13:14:33.777305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.778688 (+  1383us) service_pool.cc:224] Handling call
0709 13:14:33.779025 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:34.762802 (+983777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:34.845167 (+ 82365us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b5400439ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:34.850182 (+  5015us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2134398336}
I20260709 13:14:34.867290  3753 consensus_queue.cc:1243] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:34.882591  3716 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:34.896502  3725 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.899704  3703 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:34.921622  3791 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:14:34.906749  3719 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:14:34.924115  3716 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:34.913957  3728 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:34.944134 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:34.920938  3784 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:14:34.946792  3627 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:34.965884 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:34.968789 28232 raft_consensus.cc:2468] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:34.970788 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:34.982100 28728 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:34.985001 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:34.988765 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 87804.9us
I20260709 13:14:34.975474  3223 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:34.993464 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:34.997115 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:35.002241 28227 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:14:35.004802 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.008826  3255 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.009572  3255 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:34.997784  3716 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:35.011528  3716 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.011945  3716 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:35.012020  3255 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:34.998973 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:34.991703  3688 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:35.013963 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 215191us
I20260709 13:14:35.018664 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:34.996003  3701 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:35.030087  3703 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:35.032364 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:35.033200 28156 raft_consensus.cc:2368] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18.
I20260709 13:14:35.031226 28199 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:35.039249 28232 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:35.042928 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:35.043629 28156 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:35.046093  3783 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:35.046518  3783 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:35.036523  3775 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:14:35.051254  3701 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:35.051147  3710 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.053831  3710 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:35.046397  3533 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:35.056689  3533 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:35.056988  3775 raft_consensus.cc:697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 29 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:35.057595 28235 raft_consensus.cc:2368] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20.
I20260709 13:14:35.058065  3775 consensus_queue.cc:237] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.060328  3223 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.060943  3223 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.047482  3783 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.064118  3223 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:35.065163 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:35.066098  3533 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:35.066624  3533 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:35.067610  3533 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:35.067987  3716 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:35.078928 28226 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:35.079679  3703 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:35.089529 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 49733us
I20260709 13:14:35.093997 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:35.097293 28198 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:35.099406 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:35.050777  3728 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:35.104681 28755 raft_consensus.cc:1275] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:14:35.089076  3725 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:35.105604 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:35.109849  3703 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.108840 28762 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 23.
I20260709 13:14:35.112383  3703 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:35.113343  3712 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:35.116797 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:35.117012 28153 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 13:14:35.117533 28157 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:14:35.117427 28158 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:35.117352  3795 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:35.121923 28156 raft_consensus.cc:1240] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:35.123020 28161 raft_consensus.cc:1240] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 26. Current term is 27. Ops: []
I20260709 13:14:35.098564  3688 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.029s
I20260709 13:14:35.125190 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:35.125941 28762 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 27: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:35.119836 28153 pending_rounds.cc:85] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 16
I20260709 13:14:35.126814 28153 pending_rounds.cc:107] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 13:14:35.127097 28153 raft_consensus.cc:2889] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:35.129248  3688 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:35.129830  3688 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:35.130962  3688 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.132517  3688 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:35.133155  3795 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:35.133634 28155 raft_consensus.cc:1240] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:35.135377 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:35.151351  3783 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:35.152012 28760 raft_consensus.cc:1240] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:35.120199  3712 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:35.154909  3712 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.156360  3712 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:35.169056 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.159300 28760 raft_consensus.cc:1240] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 26. Current term is 27. Ops: []
I20260709 13:14:35.118160  3727 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:35.156917 28125 leader_election.cc:336] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:35.176774 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 79584.5us
I20260709 13:14:35.143577  3762 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.182668 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:35.191594 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.210578 28157 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:14:35.225952 28152 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:14:35.226369 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:35.227689 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:35.228796 28226 raft_consensus.cc:2368] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19.
I20260709 13:14:35.229399  3717 consensus_queue.cc:1059] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 13:14:35.229605  3753 consensus_queue.cc:1059] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:14:35.230532 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:35.231532  3753 raft_consensus.cc:3055] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:35.231952  3753 raft_consensus.cc:740] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:35.232980  3753 consensus_queue.cc:260] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.238335 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:35.239383  3753 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:35.239724 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:35.241004 28758 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:35.242563 28758 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
W20260709 13:14:35.246697 28730 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:35.253575 28727 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.226058 28755 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:14:35.258841 28155 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 13:14:35.260258 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:35.261240 28153 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:35.261130 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:35.227708  3710 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:14:35.265260 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:35.265741  3705 raft_consensus.cc:3055] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:14:35.266325  3705 raft_consensus.cc:740] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:35.267707  3705 consensus_queue.cc:260] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.221526  3691 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:14:35.269188 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:35.270656 28230 raft_consensus.cc:1240] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:14:35.272157 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:35.309938 28234 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:35.272805 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:35.274351 28231 pending_rounds.cc:85] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
I20260709 13:14:35.310760 28231 pending_rounds.cc:107] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:14:35.274811 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
W20260709 13:14:35.276532 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 119894us
I20260709 13:14:35.276989  3795 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:35.278776  3716 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:35.283269  3715 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:35.260494 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:35.292192 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:35.266598 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:35.214774  3728 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:35.215446 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:35.305562 28124 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:35.306200 28755 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:14:35.306265 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:35.307687 28152 raft_consensus.cc:1240] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 24. Current term is 25. Ops: []
I20260709 13:14:35.311115 28231 raft_consensus.cc:2889] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:35.330518 28232 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:35.330837 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:35.334300 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:35.301668  3725 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.342852 28226 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:35.341435 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:35.344798 28728 leader_election.cc:336] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:35.346287 28125 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:35.351063 28161 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 27: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:35.346691 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:35.357843 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:35.358733 28154 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:35.359793  3728 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.361780 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:35.355026 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:35.364854  3715 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:35.339668  3701 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:35.389009  3691 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:35.354070  3725 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.389613  3691 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.389812  3715 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:35.390131  3691 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:35.391534  3725 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:35.392134 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 224224us
W20260709 13:14:35.392983 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48252.7us
I20260709 13:14:35.393869 28159 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
I20260709 13:14:35.361521 28757 raft_consensus.cc:2393] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 21: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:35.361708  3747 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:35.317132  3692 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:35.411587 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:35.413251 28157 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
W20260709 13:14:35.393827 28728 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:35.416460 28728 leader_election.cc:336] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:35.417546 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.422564 28234 raft_consensus.cc:2410] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
W20260709 13:14:35.424940 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 200180us
W20260709 13:14:35.427150 28728 leader_election.cc:336] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:35.435353 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:35.341562  3753 consensus_queue.cc:1059] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.086s
I20260709 13:14:35.449587 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:35.448133 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 121550us
I20260709 13:14:35.440212 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:35.453058 28226 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:35.463546 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:35.445742  3744 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:35.445739  3705 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:35.470150 28233 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:35.473395 28154 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:35.423470  3692 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:35.476250 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 170828us
I20260709 13:14:35.446341  3747 consensus_queue.cc:1059] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 13:14:35.474002  3710 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:35.489538 28233 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:35.489981 28232 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:35.491734 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:35.496891 28152 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:35.472067  3744 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:35.492766 28234 raft_consensus.cc:1275] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:14:35.507938 28050 leader_election.cc:400] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18.
I20260709 13:14:35.508359 28050 leader_election.cc:403] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:35.489225  3753 raft_consensus.cc:3055] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:14:35.512017 28234 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
W20260709 13:14:35.521221  3801 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:14:35.522790  3692 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:35.528968 28226 raft_consensus.cc:2468] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:35.523692 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:35.501770  3744 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:35.508876 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 97367.8us
I20260709 13:14:35.498484  3728 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:35.531940 28125 consensus_peers.cc:597] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:35.542277 28125 consensus_peers.cc:597] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:35.545792 28052 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:35.547268  3802 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:14:35.548655 28229 raft_consensus.cc:1275] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
I20260709 13:14:35.524938  3753 raft_consensus.cc:740] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:35.553537 28125 consensus_peers.cc:597] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:35.555256 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 68510.4us
I20260709 13:14:35.556294 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:35.535641  3747 raft_consensus.cc:3055] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:35.553838 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:35.538853  3727 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:35.552269 28730 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:35.563436 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 106015us
W20260709 13:14:35.565959 28125 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:35.566982 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:14:35.569154 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:35.569262 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 131150us
I20260709 13:14:35.571384  3223 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:14:35.573125 28229 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 16. Current term is 17. Ops: []
W20260709 13:14:35.585403 28125 leader_election.cc:336] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:35.585960 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:35.590849 28756 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:35.538712  3255 consensus_queue.cc:1048] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:14:35.600106  3717 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:14:35.605314  3803 log.cc:927] Time spent T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:14:35.604983  3783 consensus_queue.cc:1048] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:14:35.606350 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.549391  3750 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:14:35.609148  3750 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:35.609683  3750 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:35.610873  3750 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.611544 28728 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:35.612223  3750 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:35.614697  3804 log.cc:927] Time spent T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:14:35.619127  3813 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:14:35.612516 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:35.627775  3762 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:35.559660  3727 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:35.628949  3727 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:35.632300 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:35.633011 28158 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 16.
W20260709 13:14:35.637606 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 91923.9us
I20260709 13:14:35.638517 28052 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:35.639467  3717 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:35.639838 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:35.640472  3717 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:35.559319  3747 raft_consensus.cc:740] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:35.643071  3717 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:35.643718  3717 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:35.643860  3747 consensus_queue.cc:260] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.644402  3717 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:35.645273  3747 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:35.653059  3753 consensus_queue.cc:260] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.654819  3753 raft_consensus.cc:3060] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:35.658344  3255 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.658962  3255 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.661772  3812 raft_consensus.cc:2804] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:35.662295  3812 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:35.662678  3812 raft_consensus.cc:3060] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:35.664986  3810 consensus_queue.cc:1048] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:14:35.569171  3725 raft_consensus.cc:2804] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:35.667357  3725 raft_consensus.cc:697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:35.665439 28760 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:35.668427  3725 consensus_queue.cc:237] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:35.600589 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 44417.7us
I20260709 13:14:35.563144  3811 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:14:35.702814 28052 leader_election.cc:400] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20.
I20260709 13:14:35.628575  3762 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:14:35.714349 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:35.715906 28729 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.717175 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:35.725116  3710 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:35.725878  3710 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:35.726462  3710 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:35.735239 28232 raft_consensus.cc:2468] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 27.
W20260709 13:14:35.733484 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 165757us
I20260709 13:14:35.624529  3691 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.741209  3692 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:14:35.750335 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 51593.8us
W20260709 13:14:35.754130 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 76461.2us
W20260709 13:14:35.761363 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 148903us
W20260709 13:14:35.765125 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 277971us
I20260709 13:14:35.752076  3255 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:35.758656  3750 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:35.793670 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:35.798293 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 212726us
W20260709 13:14:35.783747 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:35.786713 27983 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 11 to 16, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 16 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:35.809629 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 69302.5us
I20260709 13:14:35.805585 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:35.811731 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 55155.5us
W20260709 13:14:35.814972 28125 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:35.816079 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 61101.8us
W20260709 13:14:35.821239 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 78770.9us
W20260709 13:14:35.853083 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 69359.5us
I20260709 13:14:35.865492 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:35.880383 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:35.826354 28125 leader_election.cc:336] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:35.866067 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2186033780001253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 13:14:35.904894 28199 connection.cc:256] RPC call timeout handler was delayed by -1.1718033780000496s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
W20260709 13:14:35.905547  3799 log.cc:927] Time spent T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.485s	user 0.003s	sys 0.006s
I20260709 13:14:35.907066 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:35.907688 28235 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:35.914395  3750 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:35.914934  3750 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:35.917181  3816 log.cc:927] Time spent T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:14:35.924275  3811 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757) with higher term. Message: Invalid argument: T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 17. Current term is 18.
W20260709 13:14:35.925565 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 60211.3us
I20260709 13:14:35.932018 28125 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:35.821919  3725 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.857435  3712 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 13:14:35.940994 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 127752us
I20260709 13:14:35.941486 28235 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
W20260709 13:14:35.941705 28728 leader_election.cc:336] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:35.934942 28127 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.698s (SENT) [suppressed 96 similar messages]
I20260709 13:14:35.943722 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:35.946046 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 138760us
I20260709 13:14:35.920162  3717 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:35.947662 28052 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:35.936345  3783 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:14:35.950531 28728 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:35.886314  3822 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.096s
W20260709 13:14:35.960278 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 142627us
I20260709 13:14:35.972532 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:35.924719  3744 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:35.931699  3812 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:35.930487  3692 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 13:14:35.954182  3715 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.955768  3783 raft_consensus.cc:493] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.984874 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:35.992982 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:35.991796 28730 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:35.977574  3691 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:35.951912  3811 consensus_queue.cc:1059] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 13:14:36.001129 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 174812us
I20260709 13:14:35.948899  3725 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:35.992575  3783 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:36.003877 28125 consensus_peers.cc:597] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:35.988644  3762 raft_consensus.cc:697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:36.008759 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56296.3us
W20260709 13:14:36.009516 28728 leader_election.cc:336] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:36.012673 28125 leader_election.cc:336] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:36.013213 28125 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:36.016656 28728 leader_election.cc:336] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:36.018393 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 76691.1us
I20260709 13:14:36.021574 28052 leader_election.cc:400] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19.
I20260709 13:14:36.022010 28052 leader_election.cc:403] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:14:36.018971 28730 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:36.010967  3712 raft_consensus.cc:2804] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:14:36.035727 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:36.030192 28227 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:35.997781  3715 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.013329  3811 consensus_queue.cc:1243] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:14:36.036806 28226 raft_consensus.cc:2410] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 24 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 17.
I20260709 13:14:36.037798  3727 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.050973 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:36.053335 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.044893  3705 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:36.062376 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:36.064889 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 47011.3us
W20260709 13:14:36.070746 28728 leader_election.cc:336] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:36.071481 28728 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:36.071791 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:36.074760 28728 leader_election.cc:336] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:36.075251 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:36.075829 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.077144 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:36.085289 28728 leader_election.cc:336] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:36.087127 28728 leader_election.cc:336] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:36.088716  3717 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:36.051612  3712 raft_consensus.cc:697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 27 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:36.089339 27983 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
W20260709 13:14:36.098124 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 61487.8us
I20260709 13:14:36.100663 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:36.102437  3740 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:36.069437  3255 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:36.102453  3728 raft_consensus.cc:3055] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:36.065177  3725 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:36.066184  3715 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:36.118171 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:36.117484 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 156286us
I20260709 13:14:36.063433  3812 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:36.123072 28226 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:14:36.124022 28049 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.122397 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:36.113986  3823 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.101s	user 0.001s	sys 0.001s
I20260709 13:14:36.134105 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:36.116232  3821 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:14:36.113709  3815 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.117170  3728 raft_consensus.cc:740] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:36.148346 28226 raft_consensus.cc:1240] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:36.126842  3820 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:36.116067  3839 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:14:36.141923 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 161076us
I20260709 13:14:36.129969  3825 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:36.150676 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:36.128206  3835 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:36.155597 28124 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:36.141469  3812 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:36.161413 28155 raft_consensus.cc:1275] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 13:14:36.130913  3811 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:14:36.145730  3744 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:36.119303  3762 consensus_queue.cc:237] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.164620  3837 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:36.185045 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:14:36.195326 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 188020us
W20260709 13:14:36.195402 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.168326  3839 raft_consensus.cc:697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:36.199496 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:36.148887  3815 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:36.205207 28125 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:36.201529 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 57387.9us
I20260709 13:14:36.205606 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:36.209594 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 68323.2us
I20260709 13:14:36.260284 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.261008 28226 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:36.264036 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 58522.7us
I20260709 13:14:36.196314  3710 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.270437  3845 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:36.270653  3851 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:36.271055  3836 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:36.271246 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 75125.7us
I20260709 13:14:36.179033  3712 consensus_queue.cc:237] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:36.260699  3828 log.cc:927] Time spent T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:14:36.180603  3847 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.093s
I20260709 13:14:36.251725  3727 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:36.297199 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:36.301057 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:36.260710  3832 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:36.260560  3842 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:36.308717 28759 raft_consensus.cc:1240] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 22. Current term is 23. Ops: []
W20260709 13:14:36.308892 28125 leader_election.cc:336] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:36.260676  3838 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:36.270488  3715 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:36.319293  3848 log.cc:927] Time spent T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:14:36.319176  3747 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:36.195989  3728 consensus_queue.cc:260] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.327730  3839 consensus_queue.cc:237] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:36.335255  3753 consensus_meta.cc:287] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.680s	user 0.015s	sys 0.007s
I20260709 13:14:36.314194  3836 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.336475  3847 consensus_queue.cc:1243] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:14:36.340148  3844 log.cc:927] Time spent T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:14:36.260907  3826 consensus_queue.cc:1059] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.361s
I20260709 13:14:36.260689  3829 consensus_queue.cc:1059] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.336s
I20260709 13:14:36.309711  3845 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:14:36.343704 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.342321  3812 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
W20260709 13:14:36.350072  3852 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:14:36.352979  3826 consensus_queue.cc:1059] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:36.353579  3856 raft_consensus.cc:3055] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:14:36.354110  3856 raft_consensus.cc:740] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:36.354329  3832 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.354959  3832 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.357223  3811 raft_consensus.cc:3055] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:36.360039 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.357688  3856 consensus_queue.cc:260] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.362689  3811 raft_consensus.cc:740] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:36.365973 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 62550.2us
W20260709 13:14:36.366748 28125 consensus_peers.cc:597] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:36.366292  3811 consensus_queue.cc:260] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.367834  3811 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:36.368121 28125 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:36.369807  3815 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:36.371563  3829 raft_consensus.cc:3055] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:36.371964  3829 raft_consensus.cc:740] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:36.372785  3829 consensus_queue.cc:260] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.374153  3829 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:36.337335  3851 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:36.365372  3710 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:36.337479  3715 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.357370  3728 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:36.391283 28761 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
W20260709 13:14:36.378190  3744 consensus_queue.cc:1175] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "11f10c12c03e47dcac88b050d4c1431b" responder_term: 22 status { last_received { term: 22 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 22 index: 13 } }
I20260709 13:14:36.403460  3783 raft_consensus.cc:515] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.405347  3715 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:36.411175 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.411870 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.412184 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:36.412588 28234 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 18. (term mismatch)
I20260709 13:14:36.413350 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.413923 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:36.431823 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 63728.4us
I20260709 13:14:36.432440 28232 raft_consensus.cc:2393] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:36.414628  3836 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:36.435180  3845 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.432009 28228 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:36.448990  3832 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:36.450598 28228 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:36.416189  3851 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.433436 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.434962 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:36.431695 28234 pending_rounds.cc:85] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 17
I20260709 13:14:36.455801 28234 pending_rounds.cc:107] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 13:14:36.456102 28234 raft_consensus.cc:2889] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:36.432682 28125 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:36.457504 28125 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:36.458806 28125 leader_election.cc:336] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:36.458320 27983 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:36.459292 28125 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:36.460583 28233 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 25: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:36.362447  3750 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.461548 28229 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:36.452809 28227 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 17.
I20260709 13:14:36.455888 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.443475  3705 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875) with higher term. Message: Invalid argument: T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19.
I20260709 13:14:36.506247 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:36.530339 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:36.531889 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.465106  3810 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:36.462626 28226 raft_consensus.cc:2468] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:36.502561  3815 raft_consensus.cc:2749] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:14:36.545529 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:36.551440 28199 connection.cc:256] RPC call timeout handler was delayed by -1.3562310229999639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1773s for a response
I20260709 13:14:36.564426 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:36.569525 28728 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:36.572693 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 113909us
I20260709 13:14:36.542673  3705 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:36.580189 28199 leader_election.cc:336] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.515s (SENT)
W20260709 13:14:36.573654 28125 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:36.583770 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:36.571820  3716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.022s	sys 0.045s
I20260709 13:14:36.586977 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:36.533306  3842 consensus_queue.cc:1059] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.051s
W20260709 13:14:36.591542 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 384364us
W20260709 13:14:36.591727 28125 consensus_peers.cc:597] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:36.592928 28125 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:36.591037 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 339879us
I20260709 13:14:36.593940 28226 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
W20260709 13:14:36.597738 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:36.599917 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.602136 28228 raft_consensus.cc:1240] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: [22.16-22.16]
W20260709 13:14:36.588918  3716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.023s	sys 0.045s
I20260709 13:14:36.617389  3846 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.090s
I20260709 13:14:36.618676  3810 consensus_queue.cc:1059] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:36.619509  3810 consensus_queue.cc:1243] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:14:36.594964  3705 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:36.633730  3866 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 13:14:36.601202  3843 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:14:36.674743 28125 leader_election.cc:336] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:36.675956 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 119814us
W20260709 13:14:36.673120 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.682765 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:36.678627  3710 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:14:36.683791 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.684569 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.682447 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.689536 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.672856  3715 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:36.690312 28050 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:36.674707  3692 raft_consensus.cc:3055] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:36.685405  3857 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:36.716326  3783 leader_election.cc:290] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:36.682430  3847 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:14:36.719853  3854 log.cc:927] Time spent T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.377s	user 0.001s	sys 0.000s
I20260709 13:14:36.720548  3847 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:36.721897  3847 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.723320  3847 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:36.723650  3860 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.724188  3860 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.725703  3856 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:36.726297  3860 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:36.730472 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55823.8us
W20260709 13:14:36.700707 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.689646  3810 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.021s
W20260709 13:14:36.731501 28125 consensus_peers.cc:597] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:36.702544  3710 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:36.689760  3815 consensus_queue.cc:1059] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:14:36.703562  3692 raft_consensus.cc:740] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:36.740276  3860 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.694211  3864 consensus_queue.cc:1059] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:14:36.740474  3750 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:36.791373  3833 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.000s	sys 0.019s
I20260709 13:14:36.796550 27983 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 18 to 21, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 21 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:36.804054 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:36.790585  3710 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:36.808027  3833 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.000s	sys 0.020s
I20260709 13:14:36.833802 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:36.789700 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 94261.8us
I20260709 13:14:36.793483  3868 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:36.840673 28762 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
W20260709 13:14:36.842569 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 262393us
I20260709 13:14:36.843870 28233 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 26.
I20260709 13:14:36.837987 28730 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:36.852692 28199 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.543s (SENT)
I20260709 13:14:36.827219  3744 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:36.793969  3860 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.833295  3753 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:36.841395  3710 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.872792  3688 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:36.883839 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:36.887071 28759 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:36.884459 26959 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:36.892872 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:36.872404  3705 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:36.853713  3811 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 13:14:36.872119  3832 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:36.900466 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:36.895345  3874 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:14:36.826574  3692 consensus_queue.cc:260] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.884563  3688 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:36.910625 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 65190us
W20260709 13:14:36.897066  3873 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.211s	user 0.001s	sys 0.001s
I20260709 13:14:36.894155  3825 raft_consensus.cc:2804] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:36.889394  3864 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 13:14:36.923019 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:36.919293  3869 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:14:36.910264  3845 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:36.893257  3753 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.867692  3868 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.895015  3744 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:36.918265  3691 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:36.947543 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:36.964829  3861 log.cc:927] Time spent T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:14:36.964304  3872 log.cc:927] Time spent T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.184s	user 0.002s	sys 0.000s
I20260709 13:14:36.942670  3846 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 13:14:36.969149 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:36.941269  3825 raft_consensus.cc:697] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:36.970448 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:36.970256 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 48930.8us
I20260709 13:14:36.973397 28154 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:36.973417 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:36.948800  3875 raft_consensus.cc:3055] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:36.926822  3832 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:36.955895  3834 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:14:36.976078 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 53200.5us
I20260709 13:14:36.979285 28226 raft_consensus.cc:1275] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:14:36.943153  3810 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:36.987890 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:36.988642 28156 raft_consensus.cc:2468] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 29.
I20260709 13:14:36.991478 28154 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:36.972874  3692 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:36.972157  3845 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:36.999833 28160 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:14:37.001669 28763 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:36.978713  3875 raft_consensus.cc:740] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:36.984154  3783 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:14:36.997991  3829 consensus_meta.cc:287] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.623s	user 0.005s	sys 0.010s
I20260709 13:14:37.020179 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:37.023120 28161 raft_consensus.cc:2410] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 13:14:37.024597 28761 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:37.025942 28050 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:37.027525 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:37.027979 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:37.028873  3887 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:14:37.032835  3882 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:37.033972  3884 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:14:37.048251 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:14:37.054888 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 63200.6us
I20260709 13:14:37.056493 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:37.022804 28228 raft_consensus.cc:2955] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 8 to 13, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:37.058951 28050 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:37.057308 28160 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:37.014006  3710 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:36.996925  3762 consensus_queue.cc:1048] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.060s
I20260709 13:14:37.021667  3783 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.064881 27983 catalog_manager.cc:5697] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 11 to 19, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 19 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:37.070667 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 77649.9us
I20260709 13:14:36.994946  3810 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:37.075227 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.076072 28050 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:37.058014  3812 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:37.065171  3826 raft_consensus.cc:2804] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 13:14:37.065465  3829 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:37.064806  3845 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:37.087327 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.044778  3834 consensus_queue.cc:1048] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 13:14:37.073117  3783 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:37.090678 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:37.098026 28125 consensus_peers.cc:597] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:37.098544 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 352250us
I20260709 13:14:37.083209  3710 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:37.083457  3826 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.081861  3860 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:37.074117  3837 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:14:37.110549 26959 tablet_replica.cc:333] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:37.115901 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:37.112331 28125 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:37.117034 28125 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:37.118683 28125 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:37.092803  3829 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:37.074565  3825 consensus_queue.cc:237] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:37.079362  3875 consensus_queue.cc:260] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:37.123428  3858 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:37.141289  3688 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:37.139149  3753 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:37.142905  3842 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:37.143570 27983 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:37.145944  3810 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:37.112932  3826 raft_consensus.cc:3060] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 29 FOLLOWER]: Advancing to term 30
W20260709 13:14:37.146874  3891 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:14:37.146471  3829 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:37.148021  3849 raft_consensus.cc:3055] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:37.148530  3849 raft_consensus.cc:740] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:37.148998 26959 raft_consensus.cc:2243] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:37.149820 26959 raft_consensus.cc:2272] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:37.149483  3849 consensus_queue.cc:260] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:37.150722  3849 raft_consensus.cc:3060] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:37.151090 27982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58554 (request call id 287) took 22111 ms (client timeout 15000 ms). Trace:
I20260709 13:14:37.152966 26959 tablet_replica.cc:333] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:37.153918 26959 raft_consensus.cc:2243] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:37.154819 26959 raft_consensus.cc:2272] T 66540b865ca24721910c667ccd790b77 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:37.156227 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:37.125332  3889 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 13:14:37.126303  3880 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:14:37.128710  3895 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.108582  3710 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.156388  3875 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:37.124166  3892 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:14:37.209573 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f5f1be456de4894af76dffdad9ea6a3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:37.210320 28160 raft_consensus.cc:3060] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:37.198086 28155 raft_consensus.cc:1275] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:14:37.157730  3832 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:37.163710 28226 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 22. Current term is 23. Ops: []
W20260709 13:14:37.168826 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:37.172816 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 233937us
I20260709 13:14:37.196791 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:37.162505 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:37.212931 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:37.213219 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:37.224334 28154 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:37.224418 28756 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:37.228022 28231 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
W20260709 13:14:37.229547 28728 leader_election.cc:336] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:37.233662 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:37.236914 28154 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:37.238821 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:37.209055  3710 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:37.244463 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:37.197825  3868 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:37.243908 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:37.244025 28155 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:37.252841 28231 raft_consensus.cc:2393] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 26: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:37.221822  3842 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.258327 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:37.260960 28155 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:37.266827 28199 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.575s (SENT)
I20260709 13:14:37.266933 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:37.267396 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:37.272024 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 147122us
I20260709 13:14:37.283150 28229 raft_consensus.cc:1275] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:14:37.284554 28762 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:37.285980 27983 catalog_manager.cc:5697] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:37.287618 28763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:37.214154  3895 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:37.292790 28231 raft_consensus.cc:1275] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:14:37.297566 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:37.297992 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:37.298640 28728 leader_election.cc:336] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:37.293114 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:37.288509 27983 catalog_manager.cc:5697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:37.293939 28763 raft_consensus.cc:2393] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 22: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
W20260709 13:14:37.299481 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:37.151481 27982 rpcz_store.cc:269] 0709 13:14:15.040234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.078023 (+ 37789us) service_pool.cc:224] Handling call
0709 13:14:37.150847 (+22072824us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:34.038080 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:34.069259 (+ 31179us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:34.069312 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:14:34.069342 (+    30us) tablet.cc:662] Decoding operations
0709 13:14:34.071180 (+  1838us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:34.071264 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:34.071308 (+    44us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:14:34.093756 (+ 22448us) tablet.cc:701] Row locks acquired
0709 13:14:34.096832 (+  3076us) write_op.cc:260] PREPARE: finished
0709 13:14:34.125000 (+ 28168us) write_op.cc:270] Start()
0709 13:14:34.134713 (+  9713us) write_op.cc:276] Timestamp: P: 1783602874122123 usec, L: 0
0709 13:14:34.134757 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:14:34.181093 (+ 46336us) log.cc:844] Serialized 5573 byte log entry
0709 13:14:34.374522 (+193429us) op_driver.cc:464] REPLICATION: finished
0709 13:14:34.409674 (+ 35152us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54003c91d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67432a 00007f59ce5f9f14 00007f59ce5f9ec3 00007f59ce5f9e38 00007f59ce5f9df3 00007f59ce5f90dc 00007f59ce622e16 00007f59ce60acbb 00007f59ce5f539d 00007f59ce5f8d75
0709 13:14:34.559840 (+150166us) write_op.cc:301] APPLY: starting
0709 13:14:34.559994 (+   154us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:34.652113 (+ 92119us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:34.652159 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:34.854212 (+202053us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:34.929628 (+ 75416us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=28,mrs_lookups=0
0709 13:14:34.938878 (+  9250us) write_op.cc:312] APPLY: finished
0709 13:14:35.016888 (+ 78010us) log.cc:844] Serialized 129 byte log entry
0709 13:14:35.037311 (+ 20423us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:35.072424 (+ 35113us) write_op.cc:454] Released schema lock
0709 13:14:35.072496 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:35.076989 (+  4493us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:35.077022 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17362,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":112345,"apply.run_cpu_time_us":26867,"apply.run_wall_time_us":546169,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62843,"num_ops":14,"prepare.queue_time_us":285120,"prepare.run_cpu_time_us":21739,"prepare.run_wall_time_us":312610,"raft.queue_time_us":1033,"raft.run_cpu_time_us":8235,"raft.run_wall_time_us":209369,"replication_time_us":234948,"spinlock_wait_cycles":375373184,"thread_start_us":162405,"threads_started":4,"wal-append.queue_time_us":16514,"wal-append.run_cpu_time_us":4567,"wal-append.run_wall_time_us":1833698}]]}
I20260709 13:14:37.302660 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:37.303530  3847 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:37.306120 28052 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:37.307206  3859 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:37.307435 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:37.308372  3859 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:37.309247  3851 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.309926  3851 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:37.312664  3851 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:37.266625  3868 raft_consensus.cc:493] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.318969 28230 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:37.323139 26959 tablet_replica.cc:333] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:37.292344  3876 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:37.260278  3842 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:37.330919 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 70942us
I20260709 13:14:37.346735 26959 raft_consensus.cc:2243] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:37.285439  3815 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
W20260709 13:14:37.489143 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.329861  3825 consensus_queue.cc:1048] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
W20260709 13:14:37.565362 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 69575.6us
I20260709 13:14:37.384949  3858 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:37.371338  3888 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:37.446686 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 228486us
I20260709 13:14:37.492043 28160 raft_consensus.cc:2468] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
W20260709 13:14:37.377775  3900 log.cc:927] Time spent T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:14:37.566476  3860 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.329843  3868 raft_consensus.cc:515] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:37.494203  3856 consensus_meta.cc:287] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.768s	user 0.011s	sys 0.003s
I20260709 13:14:37.329306  3893 consensus_queue.cc:1048] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.161s
I20260709 13:14:37.488921  3712 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:37.368783  3859 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.133s
W20260709 13:14:37.564810  3692 consensus_meta.cc:287] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.010s
I20260709 13:14:37.599735 28124 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:37.632252 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:37.600020  3860 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:37.566042  3837 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:37.601356  3842 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:37.602463 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:37.602620  3868 leader_election.cc:290] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:37.603044  3864 consensus_queue.cc:1059] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:14:37.610946 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2953517089999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
W20260709 13:14:37.602632  3903 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:14:37.619554 26959 raft_consensus.cc:2272] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:37.645846 28230 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 22: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:37.690794 28156 raft_consensus.cc:2410] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 13:14:37.690253 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:37.705113 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:37.775924 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.894876 28158 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:37.635219  3832 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:37.901804 28155 consensus_meta.cc:287] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.633s	user 0.023s	sys 0.012s
I20260709 13:14:37.902443 28155 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:37.903647 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:37.905086 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:37.906005 28155 raft_consensus.cc:2393] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 25: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:37.584148  3858 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:37.596242  3888 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:37.914021 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.631651  3712 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:37.924669 28155 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:37.690546  3826 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:37.975636 28160 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:37.905303  3859 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.905421  3856 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.905612  3692 raft_consensus.cc:2764] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 13:14:37.981400 28199 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Timed out: RequestConsensusVote RPC to 127.26.83.193:45587 timed out after 1.724s (SENT)
I20260709 13:14:37.927134  3847 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.991839  3918 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:37.992373  3783 consensus_meta.cc:287] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.883s	user 0.010s	sys 0.001s
I20260709 13:14:37.913901  3811 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.909430  3833 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
W20260709 13:14:37.976537  3691 consensus_meta.cc:287] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.999s	user 0.007s	sys 0.004s
I20260709 13:14:38.013827 28759 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:37.911868  3832 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:37.724543  3837 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:37.996351 28125 consensus_peers.cc:597] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:38.020166 28160 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:37.993345  3896 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:38.034571 28125 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:38.035645 28125 consensus_peers.cc:597] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:38.036229 28125 consensus_peers.cc:597] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:38.038172 28125 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.038695 28125 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:38.034322 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 434626us
W20260709 13:14:38.041014 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 71077us
W20260709 13:14:38.043860 28125 leader_election.cc:336] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.021937  3691 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:38.000595  3856 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:37.994144  3859 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.023658  3705 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:38.053107 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.055564 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.007362  3847 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.014840  3811 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.034479  3223 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:38.058456 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.055200 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.059867 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.061956 28153 raft_consensus.cc:1240] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 19. Current term is 20. Ops: [17.12-19.13]
I20260709 13:14:38.040589  3920 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:38.012408  3783 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.029248  3833 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.084637 28761 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:38.013134  3918 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:38.085879 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.041970  3896 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.075893  3691 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:38.088992  3905 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
I20260709 13:14:38.115911 28158 raft_consensus.cc:2468] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:38.063130  3705 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.073081  3223 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.089854  3925 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:38.114424 28759 raft_consensus.cc:2368] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 26. Current term is 27.
W20260709 13:14:38.086448  3851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.002s	sys 0.009s
I20260709 13:14:38.176200 28155 raft_consensus.cc:1217] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Deduplicated request from leader. Original: 20.17->[22.18-22.18]   Dedup: 22.18->[]
I20260709 13:14:38.176822 28155 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 13:14:38.177079 28049 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.180323 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.113824  3860 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.120752  3927 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.196455  3896 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.233955 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:38.234764 28152 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.000s
I20260709 13:14:38.235251 28152 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
W20260709 13:14:38.201699  3916 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:14:38.209851  3811 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.214772  3918 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:38.216459  3847 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.218489  3223 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:38.218814  3832 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.219609  3931 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.222155  3705 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.223539  3856 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.223748  3860 consensus_queue.cc:1059] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:38.175876  3925 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.329879  3931 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.335503  3931 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.335551  3925 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:38.225832  3837 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.224905  3932 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:38.228036  3833 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.123049  3691 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.196455 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:38.228668  3710 consensus_meta.cc:287] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.973s	user 0.009s	sys 0.004s
I20260709 13:14:38.336836  3932 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.337790  3710 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.257077  3927 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.338572  3691 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.187862  3888 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:38.230027 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2453803529999732s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22450000000000003s for a response
I20260709 13:14:38.181094  3858 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.323498 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:38.339326 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2422517090001293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
I20260709 13:14:38.339707 28152 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 24.
I20260709 13:14:38.340020  3927 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:38.340178 28199 connection.cc:256] RPC call timeout handler was delayed by -1.231131022999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1912s for a response
I20260709 13:14:38.193430  3826 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 30 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:38.281051  3875 consensus_meta.cc:287] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.066s	user 0.010s	sys 0.002s
I20260709 13:14:38.341446  3783 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.341578 28154 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:38.329494  3847 raft_consensus.cc:3055] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:38.195420  3712 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:38.230060  3859 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.290990 28157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50460 (request call id 3529) took 1297 ms. Trace:
I20260709 13:14:38.313970  3851 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:38.348232  3851 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.350471  3851 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.350572  3710 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:38.340714 28199 connection.cc:256] RPC call timeout handler was delayed by -1.2197517090000929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
I20260709 13:14:38.342427  3847 raft_consensus.cc:740] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:38.323648  3811 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:38.343051 28157 rpcz_store.cc:276] 0709 13:14:36.938099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.938202 (+   103us) service_pool.cc:224] Handling call
0709 13:14:38.233727 (+1295525us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b5800215198. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:14:38.235027 (+  1300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5637888}
I20260709 13:14:38.353997  3847 consensus_queue.cc:260] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.354990 28157 raft_consensus.cc:1240] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:38.355249  3847 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:38.322952  3849 consensus_meta.cc:287] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 1.172s	user 0.010s	sys 0.006s
I20260709 13:14:38.344023  3860 consensus_queue.cc:1243] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 70996712c3c942468a6006602de6477f log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:14:38.345026  3932 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.312922  3910 raft_consensus.cc:493] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:38.357626  3910 raft_consensus.cc:515] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:38.358294 28199 leader_election.cc:336] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.358429  3744 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.358873 28199 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.360150  3910 leader_election.cc:290] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.362614 28161 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1556) took 1166 ms. Trace:
I20260709 13:14:38.363085 28161 rpcz_store.cc:276] 0709 13:14:37.195984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.196185 (+   201us) service_pool.cc:224] Handling call
0709 13:14:37.222166 (+ 25981us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:37.326438 (+104272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:37.326473 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:37.349405 (+ 22932us) log.cc:844] Serialized 26 byte log entry
0709 13:14:37.698791 (+349386us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:14:37.698837 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.698913 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:38.362380 (+663467us) raft_consensus.cc:1685] finished
0709 13:14:38.362400 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:38.362533 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19332,"raft.queue_time_us":62143,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":3905,"thread_start_us":430487,"threads_started":2,"wal-append.queue_time_us":701192}
I20260709 13:14:38.365206 28158 raft_consensus.cc:2410] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b for term 22 because replica has last-logged OpId of term: 21 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 18.
I20260709 13:14:38.366016 28234 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:14:38.366505 28234 pending_rounds.cc:85] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 10
I20260709 13:14:38.366812 28234 pending_rounds.cc:107] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:14:38.367100 28234 raft_consensus.cc:2889] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:38.367933 28234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39712 (request call id 3854) took 1209 ms. Trace:
I20260709 13:14:38.375288  3820 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:38.388427 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.388798 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.389480 28231 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:38.390317 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.391227 28227 raft_consensus.cc:2468] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:38.392966 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.393718 28227 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 25.
W20260709 13:14:38.400487 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.381848  3750 raft_consensus.cc:2955] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 8 to 13, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:38.368314 28234 rpcz_store.cc:276] 0709 13:14:37.158483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.160861 (+  2378us) service_pool.cc:224] Handling call
0709 13:14:38.365271 (+1204410us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b58001c0598. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552eee 0000000000375939 00007f59d35c61e0 00007f59d368f0c4 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:14:38.365472 (+   201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.367599 (+  2127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.367841 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8828544}
I20260709 13:14:38.408116 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.411968 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:38.414301 28051 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.415179 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.416139 28235 raft_consensus.cc:2468] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:38.417940 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.418808 28233 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:38.420534 28231 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:38.407577  3712 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:38.429423 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:38.430485 28229 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:38.437728  3907 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:38.438390  3907 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:38.440913  3223 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.441485  3223 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.442421 28153 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3527) took 1505 ms. Trace:
I20260709 13:14:38.442979 28153 rpcz_store.cc:276] 0709 13:14:36.937298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.937436 (+   138us) service_pool.cc:224] Handling call
0709 13:14:36.948236 (+ 10800us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:38.100941 (+1152705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.442019 (+341078us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400196ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:38.442338 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2560657280}
I20260709 13:14:38.443982  3223 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.444909  3918 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:38.445626  3918 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.447235  3716 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:38.447937  3918 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:38.449167  3904 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.880s	user 0.002s	sys 0.000s
I20260709 13:14:38.411266  3832 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.419265  3932 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.457635 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.459513 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.439646  3907 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.422922  3860 raft_consensus.cc:493] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.424903  3925 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.485520 28231 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:14:38.476657 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 301197us
I20260709 13:14:38.479513 28159 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50460 (request call id 3535) took 1499 ms. Trace:
I20260709 13:14:38.444619  3223 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.472060 28233 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:38.488633 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.495216 28234 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:38.501552 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:38.509055 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.511386 28235 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
I20260709 13:14:38.512439 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:38.513331 28231 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:38.513911 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:38.483355  3857 raft_consensus.cc:3055] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:38.407744  3820 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.514803 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:38.482679  3932 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.482515  3728 raft_consensus.cc:3055] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:38.487695 28159 rpcz_store.cc:276] 0709 13:14:36.979606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.979770 (+   164us) service_pool.cc:224] Handling call
0709 13:14:36.980192 (+   422us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:37.002328 (+ 22136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:37.002369 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:37.024495 (+ 22126us) log.cc:844] Serialized 52 byte log entry
0709 13:14:37.243500 (+219005us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:37.317165 (+ 73665us) log.cc:844] Serialized 14 byte log entry
0709 13:14:37.319973 (+  2808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.320180 (+   207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:38.384330 (+1064150us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:38.453004 (+ 68674us) raft_consensus.cc:1685] finished
0709 13:14:38.479215 (+ 26211us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:38.479425 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18233,"prepare.queue_time_us":361056,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":32384,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":9936,"spinlock_wait_cycles":29929344,"thread_start_us":211407,"threads_started":2,"wal-append.queue_time_us":263713}
I20260709 13:14:38.516075  3717 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:38.455452  3712 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.529119 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.529784 28233 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:38.523952 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.470885  3832 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.522993 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:38.516176  3864 raft_consensus.cc:2804] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:38.526290 28052 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:38.534031 28229 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:38.516651  3857 raft_consensus.cc:740] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:38.493889  3925 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.519727  3728 raft_consensus.cc:740] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:38.506460  3860 raft_consensus.cc:515] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.505800  3223 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.527563  3717 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.538642 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.541044 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:38.548182 28052 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:14:38.556937 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 198664us
W20260709 13:14:38.557894 28199 leader_election.cc:336] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.558394 28199 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.558548  3820 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:38.535427  3938 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:38.559609  3938 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.559964  3938 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:38.560262  3710 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:38.559854  3811 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:38.561935  3932 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.562343  3811 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:38.533809  3864 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.569005 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.568706  3811 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:38.574903 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.575963  3940 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:38.576602  3940 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.578974  3940 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:38.579313  3924 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.677s	user 0.006s	sys 0.007s
I20260709 13:14:38.557307  3857 consensus_queue.cc:260] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:38.582554 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.598121 28762 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35892 (request call id 1389) took 1699 ms. Trace:
I20260709 13:14:38.598582 28762 rpcz_store.cc:276] 0709 13:14:36.899149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.899336 (+   187us) service_pool.cc:224] Handling call
0709 13:14:36.916293 (+ 16957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:37.292202 (+375909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.597818 (+1305616us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b5400563954. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:38.598051 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844392704}
I20260709 13:14:38.606698 28155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50458 (request call id 3068) took 1393 ms. Trace:
I20260709 13:14:38.607157 28155 rpcz_store.cc:276] 0709 13:14:37.213291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.925862 (+712571us) service_pool.cc:224] Handling call
0709 13:14:37.926248 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:38.177221 (+250973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.606455 (+429234us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5400376ed4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:14:38.606635 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574540544}
I20260709 13:14:38.607901 28049 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:38.609187 28154 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 3070) took 1349 ms (client timeout 1586 ms). Trace:
W20260709 13:14:38.609665 28154 rpcz_store.cc:269] 0709 13:14:37.260261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.266594 (+  6333us) service_pool.cc:224] Handling call
0709 13:14:38.609011 (+1342417us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400053554. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:38.609126 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2469704960}
I20260709 13:14:38.582403  3832 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.528465  3898 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:14:38.612187  3925 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:38.613366 28156 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 3064) took 1403 ms (client timeout 1770 ms). Trace:
W20260709 13:14:38.613838 28156 rpcz_store.cc:269] 0709 13:14:37.209819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.211858 (+  2039us) service_pool.cc:224] Handling call
0709 13:14:38.613142 (+1401284us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b5400342bd4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:38.613305 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948080384}
W20260709 13:14:38.614382 28125 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.614894 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.616958 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:38.617240  3815 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 13:14:38.585217 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:38.618256 28124 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.551113  3717 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:38.588924 28728 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:38.589533 28760 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:14:38.591049  3860 leader_election.cc:290] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:38.591363  3753 raft_consensus.cc:670] T 6f5f1be456de4894af76dffdad9ea6a3 P 0faaace2b36c4bcebf2fa6835235e92b: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping [suppressed 2 similar messages]
I20260709 13:14:38.593247 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.593453  3712 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:38.584239 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.546365  3936 raft_consensus.cc:2804] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:38.615850 28160 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50458 (request call id 3073) took 1255 ms. Trace:
I20260709 13:14:38.582471 28228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44070 (request call id 1802) took 1302 ms. Trace:
W20260709 13:14:38.618410 28125 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.635419 28125 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.564352  3865 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:38.573007  3864 raft_consensus.cc:3060] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:38.631629 28231 raft_consensus.cc:2468] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 27.
I20260709 13:14:38.632434 28049 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.566916 28052 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:38.635236 28228 rpcz_store.cc:276] 0709 13:14:37.280337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.280765 (+   428us) service_pool.cc:224] Handling call
0709 13:14:37.293870 (+ 13105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:37.331956 (+ 38086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:37.331990 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:37.360627 (+ 28637us) log.cc:844] Serialized 26 byte log entry
0709 13:14:37.902197 (+541570us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:14:37.902256 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.902327 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:38.578648 (+676321us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:14:38.582174 (+  3526us) raft_consensus.cc:1685] finished
0709 13:14:38.582196 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:38.582412 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46944,"raft.queue_time_us":64952,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":5641,"spinlock_wait_cycles":5380480,"thread_start_us":163258,"threads_started":1,"wal-append.queue_time_us":323944}
I20260709 13:14:38.636935 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.634738 28160 rpcz_store.cc:276] 0709 13:14:37.360338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.602156 (+241818us) service_pool.cc:224] Handling call
0709 13:14:38.615678 (+1013522us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54000a37d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:38.615787 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758151424}
I20260709 13:14:38.627046  3935 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:14:38.639912 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 73119.5us
I20260709 13:14:38.648366 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:38.649787 28052 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:38.639360  3815 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:38.638408  3865 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.634379  3936 raft_consensus.cc:493] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:38.642627  3932 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:38.647143 28761 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 27.
I20260709 13:14:38.635062  3860 consensus_queue.cc:1059] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
W20260709 13:14:38.641017 28125 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:38.657821 28127 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.584s (SENT)
I20260709 13:14:38.643189  3744 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:14:38.661180 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.677078 28125 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:38.674268 28198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 150807us
I20260709 13:14:38.660871 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.679374 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.680197 28233 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:38.680366 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60a7d81169af4acc91d8f3b342ac4885" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.680718 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:38.677098 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:38.651811  3919 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:38.684190 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.682755 28730 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:38.652041  3753 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:14:38.689682 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48714.6us
W20260709 13:14:38.690452 28125 leader_election.cc:336] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.690850 28125 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:38.690910 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.657068  3865 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:38.655965  3832 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:14:38.701820 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.703969 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.709368 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.693302 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:38.710785 28125 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.660169  3932 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.659502  3936 raft_consensus.cc:3060] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:38.732259 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.712530 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:38.734565 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.735538 28161 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:38.736320  3895 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:38.734532 28125 leader_election.cc:336] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.737263 28125 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.738283  3875 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:14:38.738474 28125 leader_election.cc:336] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.739022  3875 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:38.739744 28152 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43184 (request call id 1558) took 1250 ms. Trace:
W20260709 13:14:38.740489 28125 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:38.712916 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 84637us
I20260709 13:14:38.721479 28756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.712105 28154 raft_consensus.cc:2393] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 22: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:38.743932 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:38.743950 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 212785us
I20260709 13:14:38.696381  3712 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:38.695849  3910 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.746639 28762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.740127 28152 rpcz_store.cc:276] 0709 13:14:37.488960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.340459 (+851499us) service_pool.cc:224] Handling call
0709 13:14:38.368743 (+ 28284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.368960 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 13:14:38.369243 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 13:14:38.548763 (+179520us) log.cc:844] Serialized 14 byte log entry
0709 13:14:38.549146 (+   383us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:38.549221 (+    75us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:14:38.549291 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.549383 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:38.739013 (+189630us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b18004ed000. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb5582a6 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4
0709 13:14:38.739576 (+   563us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b5400317750. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:38.739691 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1297,"prepare.queue_time_us":244,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":13239,"raft.queue_time_us":11930,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":154,"spinlock_wait_cycles":97127040,"thread_start_us":177751,"threads_started":1,"wal-append.queue_time_us":265007}
I20260709 13:14:38.699962  3846 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:38.748788 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:38.693904  3943 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.699677  3882 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:38.692579  3753 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:38.709192  3921 raft_consensus.cc:2804] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:14:38.694247  3835 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:38.751631 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:38.757553 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 199713us
I20260709 13:14:38.723016  3849 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:38.733336  3932 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:38.766541 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.755172 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.760589 28154 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:38.767766 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.768006 28229 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:38.760983 28155 raft_consensus.cc:1240] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 13:14:38.768231 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.804019 28757 raft_consensus.cc:1240] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
W20260709 13:14:38.765328 28125 leader_election.cc:336] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.760828 28761 raft_consensus.cc:1240] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 21. Current term is 22. Ops: []
I20260709 13:14:38.761106 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.742908  3832 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:38.755055  3910 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.758973  3753 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:38.754539  3860 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.807767 28152 raft_consensus.cc:2468] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:38.773131 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.769304 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.833031 28161 raft_consensus.cc:2468] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 26.
I20260709 13:14:38.833297 28158 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 27.
I20260709 13:14:38.750334  3712 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:38.786912  3855 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.834152  3712 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:38.834321  3855 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.836705  3855 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.760434  3943 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.838253 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.838716  3943 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:38.783073 28199 leader_election.cc:336] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.839454 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.769681  3835 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.839840 28199 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.840632 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:38.840899 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 57923.6us
W20260709 13:14:38.841619 28199 leader_election.cc:336] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.841912 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:38.842012 28199 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.842376  3835 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.787501  3938 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.842839  3942 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:38.843026 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:38.843304 28127 leader_election.cc:336] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.508s (SENT)
I20260709 13:14:38.843588  3942 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:38.844517  3938 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.754391  3750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:53636 (request call id 0) took 4465 ms. Trace:
I20260709 13:14:38.755785  3935 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.769929 28159 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 25.
W20260709 13:14:38.844353 28127 leader_election.cc:336] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.698s (SENT)
I20260709 13:14:38.845140 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:38.849579 28127 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.700s (SENT)
I20260709 13:14:38.850730 28161 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:38.844132 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.852459 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.798756  3936 raft_consensus.cc:515] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:38.801831  3941 log.cc:927] Time spent T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:14:38.797005  3864 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.856933 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.767253 28153 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
W20260709 13:14:38.825911 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54676.4us
W20260709 13:14:38.827931 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 62647.7us
I20260709 13:14:38.861429 28157 raft_consensus.cc:2468] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 18.
I20260709 13:14:38.763772  3716 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:38.819552 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:38.863335 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 43911.9us
I20260709 13:14:38.767448  3921 raft_consensus.cc:697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 27 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:38.847029 28199 leader_election.cc:336] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.854216 28158 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
W20260709 13:14:38.876488 28052 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.829946  3945 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:38.889536 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:38.809496  3933 log.cc:927] Time spent T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.536s	user 0.005s	sys 0.001s
I20260709 13:14:38.850594 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:38.806408 26959 tablet_replica.cc:333] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:38.848623  3750 rpcz_store.cc:276] 0709 13:14:34.281094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.324141 (+ 43047us) service_pool.cc:224] Handling call
0709 13:14:34.443498 (+119357us) log.cc:844] Serialized 501 byte log entry
0709 13:14:34.699294 (+255796us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b180012b6e0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d445f5b1 00007f59cb504c0a 00007f59d4c44219 00007f59cb58ca40 00007f59cb5874e4 00007f59cb587455 00007f59ce5d0b0b 00007f59ce655b63 00007f59ce67d752 00007f59ce6759fa
0709 13:14:38.731141 (+4031847us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":159057,"raft.queue_time_us":163720,"raft.run_cpu_time_us":1458,"raft.run_wall_time_us":1459,"spinlock_wait_cycles":15199616,"thread_start_us":88617,"threads_started":1,"wal-append.queue_time_us":221321,"wal-append.run_cpu_time_us":2361,"wal-append.run_wall_time_us":1002357}
W20260709 13:14:38.861907 28125 leader_election.cc:336] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.894243 28728 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:38.875362 28050 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:38.877494 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.853009 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:38.887368 28161 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
I20260709 13:14:38.891466 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:38.891579 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.897509 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.897975 28728 leader_election.cc:304] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:38.893940 28229 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:38.899130 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:38.899473 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:38.899959 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:38.900574 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:38.894196 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.856845  3860 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.874240 28152 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
I20260709 13:14:38.900323 28229 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:38.905942 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.901398 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:38.905153 28125 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.880837  3929 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:38.913568 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.914453 28156 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 27.
I20260709 13:14:38.917160 28153 raft_consensus.cc:1275] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:14:38.920439 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 71095.8us
I20260709 13:14:38.928128 28158 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:14:38.912223  3710 raft_consensus.cc:3060] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:38.937073 27968 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b99bb21dbacb4d63839f1403ef9b08b9 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:38.915761  3829 raft_consensus.cc:2804] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:38.938299  3829 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.938714  3829 raft_consensus.cc:3060] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:38.905678  3744 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 13:14:38.942623  3936 leader_election.cc:290] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.944456 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:38.944679  3921 consensus_queue.cc:237] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:38.922070 28125 leader_election.cc:336] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:38.945748  3956 log.cc:927] Time spent T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:14:38.915711  3927 raft_consensus.cc:2804] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:14:38.947551  3927 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:38.947975  3927 raft_consensus.cc:3060] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:38.897089  3836 consensus_queue.cc:1048] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:14:38.951683 28229 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 26.
I20260709 13:14:38.916323  3894 raft_consensus.cc:2804] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:38.953493  3894 raft_consensus.cc:697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:38.954463  3894 consensus_queue.cc:237] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:38.923014 28199 leader_election.cc:336] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.948328 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a732ff5937ea47eaaef6755cac5ad60e" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:38.958319 28199 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:38.873152  3935 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.958844 26959 raft_consensus.cc:2243] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:14:38.936141  3811 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:38.961977 28125 leader_election.cc:336] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:38.915884  3895 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:38.916688  3929 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.974036 26959 raft_consensus.cc:2272] T 0720cab120de4047b423cb89ed85154a P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:38.958021  3832 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:38.975665  3950 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:38.976970  3895 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:38.977419  3868 raft_consensus.cc:3055] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:38.977924  3868 raft_consensus.cc:740] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:38.978179  3895 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.978909  3868 consensus_queue.cc:260] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.980408  3868 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:38.898368  3945 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:38.986791  3223 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:38.959280  3855 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:14:38.965032  3851 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:38.983009  3907 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:38.982930 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:38.984969  3860 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:38.979563  3957 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:14:38.980829  3864 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:38.988888 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.990418 28050 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:39.030717 28155 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 25.
I20260709 13:14:39.031371 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.031558 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:38.991762 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:39.011355 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:39.025841 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:39.028930 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.034483 28158 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:38.987708  3929 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.038120 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:39.038972 28125 consensus_peers.cc:597] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:39.039043 28050 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:38.993261  3855 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:39.041172 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.041720 28229 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 17.
W20260709 13:14:39.047174 28127 leader_election.cc:336] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.055s (SENT)
I20260709 13:14:39.047819 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:39.007860  3223 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:39.049042 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 126044us
I20260709 13:14:39.045032 28050 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:39.051403 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:39.039871  3859 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:39.053619 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:39.053802  3826 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:14:39.058586 28199 leader_election.cc:336] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:39.059134 28199 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.058285 28050 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:39.040215  3860 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:39.067919  3705 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:39.070499  3919 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:39.075906  3728 consensus_queue.cc:260] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.076828  3712 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.078948  3945 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:39.046080  3855 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:39.058871  3223 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.051122  3744 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:39.082691  3753 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.057708  3864 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:14:39.083841  3958 log.cc:927] Time spent T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.155s	user 0.002s	sys 0.000s
W20260709 13:14:39.085175  3964 log.cc:927] Time spent T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:14:39.087604 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:39.088459 26959 tablet_replica.cc:333] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:39.161520 26959 raft_consensus.cc:2243] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:14:39.065372  3932 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.095728 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:39.152848  3860 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.101024  3716 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:14:39.102408  3951 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.157s	user 0.000s	sys 0.002s
W20260709 13:14:39.102741 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:39.104202 28199 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:39.167829 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:39.029896  3865 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.007434  3717 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.172279 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.146855 28049 leader_election.cc:304] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.173030 28230 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:39.159004  3851 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:39.160590  3728 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:39.138047  3705 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:39.152635 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 154882us
I20260709 13:14:39.156424 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:39.140234  3919 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:39.228164  3829 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.229264 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.171710  3727 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:39.231187  3727 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.198171  3717 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.231691  3727 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:39.100708  3935 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:39.232964  3886 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:14:39.166095 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 78570.6us
I20260709 13:14:39.227075 26959 raft_consensus.cc:2272] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:39.157788  3744 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.143163 28229 raft_consensus.cc:1240] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:39.173713 28049 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.092254  3710 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:39.159663  3815 raft_consensus.cc:493] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:39.177321 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 130201us
I20260709 13:14:39.160349  3864 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.242658  3815 raft_consensus.cc:515] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.235235 28233 raft_consensus.cc:3055] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:14:39.244887  3815 leader_election.cc:290] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:39.244940 28233 raft_consensus.cc:740] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:39.178077  3945 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:39.171520  3860 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.246191 28233 consensus_queue.cc:260] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:39.247277 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 73636.8us
I20260709 13:14:39.247915  3935 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:39.249315 28049 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.190439  3932 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.192965  3857 raft_consensus.cc:3060] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:39.196206  3223 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.251541 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.160317  3811 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.179205 28052 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:39.252336 28234 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:39.214449  3865 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.225747 28755 raft_consensus.cc:2410] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:14:39.200667  3712 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.226864  3705 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:14:39.257043 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 77923.9us
I20260709 13:14:39.258320 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:39.196570  3929 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.259706  3935 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:39.199682  3753 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.260540  3935 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:14:39.239195  3744 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.261125  3935 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:39.262202  3935 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:39.225298  3961 log.cc:927] Time spent T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:14:39.200719  3847 raft_consensus.cc:2749] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:39.243083  3864 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.139742  3927 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.266424  3927 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:39.186622 28759 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56420 (request call id 1750) took 1151 ms. Trace:
I20260709 13:14:39.267037 28759 rpcz_store.cc:276] 0709 13:14:38.035678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.047227 (+ 11549us) service_pool.cc:224] Handling call
0709 13:14:39.186375 (+1139148us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b54004371d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:14:39.186537 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97690496}
I20260709 13:14:39.247429  3860 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.188717  3910 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:39.250797 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:14:39.251619 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 147463us
I20260709 13:14:39.244942  3851 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:39.285458 28231 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.247277 28233 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:39.218385 28728 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:39.286394 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 68059.5us
I20260709 13:14:39.271242 26959 tablet_replica.cc:333] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:39.225674  3962 log.cc:927] Time spent T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.198s	user 0.000s	sys 0.006s
I20260709 13:14:39.287369 28199 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:14:39.290448 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:39.268529  3965 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.150s	user 0.003s	sys 0.000s
I20260709 13:14:39.293334 28049 leader_election.cc:304] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:39.291239 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:39.292169  3907 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 13:14:39.261088  3753 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:14:39.306375 28728 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:39.303449 28199 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:39.307367 28728 leader_election.cc:336] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:39.308418 28728 leader_election.cc:336] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:39.307947 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:39.312166 28049 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.302363 28231 raft_consensus.cc:2468] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:39.315527  3943 raft_consensus.cc:2764] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
W20260709 13:14:39.309193 28728 leader_election.cc:336] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:39.313298 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:39.317044 28049 leader_election.cc:400] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 26. Current term is 27.
I20260709 13:14:39.317119 28728 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:39.317795 28049 leader_election.cc:403] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:14:39.319483 28728 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:39.320866 28728 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:39.324275 28728 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:39.311475  3716 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:39.326591 28199 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.312152  3829 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:39.318188  3950 raft_consensus.cc:2804] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:39.318132  3927 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:39.316294  3865 raft_consensus.cc:2749] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:14:39.331897 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:39.332839 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:39.322376  3907 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:39.322005  3847 raft_consensus.cc:3060] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:14:39.320107  3223 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:39.345574  3940 raft_consensus.cc:2764] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:14:39.347163 28230 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
W20260709 13:14:39.350128 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.362893 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.363498 28231 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:14:39.370529 28231 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:39.370761 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.371800 28230 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:39.335501 28728 leader_election.cc:336] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:39.354182 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:39.355059 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:39.355800 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:39.380280 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.382556 28231 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:39.382488 28155 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:39.384243 28728 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:39.388630 28052 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:39.393800 28231 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
W20260709 13:14:39.395272 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 62533.5us
I20260709 13:14:39.395570 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:39.396219 28199 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.399119 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:39.400946 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:39.404716 28052 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:39.405514 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:39.406401 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:39.407027 28157 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:39.408004 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:39.408882 28161 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:39.409811 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:39.396445 28153 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:39.413303 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:39.413599 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.414505 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd81a684ef6b4e0f9cc48bc3bcc280e8" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:39.414670 28229 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:39.341917  3942 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:39.324007  3864 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.421566 28052 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:39.424221 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.427785 28234 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:39.346822  3907 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:39.360414  3950 raft_consensus.cc:697] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:39.431188 28052 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:39.435348 28230 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:39.435778 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:39.441651 28052 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:39.446547 28049 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.463208 28049 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.454957 28157 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:39.441161  3932 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:39.488404  3938 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:39.497402  3932 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:39.498369  3932 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.445907  3940 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:39.445518  3943 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:39.501506  3943 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:39.502854  3717 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:14:39.422897  3860 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:39.503753  3860 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:39.504678  3860 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.434316  3927 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:39.505955  3927 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:39.506843  3927 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.435082  3907 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:39.508440  3898 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:39.511726 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 179040us
I20260709 13:14:39.512763 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:39.513553  3895 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:39.515060 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:39.515955  3895 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:39.517382 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:39.517463  3950 consensus_queue.cc:237] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.518213  3895 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:39.520613 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:14:39.487111  3971 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:14:39.487064  3744 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.488361  3753 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:39.554234  3727 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.554939  3753 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:39.555893  3753 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.558310  3929 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.502421  3943 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.461560  3967 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:39.559950  3967 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.560407  3967 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:39.565336  3744 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.565582 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.568034  3716 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:39.524323  3864 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.525498 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.543004  3811 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:39.450208 28125 leader_election.cc:336] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:39.572386 28125 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.573238  3851 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:39.574025 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 123859us
W20260709 13:14:39.575821  3974 log.cc:927] Time spent T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:14:39.578346  3839 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:14:39.579084  3839 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.579599  3839 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:14:39.580329 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 59789.5us
I20260709 13:14:39.581180 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:39.571210  3811 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.599037  3825 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:14:39.599694  3976 log.cc:927] Time spent T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:14:39.596522 28158 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:39.582525 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.583725 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.601334  3825 consensus_queue.cc:1048] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:14:39.585289 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.602243 28235 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 13:14:39.602880 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.485913  3705 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.603572 28228 raft_consensus.cc:2468] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:39.585482 28232 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:14:39.605305 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:39.606470 28052 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:39.607517  3928 raft_consensus.cc:2804] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:39.607975  3928 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.608369  3928 raft_consensus.cc:3060] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:39.609567 28228 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:39.586179 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.613374 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:39.614338 28156 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.587738 28226 raft_consensus.cc:1217] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Deduplicated request from leader. Original: 17.14->[18.15-21.16]   Dedup: 21.16->[]
I20260709 13:14:39.617291 28226 raft_consensus.cc:1240] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 21. Current term is 22. Ops: [18.15-21.16]
I20260709 13:14:39.591483 28230 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.601621 28227 raft_consensus.cc:2393] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 22: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:39.584491 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.621411 28232 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:14:39.624075 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:39.624755  3864 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:39.625430 28759 raft_consensus.cc:2393] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 18: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:39.480077 28234 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:39.633386 28052 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:39.634708  3864 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:39.635516  3864 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:39.638432  3811 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.636653  3864 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.640609 28233 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:14:39.673992  3907 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.675874  3839 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.676088  3936 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:39.676402 28227 raft_consensus.cc:1240] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 22. Current term is 23. Ops: []
I20260709 13:14:39.657416 28730 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:39.647207 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.677878  3929 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:39.643369 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.678375  3929 raft_consensus.cc:493] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.679100  3929 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:39.669585 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:39.680646  3750 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:39.681066  3750 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:39.681444  3750 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:39.677965 28229 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:39.651400 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.687322 28234 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:39.649617  3705 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.556536  3727 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.667611 28230 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:39.670930 28156 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:39.705698 28233 raft_consensus.cc:1240] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:39.693601 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:39.662777 28155 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:39.647573 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.592057  3919 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:39.709854 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 167810us
I20260709 13:14:39.707960 28231 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:39.714354 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.716986 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:39.718714 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.719451 28156 raft_consensus.cc:1240] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:14:39.705077 28730 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:39.720777 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.720824 28052 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:39.721240 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.722222 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:39.730149 28228 raft_consensus.cc:1240] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:39.734078 28229 raft_consensus.cc:2468] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:39.737771 28234 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:39.569265  3716 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.739164 28052 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:14:39.747243 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.755491 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.757421 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b129d69618f48999f257e73b5b8b1d8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.761174 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20222bc64224781bd97e8b75f2ddfa1" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:39.735316  3855 consensus_meta.cc:287] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.001s
I20260709 13:14:39.764096 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.770627 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.771652 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.754355  3936 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:39.756716  3847 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855) with higher term. Message: Invalid argument: T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for earlier term 26. Current term is 27.
I20260709 13:14:39.784395 28155 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:14:39.757086  3907 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:39.782213 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:39.757138  3895 raft_consensus.cc:2804] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:39.797748 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.786010 26959 raft_consensus.cc:2243] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:14:39.757872  3705 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:39.788563 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.769259 28231 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:39.758919  3967 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.805594  3919 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.805496  3907 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:39.806344  3967 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.812352 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:39.803892  3943 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:39.798826  3936 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:39.773051  3855 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.785593  3846 consensus_queue.cc:1059] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 13:14:39.811987  3895 raft_consensus.cc:697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:39.800441  3910 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:14:39.787554  3851 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:14:39.816540  3705 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:39.850508  3705 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:39.843535 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.853101 27975 catalog_manager.cc:5697] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:39.808346  3945 consensus_queue.cc:1059] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:14:39.854990  3945 raft_consensus.cc:3055] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:39.855429  3945 raft_consensus.cc:740] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:39.856204 27983 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 264) took 30603 ms (client timeout 15000 ms). Trace:
I20260709 13:14:39.856359  3945 consensus_queue.cc:260] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.858052  3945 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.845679  3811 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:39.861785  3811 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.864353  3811 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.845485  3895 consensus_queue.cc:237] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.833976  3950 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.865660  3950 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.866744  3855 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:39.846830  3839 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:39.847683  3936 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.877419 28759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:39.855111 27975 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 18 to 28. New cstate: current_term: 28 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:39.879563 28759 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b in current term 28: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:39.880471 27975 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 24 to 26, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 26 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:39.856667 27983 rpcz_store.cc:269] 0709 13:14:09.252979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.253284 (+   305us) service_pool.cc:224] Handling call
0709 13:14:09.405719 (+152435us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b54002c0010. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb5530b2 00007f59d44cca89 00007f59d4464da6 00007f59d4436088 00007f59d453437e 00007f59d29f52a4 00007f59d29f5232 00007f59d29f5161 00007f59d29f50dc
0709 13:14:39.856112 (+30450393us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:38.066068 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:38.183385 (+117317us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:38.183463 (+    78us) write_op.cc:435] Acquired schema lock
0709 13:14:38.183494 (+    31us) tablet.cc:662] Decoding operations
0709 13:14:38.206033 (+ 22539us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:38.206186 (+   153us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:38.206239 (+    53us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:14:38.207334 (+  1095us) tablet.cc:701] Row locks acquired
0709 13:14:38.207399 (+    65us) write_op.cc:260] PREPARE: finished
0709 13:14:38.207649 (+   250us) write_op.cc:270] Start()
0709 13:14:38.207850 (+   201us) write_op.cc:276] Timestamp: P: 1783602878207548 usec, L: 0
0709 13:14:38.207884 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:14:38.317906 (+110022us) log.cc:844] Serialized 3248 byte log entry
0709 13:14:38.981761 (+663855us) op_driver.cc:464] REPLICATION: finished
0709 13:14:39.153888 (+172127us) write_op.cc:301] APPLY: starting
0709 13:14:39.153991 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:39.229097 (+ 75106us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:39.229206 (+   109us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:39.471464 (+242258us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:39.471746 (+   282us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:14:39.471796 (+    50us) write_op.cc:312] APPLY: finished
0709 13:14:39.472079 (+   283us) log.cc:844] Serialized 81 byte log entry
0709 13:14:39.472303 (+   224us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:39.472432 (+   129us) write_op.cc:454] Released schema lock
0709 13:14:39.472494 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:39.472965 (+   471us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:39.472992 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1217098,"reported_tablets":48,"spinlock_wait_cycles":66146176,"child_traces":[["op",{"apply.queue_time_us":167444,"apply.run_cpu_time_us":14284,"apply.run_wall_time_us":323428,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37679,"num_ops":8,"prepare.queue_time_us":726132,"prepare.run_cpu_time_us":33676,"prepare.run_wall_time_us":441674,"raft.queue_time_us":341943,"raft.run_cpu_time_us":13232,"raft.run_wall_time_us":135954,"replication_time_us":752420,"spinlock_wait_cycles":62336,"thread_start_us":540558,"threads_started":4,"wal-append.queue_time_us":203844}]]}
W20260709 13:14:39.900216 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 186072us
I20260709 13:14:39.901345  3950 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:39.902024  3875 raft_consensus.cc:3055] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:39.907377  3750 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.908059  3875 raft_consensus.cc:740] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:39.908970  3875 consensus_queue.cc:260] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.910393  3875 raft_consensus.cc:3060] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.913034  3928 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.915374  3928 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:39.818316  3815 consensus_queue.cc:1059] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:14:39.820566  3886 consensus_queue.cc:1059] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:14:39.917752  3846 raft_consensus.cc:3055] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:14:39.917848 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 107357us
I20260709 13:14:39.918282  3846 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:39.919276  3846 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.916571  3815 raft_consensus.cc:3055] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:14:39.922148  3815 raft_consensus.cc:740] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:39.922968  3815 consensus_queue.cc:260] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:39.930126 28764 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:14:39.930562 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:39.931428 28758 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 24: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:39.932922 28760 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:39.933883  3851 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:39.934413 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.932950 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:39.935305  3851 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.937985  3851 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:39.940558  3882 consensus_queue.cc:1059] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:39.990274  3882 raft_consensus.cc:3055] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:39.990778  3882 raft_consensus.cc:740] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:39.991974  3882 consensus_queue.cc:260] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.883733 27975 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 18 to 20, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 20 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:39.993403  3882 raft_consensus.cc:3060] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:39.993958 27975 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:39.940893 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 54232.3us
I20260709 13:14:39.941669  3811 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.998767  3811 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:39.943323 28755 raft_consensus.cc:1240] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:39.945044 26959 pending_rounds.cc:70] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Trying to abort 2 pending ops.
I20260709 13:14:39.948141 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:39.954327  3910 consensus_queue.cc:1059] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:40.003803 26959 pending_rounds.cc:77] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Aborting op as it isn't in flight: id { term: 26 index: 15 } timestamp: 7305637393848320000 op_type: NO_OP noop_request { }
I20260709 13:14:40.004274  3910 raft_consensus.cc:3055] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:40.004906  3910 raft_consensus.cc:740] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:40.005877  3910 consensus_queue.cc:260] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:40.007237  3910 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:40.007441 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:40.009018 28226 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 23.
I20260709 13:14:40.014467 28227 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:40.015084  3811 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:39.924489  3815 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:40.016229 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:40.017159  3811 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:40.017668  3811 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:40.018143  3811 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:39.852384  3929 raft_consensus.cc:515] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:40.022990  3929 leader_election.cc:290] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:40.024068  3927 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:39.972638 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:40.004374 26959 raft_consensus.cc:2889] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:14:40.027616 26959 pending_rounds.cc:77] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b: Aborting op as it isn't in flight: id { term: 24 index: 14 } timestamp: 7305637349454491648 op_type: NO_OP noop_request { }
I20260709 13:14:40.028071 26959 raft_consensus.cc:2889] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 26 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:14:40.028465 26959 raft_consensus.cc:2272] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.033025 26959 tablet_replica.cc:333] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:40.034153 26959 raft_consensus.cc:2243] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:14:40.035460 26959 raft_consensus.cc:2272] T 17fbe81a71fd4f8c83a65145cf2f0682 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.039161 26959 tablet_replica.cc:333] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:39.970404  3750 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:40.040661  3851 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.042285  3851 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:40.045758  3851 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:40.047453  3811 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:40.055637 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:40.040568 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 51044.2us
W20260709 13:14:40.130380 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 330118us
I20260709 13:14:40.146145 28758 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Rejecting Update request from peer 0faaace2b36c4bcebf2fa6835235e92b for earlier term 25. Current term is 26. Ops: []
W20260709 13:14:40.156963 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.190691 28730 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:40.204608 28730 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:40.206158 28198 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:40.214547 28233 raft_consensus.cc:1275] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:14:40.218466 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:40.219528 28226 raft_consensus.cc:3060] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:40.221724 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:40.194713  3927 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.223624  3894 raft_consensus.cc:2804] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:40.229555  3895 raft_consensus.cc:2804] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:40.277230  3846 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:40.293227 28230 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:14:40.313050  3705 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:40.322386 28198 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:40.324666  3894 raft_consensus.cc:697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:40.325573 28199 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.059s	sys 0.067s
W20260709 13:14:40.326828 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.996266 27975 catalog_manager.cc:5697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 18 to 26, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 26 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:40.182672 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 375050us
I20260709 13:14:40.269739 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:40.227849  3943 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:40.328764 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:40.330121 28127 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:40.183495 26959 raft_consensus.cc:2243] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:14:40.340247 28199 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.059s	sys 0.067s
W20260709 13:14:40.340768 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 540832us
W20260709 13:14:40.341533 28199 leader_election.cc:336] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:40.343290 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:40.349330 28226 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:40.352941 26959 raft_consensus.cc:2272] T 9f16469a2e44408daf3969b51ae89aa6 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.243161  3927 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:40.340034  3945 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:40.340500  3882 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:40.341051  3895 raft_consensus.cc:697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:40.362808  3847 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.064s	sys 0.114s
W20260709 13:14:40.369971 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.329849 27975 catalog_manager.cc:5697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 12 to 15, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 15 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.355716  3910 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:40.443691 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:40.444561  3910 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:40.445184  3910 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:40.355129  3864 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.447450  3864 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:40.398983 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.449882  3864 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:40.450665  3921 consensus_queue.cc:1048] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:14:40.452102 28730 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:40.433864 28231 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:40.455183  3836 raft_consensus.cc:2804] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:40.455848  3836 raft_consensus.cc:697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:40.440125 26959 tablet_replica.cc:333] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:40.440631  3847 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.064s	sys 0.115s
I20260709 13:14:40.440898  3994 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:40.456947  3836 consensus_queue.cc:237] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:40.454242 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:40.461015  3925 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 13:14:40.470326  3927 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:40.470677 28728 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:40.472642  3894 consensus_queue.cc:237] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:40.476380  3895 consensus_queue.cc:237] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:40.474691 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:40.367771  3943 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:40.429452  3882 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:40.505297 26959 raft_consensus.cc:2243] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:14:40.506965 28728 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:40.432440 27975 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 24 to 26, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 26 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:40.513199 27975 catalog_manager.cc:5697] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.539153 26959 raft_consensus.cc:2272] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.515129 27975 catalog_manager.cc:5697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 11 to 18, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 18 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:40.550232 27975 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:40.575122 28227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.012s	sys 0.011s
W20260709 13:14:40.575805 28227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.012s	sys 0.011s
I20260709 13:14:40.538380  3815 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:40.584491  3716 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:40.577386  3936 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.017s	sys 0.007s
I20260709 13:14:40.582726 27975 catalog_manager.cc:5697] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:40.596318  3936 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.018s	sys 0.007s
I20260709 13:14:40.597694 27975 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 22 to 25, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 25 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.638343 27975 catalog_manager.cc:5697] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 20, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 20 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.661311 28231 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:14:40.672230  3993 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:14:40.675426  3910 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:40.594692  3716 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:40.681133 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:40.682083 28755 raft_consensus.cc:2393] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 21: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:40.679450  3716 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:40.684427 26959 tablet_replica.cc:333] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:40.685384 26959 raft_consensus.cc:2243] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.686024 26959 raft_consensus.cc:2272] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.688457  3882 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:40.648979  3936 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:40.696499  3936 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:40.698864  3936 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:40.595283  3815 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:40.703624  3815 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:40.704185  3815 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:40.705130  3886 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:40.733259 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 43597.4us
W20260709 13:14:40.738523 28728 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:40.705869  3886 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:40.739635 28728 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:40.747187 26959 tablet_replica.cc:333] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:40.754204 26959 raft_consensus.cc:2243] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:40.660416 27975 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 11 to 15, term changed from 18 to 22, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196), 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193), VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194), VOTER 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195), VOTER 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 22 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.704694  3815 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:40.762818 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 432734us
I20260709 13:14:40.764256 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:40.769048 26959 raft_consensus.cc:2272] T 60a7d81169af4acc91d8f3b342ac4885 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:40.775611 27975 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 20 to 22, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 22 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:40.781637 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:40.781411 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:40.783064 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:40.783535 27975 catalog_manager.cc:5697] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.783213 28758 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:40.785387 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:40.781837 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 440318us
W20260709 13:14:40.786744 28199 leader_election.cc:336] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:40.787083 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:40.792819  3875 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:40.797268  3815 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:40.798650  3886 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:40.785243  3846 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:40.774197  3882 consensus_queue.cc:1059] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:14:40.802345  3882 raft_consensus.cc:3055] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:40.802713  3882 raft_consensus.cc:740] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:40.803501  3882 consensus_queue.cc:260] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:40.804822 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:40.823598 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:40.831705 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45228us
I20260709 13:14:40.834337 28127 leader_election.cc:304] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:40.828894 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:40.857733 28127 leader_election.cc:304] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:40.864147 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:40.866230 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 56687.8us
I20260709 13:14:40.879323 27975 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:40.885206 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.884470 27975 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:40.856902  3943 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:40.861092  3882 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:40.886330  3994 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.874097  3910 raft_consensus.cc:2749] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:40.882941  3875 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.874946  3851 raft_consensus.cc:2749] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:40.909101 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:40.909801 28235 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:40.875977 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 89209us
I20260709 13:14:40.910879 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:40.909611 27975 catalog_manager.cc:5697] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:40.876951  3886 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:40.916654 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:40.917233 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:40.917320 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 570060us
I20260709 13:14:40.917369 28230 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
W20260709 13:14:40.910584 28199 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:40.918854 28050 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:40.919100 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:40.920351 28228 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 13:14:40.930284 28199 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:40.934172 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:40.935079 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:40.936524  3705 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:40.915416  3886 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:40.941768  3886 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:40.914844  3811 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:40.909485  3994 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:40.946478 28728 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:40.947995 28235 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:40.925310  3928 raft_consensus.cc:493] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:40.943295 28052 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
W20260709 13:14:40.955238 28199 leader_election.cc:336] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:40.956053 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:40.951189 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:40.911520  3875 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:40.936262  3950 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:40.982029  3994 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:40.949563  3969 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:40.983248  3969 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:40.950484  3705 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.989125  3705 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:40.991854  3705 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:40.979249  3875 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:40.997117  4005 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:41.002780 28052 raft_consensus.cc:2686] T 6810464df4a245f49df908fc877020d7 P 0faaace2b36c4bcebf2fa6835235e92b: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:14:41.003211 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 59999.6us
I20260709 13:14:41.001415 26959 tablet_replica.cc:333] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:41.001807 28728 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 139 similar messages]
I20260709 13:14:41.005865 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:41.006536 26959 raft_consensus.cc:2243] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.006649 28760 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
W20260709 13:14:41.007464 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 47169.8us
I20260709 13:14:41.008296 26959 raft_consensus.cc:2272] T a732ff5937ea47eaaef6755cac5ad60e P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.008203 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:41.013500 28761 raft_consensus.cc:2393] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 28: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:41.001296  3847 raft_consensus.cc:493] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.008778  3811 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.020100 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:41.022537 28052 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:40.969198  3928 raft_consensus.cc:515] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.038946  3969 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:41.011691  3967 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:14:41.064692 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:41.068378  4009 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:41.069049  4009 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:41.069541  3928 leader_election.cc:290] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:41.024986  3847 raft_consensus.cc:515] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:41.073647 28125 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.028950  3811 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.076436  3945 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:41.133496 28049 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:41.130134  3851 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:41.080614  3811 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:41.069958  4009 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:41.135210  3851 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.132428  3969 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:41.145720 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 126774us
I20260709 13:14:41.149616 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.150442 28155 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:41.152532 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.130533  3875 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:41.153342 28153 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:41.155865 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.130237  3921 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:41.154896 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.157398 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.157758 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.158013 28157 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:41.158448 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:41.166348 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.167014 28152 raft_consensus.cc:3060] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:41.168497 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.168609 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:41.169911 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.169984 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.168104 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.171446 28156 raft_consensus.cc:2410] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:14:41.172466 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.152489  3928 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:41.233906 26959 tablet_replica.cc:333] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:41.229084 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.169106 28158 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:41.141269  3855 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.025s
I20260709 13:14:41.198632 28155 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 24.
I20260709 13:14:41.261821 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0743024f324c14b4c6fc08f70600d2" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
W20260709 13:14:41.189391 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 166954us
I20260709 13:14:41.242159 28161 raft_consensus.cc:2468] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 18.
I20260709 13:14:41.233448  3705 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.262436 28154 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:41.154186  3994 raft_consensus.cc:493] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:41.281329 28155 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:14:41.282523 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:41.245460 28152 raft_consensus.cc:2468] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:41.283633 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e53fd092d3dd48c492e04080969cb666" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:41.285169 28155 raft_consensus.cc:1275] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:41.291846 28158 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:41.155656  4003 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:41.239017 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 283849us
W20260709 13:14:41.301586 28199 leader_election.cc:336] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.302116 28199 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:41.176627 28157 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 28.
W20260709 13:14:41.279690 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.149075  3969 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:41.159799  3811 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.129s
I20260709 13:14:41.161974  3875 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.321669 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:41.324409 28227 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:41.169551  3864 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:41.289036  3705 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:41.348866 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:41.305994  3846 raft_consensus.cc:493] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.357194 26959 raft_consensus.cc:2243] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:14:41.359514 28156 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:41.297910  3994 raft_consensus.cc:515] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:41.335458 28199 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.161803  3921 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.338410 28158 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:14:41.339915 28157 raft_consensus.cc:1275] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:14:41.346438  4023 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.347473  4011 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:14:41.332919 28154 raft_consensus.cc:2468] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 19.
I20260709 13:14:41.384320 28156 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 26.
I20260709 13:14:41.381517  3994 leader_election.cc:290] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:41.390599 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 55152.5us
W20260709 13:14:41.391868 28199 leader_election.cc:336] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.392467 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:41.401579 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.346108  3864 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:41.402838 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 54112.7us
I20260709 13:14:41.408645 28052 leader_election.cc:304] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:41.411267 28199 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.411443 28161 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:14:41.412360 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:41.384603  4011 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:41.366097  3846 raft_consensus.cc:515] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:41.407080  4007 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.361s	user 0.005s	sys 0.000s
W20260709 13:14:41.417544 28127 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 1.728s (SENT)
I20260709 13:14:41.420331 28127 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:41.422581 28050 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:41.422996 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce6e6dfcbee43d5b0550ce7fc634c38" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:41.423444 28050 raft_consensus.cc:2686] T 6ef726aecabb47089dcd217bfb8bfefc P 0faaace2b36c4bcebf2fa6835235e92b: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:14:41.424268 28050 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:41.425417 28154 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:14:41.426396 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.429873 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:41.403301  4005 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:41.431047 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.432425 28157 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:41.427413 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.437165 28157 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:14:41.314837  4003 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:41.410141  3855 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:14:41.438289  4027 raft_consensus.cc:493] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.444232 28158 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:41.444528  4027 raft_consensus.cc:515] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.446043  3705 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:41.447316  3875 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:41.452553  3864 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:41.455040  3921 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:41.456439  4018 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 13:14:41.462823  4019 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:14:41.467090 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:41.417861  4012 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 13:14:41.414213  3847 leader_election.cc:290] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:41.476055 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 51883.5us
W20260709 13:14:41.439271  4016 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
I20260709 13:14:41.491472 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.492319 28152 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 24: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:41.478543 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa289960fe9a4d2c8e61deb822de8412" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:41.470062 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:41.481657 28050 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:41.476297 28161 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:14:41.485289 28234 raft_consensus.cc:1275] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:14:41.485752 28231 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:14:41.489066  4015 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
I20260709 13:14:41.494989 28156 raft_consensus.cc:1275] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:14:41.496258 28156 pending_rounds.cc:85] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 12
I20260709 13:14:41.496793 28156 pending_rounds.cc:107] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:14:41.497103 28156 raft_consensus.cc:2889] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:41.439592  3811 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.393507  4023 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:41.475250  3950 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 13:14:41.519088 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 49089.4us
I20260709 13:14:41.473981  3875 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.483760  4028 raft_consensus.cc:2749] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:41.537159  3875 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:41.506000  4017 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:41.538478  4017 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:41.539592  3875 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:41.521752 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:41.533484  4022 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:14:41.536157 28153 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 21.
I20260709 13:14:41.489725  4024 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:14:41.514703  3882 consensus_meta.cc:287] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.000s
I20260709 13:14:41.550400 28155 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:14:41.526347 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 115108us
W20260709 13:14:41.555385 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 73824.4us
I20260709 13:14:41.560356 28050 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
W20260709 13:14:41.556561 28125 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:41.562233 28125 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.563619  3929 consensus_queue.cc:1048] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:14:41.565423  3886 consensus_meta.cc:287] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Time spent flushing consensus metadata: real 0.597s	user 0.008s	sys 0.004s
I20260709 13:14:41.565857  3886 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:41.559535 28199 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.540519  4017 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:41.569504 28199 leader_election.cc:336] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.579303 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:41.544986 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:41.580412 28227 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:41.570466 28728 leader_election.cc:304] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:41.571151 28159 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:14:41.581544 28159 pending_rounds.cc:85] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Aborting all ops after (but not including) 10
I20260709 13:14:41.581828 28159 pending_rounds.cc:107] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:14:41.581858  4005 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:41.582186 28159 raft_consensus.cc:2889] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:41.562698 28125 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:41.583401 28199 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.583529  3875 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:41.583869 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:41.584354 28125 leader_election.cc:336] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.584874 28125 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:41.586745 28050 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:41.510891  3851 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:41.594110 28050 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:41.549381  3993 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:14:41.601856 28050 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:14:41.606443 28199 leader_election.cc:336] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.578305  3705 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:41.628746  3705 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:41.628153 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 219823us
W20260709 13:14:41.629213 28199 leader_election.cc:336] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.519945  3811 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:41.630297 28199 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.599778  4005 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:41.585368  4024 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
W20260709 13:14:41.602909  4033 log.cc:927] Time spent T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 13:14:41.653496 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 83098.1us
I20260709 13:14:41.578125  4027 leader_election.cc:290] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:41.554697  3750 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 13:14:41.599617  3882 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:14:41.599332  3864 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:14:41.604882  4028 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:41.632448 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:41.651350 26959 raft_consensus.cc:2272] T fc1588fd3df04c2c8354baa741ddeca9 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.651875  4029 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:41.605189  3928 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:14:41.629261  4038 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:41.604496  3847 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:41.677649 28127 leader_election.cc:336] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Timed out: RequestConsensusVote RPC to 127.26.83.195:46875 timed out after 2.076s (SENT)
W20260709 13:14:41.730921 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 53320.3us
I20260709 13:14:41.525774  4011 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.590596  3929 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.057s
I20260709 13:14:41.630775  3994 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:41.718189  3727 raft_consensus.cc:2804] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:41.741173  3727 raft_consensus.cc:697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:41.742254  3727 consensus_queue.cc:237] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.708051  3851 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.633575  3716 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:41.732331 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 102078us
W20260709 13:14:41.750875 28199 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:41.752081 28199 consensus_peers.cc:597] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:41.733327 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:41.617619  4014 consensus_queue.cc:1048] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.118s
I20260709 13:14:41.731997  3811 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:41.736766  4011 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:14:41.748246 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 119384us
I20260709 13:14:41.763311 28050 leader_election.cc:304] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:41.726370  3864 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:41.765503  3864 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.766063  3927 raft_consensus.cc:2749] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:14:41.766867  3927 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:41.767601  3811 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:41.768687  3967 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:41.768970  3811 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:41.771147 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.772190 28159 raft_consensus.cc:2410] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0faaace2b36c4bcebf2fa6835235e92b for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 13:14:41.758342  3846 leader_election.cc:290] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:41.744346  3851 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:41.773031 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 118822us
I20260709 13:14:41.774613 26959 tablet_replica.cc:333] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:41.775182 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 143276us
I20260709 13:14:41.775703 26959 raft_consensus.cc:2243] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:41.778621 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1115d5b02e774b14935e3f48488f82ea" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.784646 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.747231  4021 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:41.797363 28160 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
I20260709 13:14:41.800172  3855 raft_consensus.cc:493] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.801070  3855 raft_consensus.cc:515] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.768998  3705 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:41.805342 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:41.806356 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.807717 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:41.808745 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2929b1e07bb74c67863e4c504d8d09a6" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:41.809552 28050 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:41.810659 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:41.812573 28229 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:14:41.826646 26959 raft_consensus.cc:2272] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 0faaace2b36c4bcebf2fa6835235e92b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:41.828392 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.836684  3967 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.836918  3894 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:41.837383  3967 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:41.840826 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:41.842482  3851 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:41.813499  3855 leader_election.cc:290] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:41.843443  3855 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:41.849362 28235 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:41.827446  3836 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:14:41.817715  3886 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:41.868311 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 114579us
W20260709 13:14:41.869066 28199 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.869393 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:41.866518 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:41.872426  4039 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:14:41.840622  3969 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:41.873893 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 64411.7us
I20260709 13:14:41.880079 28235 raft_consensus.cc:1275] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:14:41.873878 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 219436us
W20260709 13:14:41.891016 28199 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.863023  3967 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:41.871212  4037 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:14:41.876643 28050 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:41.896831 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:41.891523 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:41.905361 28199 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:41.912170 28199 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:41.913146 28199 leader_election.cc:336] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.913599 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:41.892184  3925 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:41.889897  3727 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.905844 28050 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:41.922047 28157 raft_consensus.cc:3055] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:41.922669 28157 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:41.901258  4023 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:41.808040  4024 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:41.913187  4027 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:14:41.963070 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 66392.3us
I20260709 13:14:41.933492 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:41.929342 28157 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:41.909634  3716 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:41.914996  3925 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:41.924012  3967 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:41.919350  3875 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:41.970098 28157 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:41.972528 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:41.977175 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.904261  4044 consensus_queue.cc:1048] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
W20260709 13:14:41.980960 28125 leader_election.cc:336] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:41.931396  3945 raft_consensus.cc:2749] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:41.982996 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:41.983495 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:41.965219  4027 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.965720  3825 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:14:41.988416 28231 raft_consensus.cc:2468] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 22.
W20260709 13:14:41.989845 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.021618 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0720cab120de4047b423cb89ed85154a" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:42.023522 28758 raft_consensus.cc:3060] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:14:41.984925  4039 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:14:42.018029 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:41.986996 28125 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:42.031946 28052 leader_election.cc:304] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:42.025117 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 111989us
I20260709 13:14:42.019958 28760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847d15cbd80f44b18e547291eed25617" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:41.979651  3925 raft_consensus.cc:3060] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:42.035231 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54308.1us
W20260709 13:14:42.033099 28199 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.029333 28761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:42.036098 28125 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.029109 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:42.036404 28199 leader_election.cc:336] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.036959 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:41.928907  3727 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:42.023973  4027 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:42.038970 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:42.040637 28125 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:42.038878 28199 leader_election.cc:336] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.020964  3969 raft_consensus.cc:493] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.049547 28199 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:42.051281 28199 leader_election.cc:336] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.034637  4042 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:14:42.052294  4040 log.cc:927] Time spent T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
W20260709 13:14:42.052416 28199 leader_election.cc:336] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.053092  3969 raft_consensus.cc:515] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:42.053429 28199 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.053957 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:42.055428  3969 leader_election.cc:290] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:42.046924  3825 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 26 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:42.057814  3825 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:42.058557  4047 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:14:42.061048 26959 tablet_replica.cc:333] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:42.062063 26959 raft_consensus.cc:2243] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:42.062654 26959 raft_consensus.cc:2272] T aa289960fe9a4d2c8e61deb822de8412 P 0faaace2b36c4bcebf2fa6835235e92b [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:42.064195  4051 log.cc:927] Time spent T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:14:42.031853  3967 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:42.049697 28125 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:42.039856  3994 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:42.078518 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b; no voters: 
I20260709 13:14:42.040421  3945 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:14:42.081460 28125 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:42.085258 28125 leader_election.cc:336] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.042279  4023 raft_consensus.cc:2749] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:14:42.088306 28125 leader_election.cc:336] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:42.089113 28125 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:42.092763 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64472.8us
I20260709 13:14:42.028638  4045 consensus_queue.cc:1048] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 13:14:42.068692  3950 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:42.095381 26959 tablet_replica.cc:333] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:42.083688  3882 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:14:42.079125  4038 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:42.017127  3875 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:42.110469 26959 raft_consensus.cc:2243] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:42.047730  3847 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:14:42.092309  3716 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:42.092558  4042 raft_consensus.cc:2804] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:14:42.160113 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 46228.8us
W20260709 13:14:42.165380 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 51573.2us
I20260709 13:14:42.173424 28124 leader_election.cc:304] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:42.179288 26959 raft_consensus.cc:2272] T 4b129d69618f48999f257e73b5b8b1d8 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.162947  3847 consensus_queue.cc:1243] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:14:42.162207  4042 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.183475 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:42.113000  4063 consensus_queue.cc:1048] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.071s
W20260709 13:14:42.189196 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.177196  3950 raft_consensus.cc:493] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.194219  3875 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:42.184504  3994 raft_consensus.cc:2749] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:42.199859 28231 raft_consensus.cc:1217] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Deduplicated request from leader. Original: 15.11->[17.12-21.14]   Dedup: 17.12->[19.13-21.14]
I20260709 13:14:42.203881 28758 raft_consensus.cc:2468] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:42.207370 28161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:42.185843  4042 raft_consensus.cc:3060] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:42.212056 28161 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:42.204226 28231 pending_rounds.cc:85] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 12
I20260709 13:14:42.216955 28231 pending_rounds.cc:107] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:14:42.217360 28231 raft_consensus.cc:2889] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:42.219769 26959 tablet_replica.cc:333] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:42.191020  4063 consensus_queue.cc:1243] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [LEADER]: Peer 11f10c12c03e47dcac88b050d4c1431b log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:14:42.221447 26959 raft_consensus.cc:2243] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:14:42.222241 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:42.225931 26959 pending_rounds.cc:70] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Trying to abort 1 pending ops.
I20260709 13:14:42.226300 26959 pending_rounds.cc:77] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting op as it isn't in flight: id { term: 20 index: 12 } timestamp: 7305637403881431040 op_type: NO_OP noop_request { }
I20260709 13:14:42.227043 26959 raft_consensus.cc:2889] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:14:42.228078 26959 raft_consensus.cc:2272] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.203724  3950 raft_consensus.cc:515] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:42.230724  3727 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:42.232334  3950 leader_election.cc:290] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:42.235419  3994 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:42.259737 28153 raft_consensus.cc:1240] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 13:14:42.263275 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.264647 28153 raft_consensus.cc:1275] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:14:42.257336  3945 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:42.270146  4011 consensus_meta.cc:287] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.004s
I20260709 13:14:42.270280  3945 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:42.270962  3994 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:42.273344  3994 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:42.276082 28153 raft_consensus.cc:1240] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 18. Current term is 19. Ops: []
I20260709 13:14:42.265089 28227 raft_consensus.cc:1240] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:14:42.264464 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.278680 28228 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 23.
I20260709 13:14:42.279557 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab37ead976b4378b1298941cec677a8" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.282444 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.288722 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:42.258466  4009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.006s	sys 0.017s
I20260709 13:14:42.287101  4038 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:14:42.295239  4005 consensus_meta.cc:287] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.001s
I20260709 13:14:42.297216 28227 raft_consensus.cc:2393] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 24: Already voted for candidate 70996712c3c942468a6006602de6477f in this term.
I20260709 13:14:42.300340 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0a4a73cb5754c40af548cd4519da5b4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:42.303061  4009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.007s	sys 0.017s
I20260709 13:14:42.303020 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.301170  3945 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:42.308621 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.330061 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:42.333230 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d81eee50848758aae23da5ed8927b" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.334715 28227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ee90e83c2f49509f54dc87ea4d50e6" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.346228  4070 consensus_queue.cc:1059] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:14:42.307411  4005 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:42.348131 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:42.348752  4054 log.cc:927] Time spent T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.264s	user 0.000s	sys 0.003s
I20260709 13:14:42.348874  4069 consensus_queue.cc:1048] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:14:42.340147  4056 consensus_queue.cc:1059] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:14:42.354408  4070 raft_consensus.cc:3055] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:14:42.354979  4070 raft_consensus.cc:740] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:42.355556  4055 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:14:42.358129 28234 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 19.
W20260709 13:14:42.358747  4060 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:14:42.356070  4070 consensus_queue.cc:260] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:42.360302  4070 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:42.338913  3945 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:14:42.362287  4071 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:14:42.362594  3945 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:42.363198  3945 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:42.361251  4056 raft_consensus.cc:3055] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:14:42.367157  4056 raft_consensus.cc:740] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:42.368214  4056 consensus_queue.cc:260] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:42.369701  4042 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:42.372670  4042 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:42.372786 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.374447 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.349882 28228 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
I20260709 13:14:42.376184 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.378805 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:42.380455 26959 tablet_replica.cc:333] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:42.389515 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.369757  4056 raft_consensus.cc:3060] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:42.362489  3925 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:42.403432  4068 log.cc:927] Time spent T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.244s	user 0.002s	sys 0.000s
I20260709 13:14:42.403882  3925 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:42.337777  4027 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:42.408347  4027 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:42.409018 26959 raft_consensus.cc:2243] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:42.413414 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.416980 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:42.417788 28157 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:42.425086  4066 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.313s	user 0.002s	sys 0.000s
I20260709 13:14:42.429020 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:42.431913 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d3461aa17604e69b08928a9bc0f8d18" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:42.432685 28152 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 13:14:42.434578 28052 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:42.435813 28233 raft_consensus.cc:1240] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 23. Current term is 24. Ops: []
I20260709 13:14:42.421063  3886 raft_consensus.cc:493] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.430938  4003 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:42.477053 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.481648 28157 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 20.
I20260709 13:14:42.473824  4021 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:14:42.497773  4059 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.411s	user 0.003s	sys 0.000s
I20260709 13:14:42.483296  3927 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:42.500113 28226 raft_consensus.cc:1240] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Rejecting Update request from peer 611ec64ec58244fe82058d6130e4e6f3 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:14:42.503355  3945 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:42.520182 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.523065 28127 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:42.526186 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:42.526467 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:42.473929  3886 raft_consensus.cc:515] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:42.526996 28156 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: []
I20260709 13:14:42.527153 28152 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:42.534086  3945 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:42.535372  4052 log.cc:927] Time spent T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.372s	user 0.002s	sys 0.000s
I20260709 13:14:42.483397  3936 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.033s
I20260709 13:14:42.550877  3886 leader_election.cc:290] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:42.552218 28124 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:42.541862  4077 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.201s	user 0.001s	sys 0.001s
W20260709 13:14:42.540794  4050 log.cc:927] Time spent T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.445s	user 0.000s	sys 0.003s
W20260709 13:14:42.552492  4078 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:14:42.560760 26959 raft_consensus.cc:2272] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.564242 26959 tablet_replica.cc:333] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:42.564877 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.565608 28232 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:42.566255  3882 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:42.566979  3882 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.567488  3882 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:42.569900 28229 raft_consensus.cc:2410] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 13:14:42.575786 28234 pending_rounds.cc:85] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 16
I20260709 13:14:42.576439 28234 pending_rounds.cc:107] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 13:14:42.576906 28234 raft_consensus.cc:2889] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:42.577376 26959 raft_consensus.cc:2243] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 LEADER]: Raft consensus shutting down.
W20260709 13:14:42.578470 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45830.2us
W20260709 13:14:42.581851 28199 leader_election.cc:336] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:42.582064 28125 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.582350 28199 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:42.583519 28755 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 13:14:42.584107 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:42.586498 28157 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:14:42.587708 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:42.598606 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:42.591075 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b72fb1e20942d9a3907c50252930bb" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:42.599575 28758 raft_consensus.cc:2410] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:14:42.592113  3969 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:42.593515 28235 raft_consensus.cc:1275] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:14:42.600812 28235 pending_rounds.cc:85] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 14
I20260709 13:14:42.601095 28235 pending_rounds.cc:107] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:14:42.601351 28235 raft_consensus.cc:2889] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:42.594266 26959 pending_rounds.cc:70] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Trying to abort 1 pending ops.
I20260709 13:14:42.594369 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:42.589937  3945 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:42.597719 28125 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.630488 26959 pending_rounds.cc:77] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting op as it isn't in flight: id { term: 28 index: 21 } timestamp: 7305637403750469632 op_type: NO_OP noop_request { }
I20260709 13:14:42.605175 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:42.604054 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:42.601363 28728 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:42.633284 28125 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.634227 26959 raft_consensus.cc:2889] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:14:42.649554 26959 raft_consensus.cc:2272] T e09b3a74da19442187d6c447c95018a9 P 0faaace2b36c4bcebf2fa6835235e92b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.636070 28226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.662556 28160 raft_consensus.cc:2393] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f in current term 24: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:42.666898 28226 raft_consensus.cc:2410] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 13:14:42.674270 28049 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 77099.1us
I20260709 13:14:42.589342  3886 raft_consensus.cc:2804] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:14:42.678457 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.679766 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:42.682873 28230 raft_consensus.cc:1240] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 20. Current term is 21. Ops: []
I20260709 13:14:42.683837 28198 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
W20260709 13:14:42.684630 28125 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:42.686981 28125 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:42.688264 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 43939.7us
W20260709 13:14:42.673661 28050 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 199802us
I20260709 13:14:42.701932 28050 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:42.706977 28127 leader_election.cc:304] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:42.669252  3950 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:42.727077 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:42.729362 28728 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:42.676530  4044 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:42.733011 28050 raft_consensus.cc:2686] T 798c63c7bb2e4373af20f55bbe872cf2 P 0faaace2b36c4bcebf2fa6835235e92b: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:14:42.738687 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 49430.2us
W20260709 13:14:42.734943 28728 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:42.735383 28127 leader_election.cc:304] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:42.645004  4056 consensus_queue.cc:1059] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:14:42.754118 28127 leader_election.cc:304] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:42.709059  3886 raft_consensus.cc:697] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:42.764930 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c6cdf8c6704ee59d0a736273582a48" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:42.728639  3950 raft_consensus.cc:493] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.762291 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:42.733487  3994 raft_consensus.cc:2749] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:42.747893  4003 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:42.747476  4017 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:42.769852  3875 raft_consensus.cc:2749] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:14:42.753155  4044 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:42.826586 28125 leader_election.cc:336] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:42.827108 28125 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:42.780999  3950 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:42.817689 26959 tablet_replica.cc:333] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:42.853855 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 91685.4us
W20260709 13:14:42.807117  4088 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:14:42.876276 26959 raft_consensus.cc:2243] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:42.801926  3882 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:42.857950  4084 log.cc:927] Time spent T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.002s
I20260709 13:14:42.809283  3969 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:14:42.862658  4095 log.cc:927] Time spent T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.053s	user 0.003s	sys 0.000s
I20260709 13:14:42.887256 26959 raft_consensus.cc:2272] T 1115d5b02e774b14935e3f48488f82ea P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:42.862288  3875 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:42.866429  3836 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:14:42.879413  4076 log.cc:927] Time spent T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:14:42.803000  4017 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:42.935354 28126 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:42.937342 28728 leader_election.cc:336] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:42.938948 28728 consensus_peers.cc:597] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:42.943219 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:42.925058  3836 raft_consensus.cc:493] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:42.967623 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.968459 28232 raft_consensus.cc:2468] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 24.
I20260709 13:14:42.938542  4070 raft_consensus.cc:2749] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:42.991595  3945 raft_consensus.cc:2749] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:42.993549  4092 log.cc:927] Time spent T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:14:42.996313  4017 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:42.997284  4083 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 13:14:42.999899 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:43.005484  4087 log.cc:927] Time spent T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:14:42.925333  3886 consensus_queue.cc:237] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:43.020928  4045 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.021512  4045 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:43.024397  4045 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:42.960091  3836 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:14:43.040134 28199 leader_election.cc:336] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:43.041214 26959 tablet_replica.cc:333] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:43.043267 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 51018.7us
W20260709 13:14:43.077271 28199 leader_election.cc:336] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:43.049818  4044 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:43.077716 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:43.079872 26959 raft_consensus.cc:2243] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 13:14:43.079267  4093 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: Append to log took a long time: real 0.152s	user 0.006s	sys 0.001s
I20260709 13:14:43.087064 26959 pending_rounds.cc:70] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Trying to abort 1 pending ops.
I20260709 13:14:43.087574 26959 pending_rounds.cc:77] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting op as it isn't in flight: id { term: 18 index: 12 } timestamp: 7305637343010553856 op_type: NO_OP noop_request { }
I20260709 13:14:43.088330 26959 raft_consensus.cc:2889] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:14:43.088816 26959 raft_consensus.cc:2272] T b99bb21dbacb4d63839f1403ef9b08b9 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:43.085886 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 211150us
W20260709 13:14:43.087230 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:43.089457  3945 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:14:43.111560 26959 tablet_replica.cc:333] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:43.099730  4103 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:14:43.113410  4011 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.114787  4045 consensus_queue.cc:1059] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:14:43.116343 28730 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:43.117749 28199 leader_election.cc:336] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:43.122946  4045 raft_consensus.cc:3055] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:14:43.123553  4045 raft_consensus.cc:740] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:43.123281  4011 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:43.136845 28125 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:43.130646 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 376572us
I20260709 13:14:43.138183 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:43.132879 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:43.139315 28233 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 20.
I20260709 13:14:43.120545 28199 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
W20260709 13:14:43.141490 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 44926.3us
W20260709 13:14:43.142436 28199 consensus_peers.cc:597] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:43.143239 28199 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:43.147686 28127 leader_election.cc:304] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:43.148125  3945 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:43.151368 28127 leader_election.cc:304] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:43.167774 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:43.167948 28728 leader_election.cc:336] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:43.171569 26959 raft_consensus.cc:2243] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.147104  4103 raft_consensus.cc:2804] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:43.152686  4017 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:43.124722  4045 consensus_queue.cc:260] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:43.177728 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:43.177095 28125 leader_election.cc:336] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:43.181900 28125 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:43.152160  4056 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:43.155918  4021 raft_consensus.cc:2749] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:14:43.190845 28199 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:43.154081  4099 raft_consensus.cc:2749] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:43.202824 28227 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
I20260709 13:14:43.216365 28227 pending_rounds.cc:85] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Aborting all ops after (but not including) 13
I20260709 13:14:43.171360  3882 raft_consensus.cc:2749] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:43.200690  4023 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.216693 28227 pending_rounds.cc:107] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 13:14:43.223474 28227 raft_consensus.cc:2889] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:43.223979 28161 raft_consensus.cc:1240] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:14:43.203878 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:43.223580 26959 raft_consensus.cc:2272] T 8d3461aa17604e69b08928a9bc0f8d18 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.226933 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:43.235323 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:43.246376 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:43.248328  3994 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:43.238911  3882 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:43.250942  3945 raft_consensus.cc:2749] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:43.252503  4011 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:43.236810  4099 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:43.253896  4069 raft_consensus.cc:3055] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:14:43.257072 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:43.218726  3875 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:14:43.280556 28125 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:43.218590  4108 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:43.226056  4045 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:43.231700  4017 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:43.257637 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:43.206753  4056 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:43.283470 28125 leader_election.cc:336] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:43.269424  3882 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:43.287557 28125 consensus_peers.cc:597] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:43.289319 28052 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:43.290681 28125 leader_election.cc:336] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:43.228969  4023 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:43.265544  4038 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
W20260709 13:14:43.297043 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 129206us
W20260709 13:14:43.292325 28125 consensus_peers.cc:597] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:43.282568  4069 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:43.290470  4103 raft_consensus.cc:697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:43.302235 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 55967.6us
W20260709 13:14:43.302496 28125 consensus_peers.cc:597] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:43.291550  3882 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:43.304337 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 94828.8us
I20260709 13:14:43.300846  4091 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:14:43.303799 28127 leader_election.cc:304] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:43.300186  4103 consensus_queue.cc:237] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:43.309700  3994 raft_consensus.cc:2749] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:43.308045 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:43.288455  3950 raft_consensus.cc:515] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:43.311358 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:43.315898  4017 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:43.324638 28229 raft_consensus.cc:2393] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 24: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:43.339600 28125 consensus_peers.cc:597] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:43.342329 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:43.345009 28231 raft_consensus.cc:2410] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:14:43.342731 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:43.346623 28234 raft_consensus.cc:2393] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 25: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:43.354579 26959 tablet_replica.cc:333] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:43.355645 26959 raft_consensus.cc:2243] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:14:43.361661  4023 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:43.305708  3836 raft_consensus.cc:515] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:43.356623 26959 raft_consensus.cc:2272] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:43.378736 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 46829.1us
I20260709 13:14:43.383250 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:43.386094 28153 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:14:43.338819  3929 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:43.393837  4056 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:43.336244  4069 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:43.398655 28127 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:43.403265 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:43.403505  4063 consensus_queue.cc:1048] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:43.415974  4057 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:43.337147  4118 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:43.418550  4118 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:43.419648 28125 consensus_peers.cc:597] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:43.405579 28154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:43.421500 28154 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 26.
I20260709 13:14:43.422799 28154 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 13:14:43.424228 28125 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:43.427727 28127 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:43.406389  3929 raft_consensus.cc:493] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:43.485566 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.406818  3950 leader_election.cc:290] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:43.485011 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 67053.2us
W20260709 13:14:43.496043 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.430820  3815 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:43.454861  3875 raft_consensus.cc:2749] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:43.512526  4023 raft_consensus.cc:493] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.482182  4118 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:43.494330  3929 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:43.570775 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 63221.3us
I20260709 13:14:43.571900 26959 tablet_replica.cc:333] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:43.572947 26959 raft_consensus.cc:2243] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.573182  3836 leader_election.cc:290] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:43.573657 26959 raft_consensus.cc:2272] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.525249  4023 raft_consensus.cc:515] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:43.601902 26959 tablet_replica.cc:333] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:43.602612 26959 raft_consensus.cc:2243] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:43.617924 26959 pending_rounds.cc:70] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Trying to abort 1 pending ops.
I20260709 13:14:43.618330 26959 pending_rounds.cc:77] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting op as it isn't in flight: id { term: 22 index: 16 } timestamp: 7305637358357393408 op_type: NO_OP noop_request { }
I20260709 13:14:43.618775 26959 raft_consensus.cc:2889] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:14:43.619290 26959 raft_consensus.cc:2272] T 2929b1e07bb74c67863e4c504d8d09a6 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.622991 26959 tablet_replica.cc:333] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:43.629034  4023 leader_election.cc:290] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:43.638572  4126 log.cc:927] Time spent T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:14:43.497215  4021 consensus_queue.cc:1175] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "611ec64ec58244fe82058d6130e4e6f3" responder_term: 20 status { last_received { term: 17 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 11f10c12c03e47dcac88b050d4c1431b for earlier term 19. Current term is 20. Ops: [19.12-19.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:43.688359  3882 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:43.699834 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f16469a2e44408daf3969b51ae89aa6" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:14:43.727254 28125 consensus_peers.cc:597] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:43.695845 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:43.766158 28231 raft_consensus.cc:3060] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:43.800017 26959 raft_consensus.cc:2243] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:14:43.802353 28231 raft_consensus.cc:2468] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 23.
W20260709 13:14:43.806303 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 192336us
W20260709 13:14:43.838254 28050 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.846235 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "923fa739e01e4f05a05099628c53a6c3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:43.862450 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 243787us
W20260709 13:14:43.865114 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 477529us
W20260709 13:14:43.878676  4125 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:14:43.882084 28049 leader_election.cc:304] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:43.884331 28728 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.889372 26959 raft_consensus.cc:2272] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:43.890203  4027 raft_consensus.cc:2749] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:14:43.897118  4045 consensus_meta.cc:287] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.000s
I20260709 13:14:43.915488  3994 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.916097  3994 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:43.918706  3994 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:43.923373  3882 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:43.935364 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:43.938328  4023 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:14:43.938793  4023 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.939186  4023 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:43.938802  4069 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:43.952980 28125 consensus_peers.cc:597] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:43.954399  3815 raft_consensus.cc:2749] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:43.976285 28730 leader_election.cc:304] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:43.980413  3993 raft_consensus.cc:2804] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:43.980985  3993 raft_consensus.cc:697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:43.982435  3993 consensus_queue.cc:237] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:43.936187  3929 raft_consensus.cc:515] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:44.010620  3929 leader_election.cc:290] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:44.027834 28152 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 13:14:44.029559 28125 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:44.047235 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6810464df4a245f49df908fc877020d7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:44.048163 28232 raft_consensus.cc:3060] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:44.049947 28125 consensus_peers.cc:597] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:44.051445 28125 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:44.052640 28728 leader_election.cc:336] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:44.063373 28232 raft_consensus.cc:2468] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:44.049785  3967 raft_consensus.cc:493] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.083582  4023 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:44.086918 26959 tablet_replica.cc:333] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:44.087925 26959 raft_consensus.cc:2243] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:14:44.089355  4023 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:44.095345 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:44.106351 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:44.106395 26959 raft_consensus.cc:2272] T 8ada0581aa09483e91a300220a129859 P 0faaace2b36c4bcebf2fa6835235e92b [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:44.123984 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:44.140851 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:44.141562 28152 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:44.114691  4103 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:44.153205 28730 leader_election.cc:304] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:44.161119 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 295208us
I20260709 13:14:44.163709 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66540b865ca24721910c667ccd790b77" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:44.109861  3967 raft_consensus.cc:515] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:44.171237 28157 raft_consensus.cc:3060] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:44.138329  4045 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 13:14:44.191143 28757 raft_consensus.cc:1275] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:14:44.195380 28152 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0faaace2b36c4bcebf2fa6835235e92b in term 27.
I20260709 13:14:44.198362 28157 raft_consensus.cc:2468] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 24.
I20260709 13:14:44.200579 28127 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:44.180526  3929 raft_consensus.cc:2804] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:44.170755  4117 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:14:44.212327 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.183089  4027 raft_consensus.cc:493] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.215860 28152 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:44.208997  3945 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:44.225010  4027 raft_consensus.cc:515] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.225615  3945 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.227226  4027 leader_election.cc:290] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:44.227825  3945 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:44.236322  3967 leader_election.cc:290] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:44.211563  4113 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:44.238968  4142 log.cc:927] Time spent T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:14:44.238739  4113 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:44.246522  4147 log.cc:927] Time spent T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:14:44.246999 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:44.258128 28227 raft_consensus.cc:1275] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:14:44.211454  4108 raft_consensus.cc:2749] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:44.264950 28152 raft_consensus.cc:1240] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Rejecting Update request from peer 70996712c3c942468a6006602de6477f for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 13:14:44.211947  4056 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:44.267971 26959 tablet_replica.cc:333] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:44.211375  4115 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:44.271761 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:44.262444  4141 log.cc:927] Time spent T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:14:44.279187 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8d07cbc383b4089b326bc02e8b33622" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.279775 26959 raft_consensus.cc:2243] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:44.267443  3929 raft_consensus.cc:697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:44.281416 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.284845 28050 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 70996712c3c942468a6006602de6477f
I20260709 13:14:44.213816  4003 consensus_queue.cc:1048] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.064s
I20260709 13:14:44.288347 28050 leader_election.cc:304] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:44.281090  4069 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:44.290308  4069 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.279194  4056 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.292382  4069 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:44.292785  4056 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:44.294735 26959 pending_rounds.cc:70] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Trying to abort 1 pending ops.
I20260709 13:14:44.295083 26959 pending_rounds.cc:77] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting op as it isn't in flight: id { term: 26 index: 19 } timestamp: 7305637343012274176 op_type: NO_OP noop_request { }
I20260709 13:14:44.295492 26959 raft_consensus.cc:2889] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:14:44.295873 26959 raft_consensus.cc:2272] T e53fd092d3dd48c492e04080969cb666 P 0faaace2b36c4bcebf2fa6835235e92b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:44.299448 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:44.288827  4045 consensus_queue.cc:1059] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.014s
I20260709 13:14:44.301450  3836 consensus_queue.cc:1048] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 13:14:44.287573  4113 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:44.297895  4143 log.cc:927] Time spent T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:14:44.288975  4023 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:14:44.296752  3967 raft_consensus.cc:2749] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:14:44.314436 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:44.310149 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64288.1us
I20260709 13:14:44.281826  4115 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.318250 28230 raft_consensus.cc:2410] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:14:44.320930 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17fbe81a71fd4f8c83a65145cf2f0682" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:44.321725 28233 raft_consensus.cc:3060] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:44.323257 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:44.324843 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.326751 28160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.328027 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef726aecabb47089dcd217bfb8bfefc" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.329363 28199 leader_election.cc:304] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:44.331163 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:44.333119 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8addb79d6158437f8d7a4822a75661ab" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.332986  4070 raft_consensus.cc:2804] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:44.339139  4118 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:14:44.341743 28052 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 194212us
I20260709 13:14:44.342206 28127 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:44.334367 28199 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:44.346654 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e12858e91dd456a9f422bca6d2d6ac3" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:44.316684  4063 raft_consensus.cc:3055] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 26 LEADER]: Stepping down as leader of term 26
W20260709 13:14:44.368789 28199 leader_election.cc:336] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:44.377518 28233 raft_consensus.cc:2468] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:44.348315  4069 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:44.332351  4023 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 LEADER]: Becoming Leader. State: Replica: 0faaace2b36c4bcebf2fa6835235e92b, State: Running, Role: LEADER
I20260709 13:14:44.382433 28127 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:44.384018 28052 leader_election.cc:304] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:44.381506 28728 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:44.389659  4146 log.cc:927] Time spent T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:14:44.389896  4149 log.cc:927] Time spent T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:14:44.334313  3929 consensus_queue.cc:237] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:44.388484 28728 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:44.350427  4070 raft_consensus.cc:697] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 24 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:44.403762  4070 consensus_queue.cc:237] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.378942  3882 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:44.352876  4063 raft_consensus.cc:740] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
I20260709 13:14:44.377007  4113 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:44.386204  4069 raft_consensus.cc:493] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:44.429930  4061 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.409010  3969 raft_consensus.cc:2804] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:44.451773  4061 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:44.423139  4038 raft_consensus.cc:2749] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:44.423815  4027 raft_consensus.cc:493] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.462976  4061 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:44.464958  4027 raft_consensus.cc:515] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:44.467109  4027 leader_election.cc:290] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:44.469048 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:44.474581 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:44.475262 28232 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:44.447249  4069 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:44.477555 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:44.480934 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 96951.5us
I20260709 13:14:44.458845  3969 raft_consensus.cc:697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:44.497306  3969 consensus_queue.cc:237] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:44.490947 28728 leader_election.cc:336] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:44.490669  4043 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:44.483103  4023 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:44.501979  4043 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.502398  4043 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:44.488567  4063 consensus_queue.cc:260] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:44.509929  4063 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:44.527832  3919 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.533967  4043 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:44.540807  3919 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:44.545125  3919 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:44.545437  4069 raft_consensus.cc:515] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.582089 26959 tablet_replica.cc:333] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:44.603066 26959 raft_consensus.cc:2243] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:44.604751 26959 raft_consensus.cc:2272] T 923fa739e01e4f05a05099628c53a6c3 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:44.637885  3967 raft_consensus.cc:493] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.639294  3967 raft_consensus.cc:515] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:44.694474  4056 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.710529  4043 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:44.727283 26959 tablet_replica.cc:333] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:44.728658 26959 raft_consensus.cc:2243] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:44.728463 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd7077e3c7d94184913764cd90b92757" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:44.729713 26959 raft_consensus.cc:2272] T 847d15cbd80f44b18e547291eed25617 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:44.731590 28052 leader_election.cc:304] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:44.735394  4151 raft_consensus.cc:2749] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:14:44.753651  3945 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.754495  3945 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.712760  4056 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.774626  3967 leader_election.cc:290] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:44.776191 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:44.789451 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:44.790647 28232 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:44.793444 26959 tablet_replica.cc:333] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:44.794947 26959 raft_consensus.cc:2243] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:14:44.795382  4115 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:44.796501  3945 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:44.797246 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 50562.8us
W20260709 13:14:44.798563 28728 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:44.799540  4069 leader_election.cc:290] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:44.806016  4056 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:44.827368 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.836875 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:44.842104 26959 raft_consensus.cc:2272] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:44.844614 28052 leader_election.cc:304] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:44.850003  3936 raft_consensus.cc:2749] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:14:44.874079 27968 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet b99bb21dbacb4d63839f1403ef9b08b9 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 70996712c3c942468a6006602de6477f (127.26.83.196:46855) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 8 but the committed config has opid_index of 13
I20260709 13:14:44.879086  3886 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:44.879966  3886 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.882479  3886 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:44.907819 28232 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
I20260709 13:14:44.911659 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.913007 28152 raft_consensus.cc:2410] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70996712c3c942468a6006602de6477f for term 29 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 20.
I20260709 13:14:44.913806 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.914717 28158 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 25.
I20260709 13:14:44.925310 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc1588fd3df04c2c8354baa741ddeca9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:44.926514 28158 raft_consensus.cc:3060] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:44.905803 27975 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 196) took 11414 ms (client timeout 15000 ms). Trace:
I20260709 13:14:44.937469 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61f69a3a4f9a410e9fc9d6e9c38de9f1" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.940644 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:44.904666  3886 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:44.953298 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:44.954702 28155 raft_consensus.cc:2393] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 20: Already voted for candidate 0faaace2b36c4bcebf2fa6835235e92b in this term.
I20260709 13:14:44.955977 28758 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 18. (term mismatch)
I20260709 13:14:44.956445 28758 pending_rounds.cc:85] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Aborting all ops after (but not including) 17
I20260709 13:14:44.956720 28758 pending_rounds.cc:107] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 13:14:44.964912 28152 raft_consensus.cc:3060] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:14:44.963369 28758 raft_consensus.cc:2889] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:44.969148 28230 raft_consensus.cc:1275] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:14:44.984355  3969 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:44.977566  3886 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:44.992650  3936 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.008s
I20260709 13:14:44.999605  3969 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:45.001220 28730 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:45.002048 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.013245  4061 consensus_queue.cc:1048] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:14:45.012487  4063 raft_consensus.cc:2804] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:14:45.012627 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:45.015316  4063 raft_consensus.cc:697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:45.015846 28728 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:45.016695  4063 consensus_queue.cc:237] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:45.018592  3886 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:45.019488  3969 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:45.002103 28125 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.022522 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:45.024258 28758 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 28.
I20260709 13:14:45.026578 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:45.027789  3969 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:14:45.028632  3969 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:45.031985  4161 log.cc:927] Time spent T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:14:45.032337 28235 raft_consensus.cc:1275] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:14:45.029311  3969 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:14:45.038617  4070 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:45.050913 28125 consensus_peers.cc:597] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:44.997417 28199 leader_election.cc:336] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.010511 26959 tablet_replica.cc:333] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:44.929064 27975 rpcz_store.cc:269] 0709 13:14:33.547723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.616097 (+ 68374us) service_pool.cc:224] Handling call
0709 13:14:44.873684 (+11257587us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:41.439932 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:41.534727 (+ 94795us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:41.534764 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:14:41.534782 (+    18us) tablet.cc:662] Decoding operations
0709 13:14:41.536355 (+  1573us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:41.536426 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:41.536461 (+    35us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:14:41.755043 (+218582us) tablet.cc:701] Row locks acquired
0709 13:14:41.755064 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:14:41.769180 (+ 14116us) write_op.cc:270] Start()
0709 13:14:41.769389 (+   209us) write_op.cc:276] Timestamp: P: 1783602881769121 usec, L: 0
0709 13:14:41.769410 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:14:41.770091 (+   681us) log.cc:844] Serialized 7940 byte log entry
0709 13:14:42.088173 (+318082us) op_driver.cc:464] REPLICATION: finished
0709 13:14:42.428696 (+340523us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59cb56d0ac 00007f59d43c7ab9 00007f59cb58969b 00007f59cb58793f 00007f59cb589c02 00007f59d20adef0
0709 13:14:42.763594 (+334898us) write_op.cc:301] APPLY: starting
0709 13:14:42.772022 (+  8428us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:43.103504 (+331482us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:43.103543 (+    39us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:43.768927 (+665384us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:43.836064 (+ 67137us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:14:43.840151 (+  4087us) write_op.cc:312] APPLY: finished
0709 13:14:43.907515 (+ 67364us) log.cc:844] Serialized 180 byte log entry
0709 13:14:44.203153 (+295638us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:44.231250 (+ 28097us) write_op.cc:454] Released schema lock
0709 13:14:44.296215 (+ 64965us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:44.348142 (+ 51927us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:44.349856 (+  1714us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":118662,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":634276,"apply.run_cpu_time_us":77444,"apply.run_wall_time_us":1569390,"cfile_cache_hit":82,"cfile_cache_hit_bytes":88007,"num_ops":20,"prepare.queue_time_us":417977,"prepare.run_cpu_time_us":16542,"prepare.run_wall_time_us":489998,"raft.queue_time_us":25344,"raft.run_cpu_time_us":31333,"raft.run_wall_time_us":485341,"replication_time_us":318538,"spinlock_wait_cycles":279595264,"thread_start_us":983392,"threads_started":5,"wal-append.queue_time_us":533586,"wal-append.run_cpu_time_us":12778,"wal-append.run_wall_time_us":1140053}]]}
I20260709 13:14:45.071054 28158 raft_consensus.cc:2468] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
W20260709 13:14:45.028986 28728 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:45.077884 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 48965.4us
W20260709 13:14:45.068535 28125 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.093439 28152 raft_consensus.cc:2468] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
W20260709 13:14:45.093955 28125 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.090770  4070 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.098433  4070 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:45.072296  3919 consensus_queue.cc:1048] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 13:14:45.109521 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c654c99561449cb823642a1b85879ac" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:45.111937 28152 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Refusing update from remote peer 0faaace2b36c4bcebf2fa6835235e92b: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:14:45.117690 26959 raft_consensus.cc:2243] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:14:45.119086 26959 raft_consensus.cc:2272] T 34353a7e5a8346119cf9f0836165a1b7 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:45.121379 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 187216us
I20260709 13:14:45.122530  3936 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:14:45.122824 28199 leader_election.cc:336] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.125037 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:45.125895  4070 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:45.126576 28199 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.146565 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:45.175173 28728 consensus_peers.cc:597] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:45.175659 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 85365.9us
I20260709 13:14:45.176600  3969 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:45.193346  4070 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:45.185359 26959 tablet_replica.cc:333] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:45.191028 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 64499.8us
W20260709 13:14:45.203168 28199 leader_election.cc:336] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.203678 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:45.206173 26959 raft_consensus.cc:2243] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:14:45.207873 26959 raft_consensus.cc:2272] T f703ac7a390a45efae315d629c1dad89 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:45.203940 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:45.214145  4070 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:45.218918 26959 tablet_replica.cc:333] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:45.233608 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:45.234470 28158 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:45.235458  3919 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:14:45.220304 26959 raft_consensus.cc:2243] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 LEADER]: Raft consensus shutting down.
W20260709 13:14:45.253270 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50549.5us
I20260709 13:14:45.254422  4113 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.255090  4113 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.277168 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:45.281339 28230 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 24.
I20260709 13:14:45.291266 26959 raft_consensus.cc:2272] T 6a669edb54e3494387ede68c8acd8422 P 0faaace2b36c4bcebf2fa6835235e92b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:45.289258 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:45.282853  4113 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:45.297631 28199 leader_election.cc:304] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:45.303012 28228 raft_consensus.cc:1275] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:14:45.305361 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:45.311082  4069 raft_consensus.cc:2804] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:45.311919  4070 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:45.316639  4070 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.298659  4056 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:14:45.319651  4056 raft_consensus.cc:493] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.320102  4056 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:45.324928  4070 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:45.314916 28125 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.329563 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:45.330287 28229 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:45.331688  3969 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:45.330417  4070 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:45.336486  4070 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.351579 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:45.352409 28158 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:45.353955 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:45.354769 28152 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 25.
I20260709 13:14:45.356537 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:45.357645  4070 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:45.359032  3886 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:45.359490  3886 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:45.359834  3886 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:45.361608  3815 consensus_queue.cc:1048] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:45.363131 26959 tablet_replica.cc:333] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:45.364236 26959 raft_consensus.cc:2243] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:14:45.373905 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:14:45.378901 28199 leader_election.cc:336] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.385767 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e09b3a74da19442187d6c447c95018a9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:45.386498 28758 raft_consensus.cc:3060] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:14:45.387400 26959 raft_consensus.cc:2272] T 7e272db0632a4262a90eaacd6e6d4378 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:45.376580  3969 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:45.379431 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:45.395130  3969 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:45.401098  3969 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:45.402901 28728 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:45.400558 26959 tablet_replica.cc:333] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:45.409267 26959 raft_consensus.cc:2243] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:45.410104 26959 raft_consensus.cc:2272] T 5c654c99561449cb823642a1b85879ac P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:45.422559 28728 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:45.421030  4113 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:45.429371 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 51239.9us
W20260709 13:14:45.430469 28199 leader_election.cc:336] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.433125  3886 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.437506  3969 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.441577 28758 raft_consensus.cc:2468] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 29.
W20260709 13:14:45.446951 28199 leader_election.cc:336] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:45.450007 28199 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.450471 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:45.456166  3886 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:45.461488  4171 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:45.461988 26959 tablet_replica.cc:333] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:45.481594 26959 raft_consensus.cc:2243] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:14:45.453312  4113 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:45.482885 26959 raft_consensus.cc:2272] T d65f83feb5bb4b468e71fc5b9b708de4 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:45.459052 28124 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:45.485842  3815 raft_consensus.cc:2804] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:14:45.486358  3815 raft_consensus.cc:697] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:14:45.486954 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:45.487296  3815 consensus_queue.cc:237] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:45.465976 28125 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.483075  4113 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.548381  4113 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:45.476584 27976 catalog_manager.cc:5697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 17 to 19, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 19 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:45.528236 28728 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:45.556903 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 90974.8us
I20260709 13:14:45.516072  3969 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:45.532322 27978 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 262) took 87858 ms (client timeout 15000 ms). Trace:
I20260709 13:14:45.429991  4069 raft_consensus.cc:697] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
W20260709 13:14:45.563446 28728 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:45.516191 26959 tablet_replica.cc:333] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:45.518839  4056 raft_consensus.cc:515] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.566398  4056 leader_election.cc:290] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:45.544232 28152 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:14:45.567306 26959 raft_consensus.cc:2243] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:45.562985  4069 consensus_queue.cc:237] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.538007  4070 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.545495 28235 raft_consensus.cc:1275] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:14:45.563889 27978 rpcz_store.cc:269] 0709 13:13:17.673658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.745781 (+ 72123us) service_pool.cc:224] Handling call
0709 13:14:45.532062 (+87786281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1739133,"reported_tablets":48}
I20260709 13:14:45.581470  4070 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:45.585268 28158 raft_consensus.cc:1275] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:14:45.593003 27976 catalog_manager.cc:5697] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 16 to 17, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 17 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:45.585466 26959 pending_rounds.cc:70] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Trying to abort 1 pending ops.
I20260709 13:14:45.586645  3825 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:14:45.581614 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:45.596428 26959 pending_rounds.cc:77] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting op as it isn't in flight: id { term: 22 index: 13 } timestamp: 7305637334532562944 op_type: NO_OP noop_request { }
I20260709 13:14:45.597007 26959 raft_consensus.cc:2889] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:14:45.597144 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:45.597498 26959 raft_consensus.cc:2272] T 9e12858e91dd456a9f422bca6d2d6ac3 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:45.587841 28125 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.598026  4056 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:45.598495  4056 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:45.598827  4171 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:45.598956  4056 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:45.599665  4171 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:14:45.600041  4171 raft_consensus.cc:493] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.600354  4171 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:45.597574  3919 consensus_queue.cc:1048] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.012s
I20260709 13:14:45.610456 26959 tablet_replica.cc:333] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:45.611531 26959 raft_consensus.cc:2243] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:14:45.612996 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:45.617448 28228 raft_consensus.cc:3060] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:45.622202  4056 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:45.624109 28199 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.614919 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:45.625176 28234 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 22: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:45.625313 26959 raft_consensus.cc:2272] T a8381961442846ab97965dc11b100d75 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:45.633000 26959 tablet_replica.cc:333] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:45.634001 26959 raft_consensus.cc:2243] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 LEADER]: Raft consensus shutting down.
W20260709 13:14:45.634032 28199 consensus_peers.cc:597] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:45.635139 26959 raft_consensus.cc:2272] T e6b1c27f68974f4a9878c7c155cb43d2 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:45.635242 28199 leader_election.cc:336] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:45.634522 28125 consensus_peers.cc:597] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:45.637270 28228 raft_consensus.cc:2468] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 25.
W20260709 13:14:45.649104 28199 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.652575 26959 tablet_replica.cc:333] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:45.653961 26959 raft_consensus.cc:2243] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:45.658628  3886 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:45.659408 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb515ffdfa04f2f977a4c5aa1aecbee" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:45.659440 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:45.659360  3886 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.660315 28155 raft_consensus.cc:3060] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:14:45.662163 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "888e64754b8d47aebb2c7c3aba7b0ee2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:45.662788  3886 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:45.661386 28152 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 18.
I20260709 13:14:45.662892 28156 raft_consensus.cc:3060] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:45.671654  3936 raft_consensus.cc:493] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.672503  3936 raft_consensus.cc:515] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:45.675588  3936 leader_election.cc:290] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:45.677019 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:45.681584  4056 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:45.684592 28156 raft_consensus.cc:2468] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:45.685369 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d952e9d07d9e4c5ba47d0f8a9601dc73" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:45.687381 28052 leader_election.cc:304] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:45.689049 26959 raft_consensus.cc:2272] T 8addb79d6158437f8d7a4822a75661ab P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:45.690404  3936 raft_consensus.cc:2749] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:45.704079 28728 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:45.705368 28125 consensus_peers.cc:597] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:45.711766 26959 tablet_replica.cc:333] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:45.716516 28125 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.722422 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:45.723775 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:45.724123 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:45.724447 28228 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 18.
I20260709 13:14:45.724861 28156 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 22: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:45.725553  4056 consensus_queue.cc:1048] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:45.716516 26959 raft_consensus.cc:2243] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:14:45.730118 26959 raft_consensus.cc:2272] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:45.719691 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:45.735689  3886 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:45.737872 28127 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:45.738902  4070 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:45.739393  4070 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:45.739753  4070 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:45.724665  4171 raft_consensus.cc:515] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.743153  4171 leader_election.cc:290] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:45.744458  3936 raft_consensus.cc:493] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.745064  3936 raft_consensus.cc:515] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:45.747287  3936 leader_election.cc:290] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:45.748116 28199 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:45.748955  3886 raft_consensus.cc:2804] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:45.749562  3886 raft_consensus.cc:697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:45.750490  3886 consensus_queue.cc:237] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.751015 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:45.752353 28229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b9a6ec9a3a1422b8723b26150db560f" candidate_uuid: "0faaace2b36c4bcebf2fa6835235e92b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:45.732546 28155 raft_consensus.cc:2468] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 27.
I20260709 13:14:45.759534 28049 leader_election.cc:304] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0faaace2b36c4bcebf2fa6835235e92b; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:45.761282  3936 raft_consensus.cc:2749] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:45.762454 28125 consensus_peers.cc:597] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:45.770283  4070 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.772455  4070 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:45.767947 28199 leader_election.cc:304] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:45.758455  4171 raft_consensus.cc:2804] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:14:45.773167 28728 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:45.776782 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:45.784533 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:45.786397  4171 raft_consensus.cc:697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:45.789326 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fbb28a09242477abe442ecc8a938738" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:45.788076  4003 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:14:45.800112  4171 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.800973  4003 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:45.801640  4003 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:14:45.800137 28156 raft_consensus.cc:3060] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:14:45.796267 26959 tablet_replica.cc:333] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
W20260709 13:14:45.780931 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:45.786306 28199 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.813794 26959 raft_consensus.cc:2243] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:14:45.821628 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 45457.7us
W20260709 13:14:45.823851 28199 leader_election.cc:336] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:45.824805 28199 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.775738  4070 raft_consensus.cc:2804] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:14:45.831987 28156 raft_consensus.cc:2468] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 26.
I20260709 13:14:45.843657 28199 leader_election.cc:304] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:45.833352  4070 raft_consensus.cc:697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 27 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:45.880486 26959 raft_consensus.cc:2272] T 2facf6a0a91849a5a177c2481e42aa47 P 0faaace2b36c4bcebf2fa6835235e92b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:45.874624  4056 raft_consensus.cc:2804] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:14:45.892499 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 50998.8us
W20260709 13:14:45.906386 28125 consensus_peers.cc:597] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:45.911813  4003 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.915778  4173 raft_consensus.cc:493] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:45.917196  4070 consensus_queue.cc:237] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.904331  4056 raft_consensus.cc:697] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 26 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:45.926324  4173 raft_consensus.cc:515] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:45.919806 28199 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.930347 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:45.932843  4003 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:45.933553 28155 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 19: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:45.932869  4056 consensus_queue.cc:237] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.934670  4173 leader_election.cc:290] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:45.937922  4171 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:14:45.942183 26959 tablet_replica.cc:333] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:45.946056 26959 raft_consensus.cc:2243] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:45.952710 28757 raft_consensus.cc:1275] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
W20260709 13:14:45.942224 28125 leader_election.cc:336] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.946779 26959 raft_consensus.cc:2272] T 6b9a6ec9a3a1422b8723b26150db560f P 0faaace2b36c4bcebf2fa6835235e92b [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:45.958002 28125 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.966228 28757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:45.973174 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:45.974462 28228 raft_consensus.cc:2393] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 19: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:45.966356 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99bb21dbacb4d63839f1403ef9b08b9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:45.964661 28199 leader_election.cc:336] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:45.983911 28199 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:45.985296 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:45.979907  4003 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:45.997253  3969 consensus_queue.cc:1048] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 13:14:45.997277  4003 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:45.999830  4003 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:46.002317 28125 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 228 similar messages]
I20260709 13:14:46.013202  4171 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:14:46.023837 28125 leader_election.cc:336] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.004931 28124 leader_election.cc:304] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 70996712c3c942468a6006602de6477f
I20260709 13:14:46.028659 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:46.032763 28127 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
W20260709 13:14:46.036361 28125 consensus_peers.cc:597] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:46.057154  4108 raft_consensus.cc:2749] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:46.062279 26959 tablet_replica.cc:333] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:46.063673 26959 raft_consensus.cc:2243] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Raft consensus shutting down.
W20260709 13:14:46.068149 28125 consensus_peers.cc:597] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:46.069190 28125 consensus_peers.cc:597] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:46.047307  3969 raft_consensus.cc:2749] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:14:46.077080 26959 pending_rounds.cc:70] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Trying to abort 1 pending ops.
I20260709 13:14:46.077463 26959 pending_rounds.cc:77] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b: Aborting op as it isn't in flight: id { term: 26 index: 19 } timestamp: 7305637343439007744 op_type: NO_OP noop_request { }
I20260709 13:14:46.077831 26959 raft_consensus.cc:2889] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:14:46.078253 26959 raft_consensus.cc:2272] T d8d07cbc383b4089b326bc02e8b33622 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:46.097090 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 114445us
I20260709 13:14:46.108032 26959 tablet_replica.cc:333] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:46.120669 26959 raft_consensus.cc:2243] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:14:46.127627 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 95009.3us
I20260709 13:14:46.130321 28127 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:46.136070  4108 raft_consensus.cc:2749] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:46.141770 28125 consensus_peers.cc:597] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:46.145285 26959 raft_consensus.cc:2272] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:46.154557 26959 tablet_replica.cc:333] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:46.155424 26959 raft_consensus.cc:2243] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:46.156164 26959 raft_consensus.cc:2272] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 0faaace2b36c4bcebf2fa6835235e92b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:46.159369 26959 tablet_replica.cc:333] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b: stopping tablet replica
I20260709 13:14:46.166729 26959 raft_consensus.cc:2243] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:46.167392 26959 raft_consensus.cc:2272] T dd7077e3c7d94184913764cd90b92757 P 0faaace2b36c4bcebf2fa6835235e92b [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:46.175428 28199 consensus_peers.cc:597] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:46.184193 28156 raft_consensus.cc:1275] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:14:46.190486  4070 consensus_queue.cc:1048] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:14:46.206682 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:46.215071 27976 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58554 (request call id 286) took 52049 ms (client timeout 15000 ms). Trace:
W20260709 13:14:46.215680 27976 rpcz_store.cc:269] 0709 13:13:54.165367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.165680 (+   313us) service_pool.cc:224] Handling call
0709 13:14:46.214979 (+52049299us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:45.833836 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:45.885193 (+ 51357us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:45.885264 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:14:45.885287 (+    23us) tablet.cc:662] Decoding operations
0709 13:14:45.885795 (+   508us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:45.885895 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:45.885932 (+    37us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:14:45.886395 (+   463us) tablet.cc:701] Row locks acquired
0709 13:14:45.886437 (+    42us) write_op.cc:260] PREPARE: finished
0709 13:14:45.886629 (+   192us) write_op.cc:270] Start()
0709 13:14:45.886798 (+   169us) write_op.cc:276] Timestamp: P: 1783602885886564 usec, L: 0
0709 13:14:45.886840 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:14:45.887692 (+   852us) log.cc:844] Serialized 929 byte log entry
0709 13:14:46.091331 (+203639us) op_driver.cc:464] REPLICATION: finished
0709 13:14:46.165145 (+ 73814us) write_op.cc:301] APPLY: starting
0709 13:14:46.165258 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:46.167554 (+  2296us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:46.167670 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:46.170643 (+  2973us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:46.170945 (+   302us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:14:46.170990 (+    45us) write_op.cc:312] APPLY: finished
0709 13:14:46.171262 (+   272us) log.cc:844] Serialized 33 byte log entry
0709 13:14:46.173801 (+  2539us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:46.173908 (+   107us) write_op.cc:454] Released schema lock
0709 13:14:46.174081 (+   173us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:46.177182 (+  3101us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:46.177214 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3070,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":73474,"apply.run_cpu_time_us":5585,"apply.run_wall_time_us":12001,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12515,"num_ops":2,"prepare.queue_time_us":152260,"prepare.run_cpu_time_us":28650,"prepare.run_wall_time_us":213969,"raft.queue_time_us":33738,"raft.run_cpu_time_us":5012,"raft.run_wall_time_us":40928,"replication_time_us":204222,"spinlock_wait_cycles":3328,"thread_start_us":260131,"threads_started":4,"wal-append.queue_time_us":154577}]]}
I20260709 13:14:46.237334  4070 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.237978  4070 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.240243  4070 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:46.242786 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:46.243455 28152 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:46.244707 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:46.246115  4070 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:46.246500  4070 raft_consensus.cc:493] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.246887  4070 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:46.265565 28228 raft_consensus.cc:1275] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:14:46.269320 28199 leader_election.cc:336] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.274682  4192 consensus_queue.cc:1048] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:46.278995  4070 raft_consensus.cc:515] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:46.309492 28125 consensus_peers.cc:597] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:46.312176  4070 leader_election.cc:290] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:46.314597 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c71e4401306a4e84a111e5443be7652c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:46.315289 28152 raft_consensus.cc:3060] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:14:46.320235 27979 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 263) took 63765 ms (client timeout 15000 ms). Trace:
I20260709 13:14:46.324687 28152 raft_consensus.cc:2468] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:46.312000 27975 catalog_manager.cc:5697] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 28 to 29, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 29 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.328660 28199 leader_election.cc:304] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:46.331488 28199 leader_election.cc:336] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:46.324914 27979 rpcz_store.cc:269] 0709 13:13:42.555317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.555780 (+   463us) service_pool.cc:224] Handling call
0709 13:14:46.320007 (+63764227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4229,"reported_tablets":48}
I20260709 13:14:46.332544  4070 raft_consensus.cc:2804] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:46.332122 27975 catalog_manager.cc:5697] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 22 to 25, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 25 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:46.333385  4070 raft_consensus.cc:697] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:46.334410  4070 consensus_queue.cc:237] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.334462 27975 catalog_manager.cc:5697] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 20, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 20 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:46.337457 27975 catalog_manager.cc:5697] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 20 to 22, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 22 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.358122 27975 catalog_manager.cc:5697] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:46.367275 28728 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:46.359824 27975 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 23 to 27, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 27 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:46.385461 27975 catalog_manager.cc:5697] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.386837 27975 catalog_manager.cc:5697] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 18 to 21, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 21 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.388219 27975 catalog_manager.cc:5697] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 19 to 20, leader changed from 70996712c3c942468a6006602de6477f (127.26.83.196) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 20 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:46.394217 27975 catalog_manager.cc:5697] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.412283 28152 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:14:46.428325 28199 consensus_peers.cc:597] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:46.435091  4070 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:14:46.446549 28199 consensus_peers.cc:597] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:46.457512 28155 raft_consensus.cc:1275] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:14:46.459368  4056 consensus_queue.cc:1048] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:46.477929  4113 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.478541  4113 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.481379  4113 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:46.487280 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
W20260709 13:14:46.492053 28199 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.396170 27975 catalog_manager.cc:5697] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.493876 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:46.499923  4113 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:46.505982 28155 raft_consensus.cc:1275] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
W20260709 13:14:46.531925 28728 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:46.540496  4113 consensus_queue.cc:1048] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:46.497455 27975 catalog_manager.cc:5697] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 26 to 27, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 27 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.557973 27975 catalog_manager.cc:5697] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.570941  4061 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.579200  4061 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:46.581552  4061 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:46.582793 27975 catalog_manager.cc:5697] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 13 to 24. New cstate: current_term: 24 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.583793 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:46.586115 27975 catalog_manager.cc:5697] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 18 to 19, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 19 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:46.590296 27975 catalog_manager.cc:5697] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 20 to 21, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 21 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.596469 27975 catalog_manager.cc:5697] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:46.600066 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:46.600925 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.601354 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:46.601452 27975 catalog_manager.cc:5697] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.603402  4061 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:46.607822  4192 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.616411  4192 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.618695  4192 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:46.608474  4173 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.610028  4193 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.621624  4173 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.622287  4193 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.624053  4173 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:46.612887  4003 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.625025  4003 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:46.627466  4003 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:46.627959 27975 catalog_manager.cc:5697] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 22 to 23, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 23 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.629912 27975 catalog_manager.cc:5697] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 21 to 23, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 23 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.632802 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:46.633147 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:46.633576 28764 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
W20260709 13:14:46.634742 28125 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.635375 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:46.636133 28228 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 24.
I20260709 13:14:46.636752 28127 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:46.638621  4173 raft_consensus.cc:2749] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:46.639744  4193 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:46.640789 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:46.641847 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:46.642968  4173 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:46.643365  4173 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.643720  4173 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:46.644366 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:46.631599 27975 catalog_manager.cc:5697] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 8 to 13, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 18 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.647792  4193 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:46.648252  4193 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.648624  4193 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:46.653762 27975 catalog_manager.cc:5697] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.656920 27975 catalog_manager.cc:5697] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 16 to 22, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 22 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.654448  3969 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.660327  3969 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.664425 27975 catalog_manager.cc:5697] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:46.665185 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:46.662637  3969 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:46.680509 28125 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:46.682785 28125 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.684260  4069 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.685011  4069 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.687359  4069 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:46.691941 28199 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.683701 28125 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:46.695101  4003 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:46.696631 28125 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.699112 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:46.699779 28152 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 24.
I20260709 13:14:46.700877 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:46.702121  4069 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:46.702543  4069 raft_consensus.cc:493] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.702934  4069 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:46.705101  4056 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.705689  4056 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.707967  4056 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:46.721370 28199 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.730505  4056 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.731711  4056 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.733281  4193 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:46.735587  4193 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:46.737908 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f703ac7a390a45efae315d629c1dad89" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:46.738598 28764 raft_consensus.cc:3060] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:46.750931  4056 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:46.756769 28199 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.724538  4173 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.763458 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:46.765472 28152 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:46.759622 28764 raft_consensus.cc:2468] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
I20260709 13:14:46.769337 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:46.769999 28235 raft_consensus.cc:3060] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:46.771068  4069 raft_consensus.cc:515] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.773754 28124 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:46.777541 28125 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.778043 28125 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:46.781528 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:46.782652 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:46.782275 28152 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:46.776989  4193 raft_consensus.cc:2804] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:46.785985  4193 raft_consensus.cc:697] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:46.786895  4193 consensus_queue.cc:237] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:46.794571 28235 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 25.
I20260709 13:14:46.784013  4003 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:14:46.801561  4069 leader_election.cc:290] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:46.803220  4070 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:46.809176  4070 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.809676  4070 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:46.816298 28728 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:46.818861  4070 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.776136  4173 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:46.821135  4070 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:46.828230 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:46.826292 28127 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:46.837668  4003 raft_consensus.cc:2804] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:46.838264  4003 raft_consensus.cc:697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
W20260709 13:14:46.835650 28125 consensus_peers.cc:597] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:46.839197  4003 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.844246 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a669edb54e3494387ede68c8acd8422" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:46.841301  3969 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.844956 28152 raft_consensus.cc:3060] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:46.845315  3969 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:46.839480  4113 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:46.850239  4113 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.850831  4113 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:46.857851 28152 raft_consensus.cc:2468] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 25.
I20260709 13:14:46.837167  4173 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:46.875927 28155 raft_consensus.cc:1275] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:14:46.876292 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:46.878998 28152 raft_consensus.cc:3060] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:46.882213 28125 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.882925  3969 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:46.883806 28125 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.888880 28152 raft_consensus.cc:2468] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 24.
W20260709 13:14:46.901991 28125 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:46.903370 28199 leader_election.cc:304] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:46.905347 28199 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:46.923697  4070 raft_consensus.cc:2804] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:46.910380  4056 consensus_queue.cc:1048] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:46.902987  4173 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:46.992791  4171 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.002413  3969 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.996831  4171 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.022913  4171 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:47.025226 28125 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:47.033972  4113 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.003225  3969 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.037621  3969 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:47.065346  4113 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:47.020537  4069 raft_consensus.cc:2804] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:14:47.078828 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 69968.8us
W20260709 13:14:47.080969 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 175688us
I20260709 13:14:47.082278 28228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:47.082881 28228 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
W20260709 13:14:47.081073 28125 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:47.086349  4069 raft_consensus.cc:697] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 24 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:47.102680  4173 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:47.102809  4069 consensus_queue.cc:237] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.103493  4173 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.104022  4173 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.108333 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:47.109311 28235 raft_consensus.cc:2410] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:14:47.116832  4173 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:47.124516  4070 raft_consensus.cc:697] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 25 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:47.126387  4070 consensus_queue.cc:237] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:47.127472 28125 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:47.149302  4192 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.153362  4192 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.155980  4192 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:47.163162 28125 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:47.159090 28199 consensus_peers.cc:597] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:47.164781 28199 leader_election.cc:336] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:47.170214 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:47.192167  4069 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.192848 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:47.193141  4069 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.193532 28155 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:47.194821  4173 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:47.195182  4069 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:47.195212  4173 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.196003  4173 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:47.196449 28127 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:47.199649  4192 raft_consensus.cc:2749] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:47.205760 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e272db0632a4262a90eaacd6e6d4378" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:47.206408 28155 raft_consensus.cc:3060] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:47.218762 28155 raft_consensus.cc:2468] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:47.219525  4215 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.220160  4215 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.222406  4215 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:14:47.223603  4173 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.225920 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:47.229672 28764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:47.233209  4212 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:47.233206 28764 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 27.
I20260709 13:14:47.233834  4212 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.234268  4212 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:47.236508 28199 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:47.238471  4069 raft_consensus.cc:2804] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:14:47.239056  4069 raft_consensus.cc:697] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:47.240067  4069 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.241578 28757 raft_consensus.cc:1275] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:14:47.247643 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:47.248404 28235 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 19.
W20260709 13:14:47.254307 28199 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:47.256160 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:47.255429 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:47.259757 28234 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 23.
W20260709 13:14:47.287343 28199 consensus_peers.cc:597] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:47.289626 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:47.290308 28157 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
I20260709 13:14:47.317530 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:47.318257 28153 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
W20260709 13:14:47.341733 28125 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:47.281841  4173 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:47.349710 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 61659.4us
I20260709 13:14:47.309038  4210 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:14:47.356846 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:47.358461 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:47.360205 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:47.301477  3969 consensus_queue.cc:1048] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 13:14:47.384903  4222 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:47.385501  4222 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.385911  4222 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:47.379429  4069 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:47.388877  4069 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.389340  4069 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:47.354691  4210 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.398418  4210 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:47.400527  4069 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.403054  4069 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:14:47.366953  4171 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:14:47.418987 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 58875us
I20260709 13:14:47.449988 28234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
W20260709 13:14:47.449339 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 81471.8us
I20260709 13:14:47.451943 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:47.413654  4171 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:47.441174 28125 connection.cc:769] client connection to 127.26.83.193:45587 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:14:47.462498 28125 consensus_peers.cc:597] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: sendmsg error: Broken pipe (error 32). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:47.472271 28199 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:47.457681  4171 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:47.478999 28199 consensus_peers.cc:597] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:47.482848 28199 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:47.481793 28728 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:47.489710 28199 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:14:47.491470 28199 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:47.475311  4212 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.488092  4228 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:14:47.520054 28728 connection.cc:573] client connection to 127.26.83.193:45587 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:14:47.530417 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:47.538235 28152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:47.541954 28234 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:47.542490 28230 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:14:47.527132  4193 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.549556  4223 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.550304  4223 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.565436  4193 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.531934  4228 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.584060  4215 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
I20260709 13:14:47.584875  4215 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.585141  4210 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:47.595237  4069 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.596218  4212 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:47.596043  4069 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.579944  4228 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:47.611021  4193 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:47.617323  4210 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 28 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:47.602347  4223 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:47.606833  4222 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.609372  4069 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:47.609987  4215 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:47.643524  4171 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:47.645776 28125 consensus_peers.cc:597] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:47.648007  4222 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:47.677691 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:47.678894 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:47.681389 28156 raft_consensus.cc:2393] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:47.681906 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:47.682523 28155 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:47.690131 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:47.690809 28159 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 22.
I20260709 13:14:47.648111  4228 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.691838 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:47.694283 28758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6b1c27f68974f4a9878c7c155cb43d2" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:47.697896 28758 raft_consensus.cc:3060] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:14:47.685403  4227 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:47.684254  4193 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:14:47.720026 28758 raft_consensus.cc:2468] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 28.
I20260709 13:14:47.728768  4171 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:47.743055 28125 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:47.694957 27975 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:56104 (request call id 197) took 2288 ms. Trace:
W20260709 13:14:47.748098 28125 leader_election.cc:336] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:47.749897 28125 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:47.753470 28152 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 24: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:47.748047 27975 rpcz_store.cc:276] 0709 13:14:45.406588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.407118 (+   530us) service_pool.cc:224] Handling call
0709 13:14:47.694426 (+2287308us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:46.732668 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:46.733301 (+   633us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:46.733347 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:14:46.733367 (+    20us) tablet.cc:662] Decoding operations
0709 13:14:46.735139 (+  1772us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:46.735225 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:46.735253 (+    28us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:14:46.738853 (+  3600us) tablet.cc:701] Row locks acquired
0709 13:14:46.738875 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:14:46.739035 (+   160us) write_op.cc:270] Start()
0709 13:14:46.739171 (+   136us) write_op.cc:276] Timestamp: P: 1783602886738983 usec, L: 0
0709 13:14:46.739194 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:14:46.739985 (+   791us) log.cc:844] Serialized 9346 byte log entry
0709 13:14:46.808459 (+ 68474us) op_driver.cc:464] REPLICATION: finished
0709 13:14:47.019027 (+210568us) write_op.cc:301] APPLY: starting
0709 13:14:47.039919 (+ 20892us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:47.203721 (+163802us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:47.203856 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:47.467759 (+263903us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:47.468353 (+   594us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=48,mrs_lookups=0
0709 13:14:47.468437 (+    84us) write_op.cc:312] APPLY: finished
0709 13:14:47.513122 (+ 44685us) log.cc:844] Serialized 212 byte log entry
0709 13:14:47.530357 (+ 17235us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:47.542154 (+ 11797us) write_op.cc:454] Released schema lock
0709 13:14:47.560822 (+ 18668us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:47.561391 (+   569us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:47.561429 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":33354,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":194982,"apply.run_cpu_time_us":38617,"apply.run_wall_time_us":550200,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104783,"num_ops":24,"prepare.queue_time_us":59233,"prepare.run_cpu_time_us":6198,"prepare.run_wall_time_us":14621,"replication_time_us":69101,"thread_start_us":138617,"threads_started":2}]]}
W20260709 13:14:47.771873 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 80110.2us
I20260709 13:14:47.791899 28124 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:47.792271 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:47.803610 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:47.804431 28235 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 22.
I20260709 13:14:47.809834 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:47.794142  4069 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:47.812049 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:47.813088 28235 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 20: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:47.819118 28125 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:47.801499  3969 raft_consensus.cc:2804] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 13:14:47.826241 28199 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:47.827227  4229 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.858701  4229 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.830503  4228 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:47.846742 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:47.808203  4212 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:47.864094  4212 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.864528  4212 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:47.824842  4069 raft_consensus.cc:493] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0faaace2b36c4bcebf2fa6835235e92b)
W20260709 13:14:47.848583 28728 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:47.870177 28728 consensus_peers.cc:597] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:47.870252 28125 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:47.846232  3969 raft_consensus.cc:697] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:47.870745 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
W20260709 13:14:47.870798 28125 consensus_peers.cc:597] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:47.869148  4069 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:47.882318  4222 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:14:47.882823  4222 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.883181  4222 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:47.885397 28153 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:14:47.876029  4215 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:47.887678 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:47.888459 28157 raft_consensus.cc:2393] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 20: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
W20260709 13:14:47.889377 28199 consensus_peers.cc:597] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:47.890188 28199 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:47.891615 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:47.894064 28152 raft_consensus.cc:1275] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:14:47.895341 28233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:47.896123 28233 raft_consensus.cc:2393] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 24: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:47.896551  3969 consensus_queue.cc:237] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:47.900102 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:47.904263  4222 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.905382  4192 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:47.909106 28157 raft_consensus.cc:1275] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:14:47.925443  4215 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:47.927909  4229 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:47.929311  4215 consensus_queue.cc:1048] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:47.934165  4215 consensus_queue.cc:1048] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:14:47.950271 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:47.951205  3969 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:14:47.951767  4215 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.952493  4215 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:47.955207  4215 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:47.956075 28199 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:47.956663 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:47.957800 28199 leader_election.cc:336] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:47.958555 28199 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:47.958971 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:47.959808  4070 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:14:47.960551  4070 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:14:47.960809 27980 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58554 (request call id 285) took 78270 ms (client timeout 15000 ms). Trace:
W20260709 13:14:47.961244 27980 rpcz_store.cc:269] 0709 13:13:29.872315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.872572 (+   257us) service_pool.cc:224] Handling call
0709 13:14:47.910569 (+78037997us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13329,"reported_tablets":49}
I20260709 13:14:47.973206 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:47.973940 28153 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:47.973969 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:47.974758 28156 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 19.
W20260709 13:14:47.976948 28199 consensus_peers.cc:597] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:47.977556 28199 consensus_peers.cc:597] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:47.979470 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:47.987407  4229 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:47.987890  4229 raft_consensus.cc:493] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.988444  4229 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:47.998008 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:47.999714  4070 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:14:48.000151  4070 raft_consensus.cc:493] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.000478  4070 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:14:48.004369 28199 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:48.010438 28199 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:48.026901 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:47.988186  4222 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:48.041848  4229 raft_consensus.cc:515] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:48.047642 28157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "798c63c7bb2e4373af20f55bbe872cf2" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:48.048336 28157 raft_consensus.cc:3060] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:48.049273 27981 catalog_manager.cc:5697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 19 to 20, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 20 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:48.062388  4212 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:48.065294 28235 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 23: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
I20260709 13:14:48.072894  4212 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:48.073068 28156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:48.081707 28156 raft_consensus.cc:2393] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 11f10c12c03e47dcac88b050d4c1431b in current term 23: Already voted for candidate 611ec64ec58244fe82058d6130e4e6f3 in this term.
I20260709 13:14:48.044562  4229 leader_election.cc:290] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:48.086621 28125 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:48.090509 27981 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:48.091534 28125 consensus_peers.cc:597] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:48.090942 28199 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:48.092762 28199 leader_election.cc:336] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:48.098903 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:48.107146 28157 raft_consensus.cc:2468] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 21.
I20260709 13:14:48.108428 28199 leader_election.cc:304] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:48.103506  4212 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:14:48.118583  4171 raft_consensus.cc:2804] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:48.119251  4171 raft_consensus.cc:697] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:48.120250  4171 consensus_queue.cc:237] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:48.130914 28127 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:48.099788  4069 raft_consensus.cc:515] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:48.148752  4173 raft_consensus.cc:2749] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:14:48.172219 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:48.158926 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet b99bb21dbacb4d63839f1403ef9b08b9 with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:38034
W20260709 13:14:48.186918  4243 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.075s	user 0.003s	sys 0.000s
I20260709 13:14:48.196333 28230 raft_consensus.cc:2393] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 611ec64ec58244fe82058d6130e4e6f3 in current term 23: Already voted for candidate 11f10c12c03e47dcac88b050d4c1431b in this term.
W20260709 13:14:48.203377 28728 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:48.204095 28728 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:48.211907  4070 raft_consensus.cc:515] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:48.215391  4069 leader_election.cc:290] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:48.235265  4252 tablet_replica.cc:333] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:48.266458  4070 leader_election.cc:290] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:48.273983 28127 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:48.286871 28158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ada0581aa09483e91a300220a129859" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:48.286854 28155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c9b061a7ca84a61ac13d29946a28169" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:48.287602 28158 raft_consensus.cc:3060] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:14:48.281186  4252 raft_consensus.cc:2243] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:48.280980  3969 raft_consensus.cc:3060] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:14:48.307762 28158 raft_consensus.cc:2468] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 20.
I20260709 13:14:48.323271  4252 raft_consensus.cc:2272] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:48.327057  4252 ts_tablet_manager.cc:1916] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:14:48.343312  3969 raft_consensus.cc:2749] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:48.344875 28125 consensus_peers.cc:597] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:48.345520 28125 consensus_peers.cc:597] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:48.354153 28155 raft_consensus.cc:2468] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
I20260709 13:14:48.354750 28199 leader_election.cc:304] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:48.377496  4070 raft_consensus.cc:2804] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:14:48.378063  4070 raft_consensus.cc:697] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:48.379014  4070 consensus_queue.cc:237] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:48.377836 28199 leader_election.cc:336] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:48.385854 28199 leader_election.cc:336] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:48.407006 28199 leader_election.cc:304] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:48.425223  4252 ts_tablet_manager.cc:1929] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.12
I20260709 13:14:48.417667  4250 raft_consensus.cc:2804] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:14:48.432823 28728 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:48.434172 28728 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:48.438664  4252 log.cc:1199] T b99bb21dbacb4d63839f1403ef9b08b9 P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/b99bb21dbacb4d63839f1403ef9b08b9
I20260709 13:14:48.443542 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet b99bb21dbacb4d63839f1403ef9b08b9 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:14:48.448948  4250 raft_consensus.cc:697] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:48.450060  4250 consensus_queue.cc:237] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:48.464437 28728 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:48.525988 28125 consensus_peers.cc:597] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:48.544073 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 68667.1us
I20260709 13:14:48.569478 27980 catalog_manager.cc:5697] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 20 to 21, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 21 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:48.570652 28125 consensus_peers.cc:597] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:48.571264 27980 catalog_manager.cc:5697] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 26 to 29, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 29 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:48.573549 28199 consensus_peers.cc:597] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:48.609145 28758 raft_consensus.cc:1275] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 13:14:48.629025 28125 consensus_peers.cc:597] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:48.632879 28125 consensus_peers.cc:597] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:48.650722 27981 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 280) took 49912 ms (client timeout 15000 ms). Trace:
I20260709 13:14:48.652143  4222 consensus_queue.cc:1048] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:14:48.651398 27981 rpcz_store.cc:269] 0709 13:13:58.738447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.742600 (+  4153us) service_pool.cc:224] Handling call
0709 13:14:48.650631 (+49908031us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:48.293282 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:48.293873 (+   591us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:48.293909 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:14:48.293931 (+    22us) tablet.cc:662] Decoding operations
0709 13:14:48.294336 (+   405us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:48.294426 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:48.294457 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:14:48.294790 (+   333us) tablet.cc:701] Row locks acquired
0709 13:14:48.294809 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:14:48.294968 (+   159us) write_op.cc:270] Start()
0709 13:14:48.295131 (+   163us) write_op.cc:276] Timestamp: P: 1783602888294917 usec, L: 0
0709 13:14:48.295155 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:14:48.295919 (+   764us) log.cc:844] Serialized 929 byte log entry
0709 13:14:48.430974 (+135055us) op_driver.cc:464] REPLICATION: finished
0709 13:14:48.479667 (+ 48693us) write_op.cc:301] APPLY: starting
0709 13:14:48.479762 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:48.484828 (+  5066us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:48.484951 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:48.521660 (+ 36709us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:48.527465 (+  5805us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:14:48.527518 (+    53us) write_op.cc:312] APPLY: finished
0709 13:14:48.527769 (+   251us) log.cc:844] Serialized 33 byte log entry
0709 13:14:48.544914 (+ 17145us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:48.545016 (+   102us) write_op.cc:454] Released schema lock
0709 13:14:48.545060 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:48.545559 (+   499us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:48.545593 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1250747,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":48410,"apply.run_cpu_time_us":6031,"apply.run_wall_time_us":65869,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12515,"num_ops":2,"prepare.queue_time_us":97986,"prepare.run_cpu_time_us":2998,"prepare.run_wall_time_us":2999,"replication_time_us":135653,"spinlock_wait_cycles":2432,"thread_start_us":123128,"threads_started":2}]]}
W20260709 13:14:48.669224 27969 connection.cc:466] server connection from 127.0.0.1:58546 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 280) could send its response
W20260709 13:14:48.672257 26959 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 28107)
I20260709 13:14:48.672941 28159 raft_consensus.cc:1275] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:14:48.677151  4056 consensus_queue.cc:1048] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:14:48.704272 28125 consensus_peers.cc:597] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:48.732939 28199 consensus_peers.cc:597] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:48.748570 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet b99bb21dbacb4d63839f1403ef9b08b9 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:38034
I20260709 13:14:48.751362 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet b99bb21dbacb4d63839f1403ef9b08b9 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
W20260709 13:14:48.752810 27984 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58554 (request call id 284) took 98266 ms (client timeout 15000 ms). Trace:
W20260709 13:14:48.756928 27984 rpcz_store.cc:269] 0709 13:13:10.486441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.486984 (+   543us) service_pool.cc:224] Handling call
0709 13:14:48.752537 (+98265553us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19015,"reported_tablets":49}
I20260709 13:14:48.822067 27977 catalog_manager.cc:5697] T 4c9b061a7ca84a61ac13d29946a28169 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 17 to 20, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 20 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:48.833549 28125 consensus_peers.cc:597] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:48.833602 27977 catalog_manager.cc:5697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:48.841249 27977 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 19 to 21, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 21 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:48.852394 28159 raft_consensus.cc:1275] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:14:48.853839 28125 consensus_peers.cc:597] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:48.856909 28728 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:48.858711 27977 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 21 to 22, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 22 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:48.866671  4214 consensus_queue.cc:1048] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:48.873647 27977 catalog_manager.cc:5697] T c71e4401306a4e84a111e5443be7652c P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:48.877614 27977 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 0faaace2b36c4bcebf2fa6835235e92b reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:48.897354 28153 raft_consensus.cc:1275] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:14:48.901458 28199 consensus_peers.cc:597] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:48.907397  4214 consensus_queue.cc:1048] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:14:48.984400 27984 catalog_manager.cc:5697] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 27 to 28, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 28 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:49.017138 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:49.024144 28199 consensus_peers.cc:597] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:49.024978 28199 consensus_peers.cc:597] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:49.021610 28728 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:49.026175 28728 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:49.139933 28758 consensus_queue.cc:237] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:49.145794 27977 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 281) took 27445 ms (client timeout 15000 ms). Trace:
I20260709 13:14:49.151340 27982 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:49.146297 27977 rpcz_store.cc:269] 0709 13:14:21.705814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.049575 (+1343761us) service_pool.cc:224] Handling call
0709 13:14:23.525665 (+476090us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b54002c0010. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb5530b2 00007f59d44cca89 00007f59d4464da6 00007f59d4436088 00007f59d453437e 00007f59d29f52a4 00007f59d29f5232 00007f59d29f5161 00007f59d29f50dc
0709 13:14:49.145674 (+25620009us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:48.936158 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:48.937194 (+  1036us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:48.937230 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:14:48.937246 (+    16us) tablet.cc:662] Decoding operations
0709 13:14:48.938334 (+  1088us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:48.938399 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:48.938427 (+    28us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:14:48.939617 (+  1190us) tablet.cc:701] Row locks acquired
0709 13:14:48.939650 (+    33us) write_op.cc:260] PREPARE: finished
0709 13:14:48.942233 (+  2583us) write_op.cc:270] Start()
0709 13:14:48.942348 (+   115us) write_op.cc:276] Timestamp: P: 1783602888942183 usec, L: 0
0709 13:14:48.942370 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:14:48.943003 (+   633us) log.cc:844] Serialized 2482 byte log entry
0709 13:14:48.945145 (+  2142us) op_driver.cc:464] REPLICATION: finished
0709 13:14:48.945628 (+   483us) write_op.cc:301] APPLY: starting
0709 13:14:48.945734 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:48.949125 (+  3391us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:48.949233 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:48.955212 (+  5979us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:48.955402 (+   190us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:14:48.955448 (+    46us) write_op.cc:312] APPLY: finished
0709 13:14:48.955685 (+   237us) log.cc:844] Serialized 65 byte log entry
0709 13:14:48.971991 (+ 16306us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:48.972101 (+   110us) write_op.cc:454] Released schema lock
0709 13:14:48.972160 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:48.972590 (+   430us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:48.972618 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5648,"reported_tablets":48,"spinlock_wait_cycles":7751936,"child_traces":[["op",{"apply.queue_time_us":237,"apply.run_cpu_time_us":9625,"apply.run_wall_time_us":26905,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25164,"num_ops":6,"prepare.queue_time_us":1694,"prepare.run_cpu_time_us":4746,"prepare.run_wall_time_us":7267,"replication_time_us":2614,"spinlock_wait_cycles":2432}]]}
W20260709 13:14:49.174921 27969 connection.cc:466] server connection from 127.0.0.1:58546 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58546 (request call id 281) could send its response
I20260709 13:14:49.179721 27982 catalog_manager.cc:5697] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 19 to 22, leader changed from 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 22 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:49.209342 28728 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:49.212437 27982 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.219538 28227 raft_consensus.cc:1275] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:14:49.228742 27982 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 20 to 22, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 22 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.241617 27982 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 16 to 18, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 18 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.253428 27982 catalog_manager.cc:5697] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.238181  4230 consensus_queue.cc:1048] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:14:49.257359 27982 catalog_manager.cc:5697] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 20 to 23, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 23 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.259155 27982 catalog_manager.cc:5697] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 17 to 20, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 20 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.288321 27982 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 19 to 23, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 23 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:49.290557 28199 consensus_peers.cc:597] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:49.298853  4216 raft_consensus.cc:2955] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Committing config change with OpId 22.18: config changed from index 15 to 18, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:49.301915 28227 raft_consensus.cc:2955] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Committing config change with OpId 22.18: config changed from index 15 to 18, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:49.295050 27982 catalog_manager.cc:5697] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 20 to 21, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 21 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.344547 27968 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5c654c99561449cb823642a1b85879ac with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:14:49.350296 28125 consensus_peers.cc:597] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:49.415494 28199 consensus_peers.cc:597] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:49.461823 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:49.465610 28728 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:49.473631 27977 catalog_manager.cc:5697] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 15 to 18, VOTER 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) evicted. New cstate: current_term: 22 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:49.487577 28199 consensus_peers.cc:597] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:49.491654 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c654c99561449cb823642a1b85879ac with delete_type TABLET_DATA_TOMBSTONED (TS 611ec64ec58244fe82058d6130e4e6f3 not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:38034
I20260709 13:14:49.500721  4250 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.501439  4250 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:49.504060  4250 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
I20260709 13:14:49.508936 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:49.509670 28159 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 24.
I20260709 13:14:49.543982 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:49.548338 27983 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 21 to 22, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193). New cstate: current_term: 22 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.550902  4250 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:14:49.551416  4250 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.551795  4250 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:49.553154 27983 catalog_manager.cc:5697] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.554996 27983 catalog_manager.cc:5697] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 17 to 20, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 20 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.559362 27983 catalog_manager.cc:5697] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 13 to 17, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 17 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.581233  4270 tablet_replica.cc:333] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:49.587505 27983 catalog_manager.cc:5697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "0faaace2b36c4bcebf2fa6835235e92b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.588824 28146 tablet_service.cc:1558] Processing DeleteTablet for tablet b99bb21dbacb4d63839f1403ef9b08b9 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:38034
I20260709 13:14:49.596868 27983 catalog_manager.cc:5697] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 17 to 20, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 20 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.599669  4270 raft_consensus.cc:2243] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:49.605700  4270 raft_consensus.cc:2272] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:49.622452  4270 ts_tablet_manager.cc:1916] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:14:49.603591 28199 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:49.606782  3969 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.601059 27983 catalog_manager.cc:5697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 17 to 18, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 18 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.625173  3969 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:49.630667  4210 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.631273  4210 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:49.632669 28125 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:49.639111  4250 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:49.641942  4250 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587)
W20260709 13:14:49.673364 26959 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 28107)
W20260709 13:14:49.684845 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 81272.4us
I20260709 13:14:49.708246  4210 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:49.709182 28125 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:49.709787  3969 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:49.710618 27982 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58554 (request call id 288) took 14594 ms (client timeout 15000 ms). Trace:
W20260709 13:14:49.729628 27982 rpcz_store.cc:269] 0709 13:14:35.115985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.301166 (+2185181us) service_pool.cc:224] Handling call
0709 13:14:49.710543 (+12409377us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:49.349391 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:49.350044 (+   653us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:49.350095 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:14:49.350120 (+    25us) tablet.cc:662] Decoding operations
0709 13:14:49.351227 (+  1107us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:49.351289 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:49.351329 (+    40us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:14:49.352268 (+   939us) tablet.cc:701] Row locks acquired
0709 13:14:49.352284 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:14:49.352453 (+   169us) write_op.cc:270] Start()
0709 13:14:49.352573 (+   120us) write_op.cc:276] Timestamp: P: 1783602889352405 usec, L: 0
0709 13:14:49.352594 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:14:49.375836 (+ 23242us) log.cc:844] Serialized 4030 byte log entry
0709 13:14:49.380919 (+  5083us) op_driver.cc:464] REPLICATION: finished
0709 13:14:49.383924 (+  3005us) write_op.cc:301] APPLY: starting
0709 13:14:49.384027 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:49.396226 (+ 12199us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:49.396357 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:49.441841 (+ 45484us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:49.442114 (+   273us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:14:49.442169 (+    55us) write_op.cc:312] APPLY: finished
0709 13:14:49.442541 (+   372us) log.cc:844] Serialized 97 byte log entry
0709 13:14:49.459984 (+ 17443us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:49.460151 (+   167us) write_op.cc:454] Released schema lock
0709 13:14:49.460207 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:49.460636 (+   429us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:49.460671 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":2738,"apply.run_cpu_time_us":16324,"apply.run_wall_time_us":76684,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41940,"num_ops":10,"prepare.queue_time_us":4080,"prepare.run_cpu_time_us":4458,"prepare.run_wall_time_us":52438,"replication_time_us":28110}]]}
W20260709 13:14:49.749914 27983 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 265) took 14178 ms (client timeout 15000 ms). Trace:
W20260709 13:14:49.750447 27983 rpcz_store.cc:269] 0709 13:14:35.571214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.897216 (+4326002us) service_pool.cc:224] Handling call
0709 13:14:49.749844 (+9852628us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:49.645856 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:49.646457 (+   601us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:49.646501 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:14:49.646523 (+    22us) tablet.cc:662] Decoding operations
0709 13:14:49.647489 (+   966us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:49.647556 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:49.647590 (+    34us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:14:49.696878 (+ 49288us) tablet.cc:701] Row locks acquired
0709 13:14:49.696899 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:14:49.697068 (+   169us) write_op.cc:270] Start()
0709 13:14:49.697185 (+   117us) write_op.cc:276] Timestamp: P: 1783602889697019 usec, L: 0
0709 13:14:49.697207 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:14:49.697898 (+   691us) log.cc:844] Serialized 2870 byte log entry
0709 13:14:49.701382 (+  3484us) op_driver.cc:464] REPLICATION: finished
0709 13:14:49.702411 (+  1029us) write_op.cc:301] APPLY: starting
0709 13:14:49.702494 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:49.708550 (+  6056us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:49.708663 (+   113us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:49.714203 (+  5540us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:49.714402 (+   199us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:14:49.714450 (+    48us) write_op.cc:312] APPLY: finished
0709 13:14:49.714759 (+   309us) log.cc:844] Serialized 73 byte log entry
0709 13:14:49.715781 (+  1022us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:49.715901 (+   120us) write_op.cc:454] Released schema lock
0709 13:14:49.715953 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:49.716303 (+   350us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:49.716333 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":48369,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":806,"apply.run_cpu_time_us":9564,"apply.run_wall_time_us":13864,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29358,"num_ops":7,"prepare.queue_time_us":1414,"prepare.run_cpu_time_us":19667,"prepare.run_wall_time_us":69742,"replication_time_us":4062}]]}
I20260709 13:14:49.805891  4270 ts_tablet_manager.cc:1929] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.17
I20260709 13:14:49.806452  4270 log.cc:1199] T 5c654c99561449cb823642a1b85879ac P 611ec64ec58244fe82058d6130e4e6f3: Deleting WAL directory at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/wals/5c654c99561449cb823642a1b85879ac
I20260709 13:14:49.809176 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet 5c654c99561449cb823642a1b85879ac (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
W20260709 13:14:49.812161 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 57630.8us
I20260709 13:14:49.815560 27969 catalog_manager.cc:5028] TS 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): tablet b99bb21dbacb4d63839f1403ef9b08b9 (table test-workload [id=33cb0926ab7040f8a1735610acb16879]) successfully deleted
I20260709 13:14:49.827848 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:49.828574 28159 raft_consensus.cc:3060] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:14:49.837569 28159 raft_consensus.cc:2468] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 25.
I20260709 13:14:49.863898 27983 catalog_manager.cc:5697] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 22 to 25, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 25 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.865648 27983 catalog_manager.cc:5697] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 22 to 27, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 27 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.867806 27983 catalog_manager.cc:5697] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 22 to 24, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 24 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.891911 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:49.892529 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:49.892674 28232 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
I20260709 13:14:49.893188 28235 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 20.
W20260709 13:14:49.895430 28199 consensus_peers.cc:597] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:49.896029 28199 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:49.910358 27983 catalog_manager.cc:5697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 15 to 21, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 21 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.912264 27983 catalog_manager.cc:5697] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 19 to 20, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 20 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.922300 27983 catalog_manager.cc:5697] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 22 to 25, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 25 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
W20260709 13:14:49.922324 28728 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:49.924155 27983 catalog_manager.cc:5697] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 15 to 20, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 20 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.926517 27983 catalog_manager.cc:5697] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.931525 27983 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.933542 27983 catalog_manager.cc:5697] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 20 to 23, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 23 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.935029 27983 catalog_manager.cc:5697] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.950439 27983 catalog_manager.cc:5697] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 24 to 26, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 26 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.952637 27983 catalog_manager.cc:5697] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 14 to 20, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 20 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.954535 27983 catalog_manager.cc:5697] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.957937 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:49.959182  3969 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:49.959623  3969 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.960000  3969 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:49.969698  3969 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:49.955837 27983 catalog_manager.cc:5697] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 17 to 22, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 22 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.973567 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:49.974682  4210 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:14:49.975174  4210 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.975541  4210 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:49.974128 27983 catalog_manager.cc:5697] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } }
I20260709 13:14:49.988013  4210 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:50.011081 28199 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:50.020848  4210 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:50.021596  3969 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:50.022384  4214 raft_consensus.cc:2804] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:14:50.022990  4214 raft_consensus.cc:697] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 25 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:50.023958  4214 consensus_queue.cc:237] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:50.032079 28125 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:50.032801 28125 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:50.047646 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:50.070025 28125 consensus_peers.cc:597] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:50.089527 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8381961442846ab97965dc11b100d75" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:50.090170 28235 raft_consensus.cc:3060] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:50.090755 28232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:50.091395 28232 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:14:50.098433 28235 raft_consensus.cc:2468] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
W20260709 13:14:50.099396 28728 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:50.098775 28232 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611ec64ec58244fe82058d6130e4e6f3 in term 21.
W20260709 13:14:50.119225 28125 consensus_peers.cc:597] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:50.137996 28127 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:50.139326  4210 raft_consensus.cc:2804] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:50.139905  4210 raft_consensus.cc:697] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:50.140889  4210 consensus_queue.cc:237] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:50.149192 28127 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b
I20260709 13:14:50.150100  3969 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:14:50.150637  3969 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Becoming Leader. State: Replica: 611ec64ec58244fe82058d6130e4e6f3, State: Running, Role: LEADER
I20260709 13:14:50.151427  3969 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:50.154973 28199 consensus_peers.cc:597] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:50.161448 28728 consensus_peers.cc:597] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:50.197392 27982 catalog_manager.cc:5697] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 19 to 21, leader changed from 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) to 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194). New cstate: current_term: 21 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:50.229476 28199 consensus_peers.cc:597] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:50.258769 28125 consensus_peers.cc:597] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:50.283447 27982 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:50.286311 28199 consensus_peers.cc:597] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:50.324254 28125 consensus_peers.cc:597] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:50.374303 27983 catalog_manager.cc:5697] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 22 to 25, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 25 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:50.389266 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:50.422616 28728 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:50.540573 28199 consensus_peers.cc:597] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:50.545701 28153 raft_consensus.cc:1275] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:14:50.547376  4214 consensus_queue.cc:1048] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:14:50.577975 28728 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:50.581977 28199 consensus_peers.cc:597] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:50.643726 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:50.656131 28235 raft_consensus.cc:1275] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:50.658304  3969 consensus_queue.cc:1048] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:14:50.660050 28728 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:50.673960 26959 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 28107)
I20260709 13:14:50.679804 28232 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:14:50.681941  3969 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:50.689942  4271 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.690600  4271 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:50.693446  4271 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:50.720978 28125 consensus_peers.cc:597] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:50.735677 28153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3" is_pre_election: true
I20260709 13:14:50.736449 28153 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 23.
W20260709 13:14:50.763831 28125 consensus_peers.cc:597] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:50.770864 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
I20260709 13:14:50.773957  4271 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:14:50.774390  4271 raft_consensus.cc:493] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.774742  4271 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:14:50.780176 28728 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:50.786793  4271 raft_consensus.cc:515] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:50.801806 28199 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:50.806264  4271 leader_election.cc:290] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Requested vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:50.818969 28159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b0e8aadf244e7eb41e7e732bc27cf9" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "611ec64ec58244fe82058d6130e4e6f3"
I20260709 13:14:50.819626 28159 raft_consensus.cc:3060] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:14:50.827701 28159 raft_consensus.cc:2468] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 24.
I20260709 13:14:50.860232 28235 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:50.878098 28199 leader_election.cc:304] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3; no voters: 
W20260709 13:14:50.881747 28198 consensus_peers.cc:597] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e272db0632a4262a90eaacd6e6d4378. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:50.884305  4214 raft_consensus.cc:2804] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:14:50.884877  4214 raft_consensus.cc:697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 24 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:50.885833  4214 consensus_queue.cc:237] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:50.894220 28125 consensus_peers.cc:597] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:50.929504 28153 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 13:14:50.930449 28125 consensus_peers.cc:597] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:50.931411 28728 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:50.934952 28199 leader_election.cc:336] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:50.950575 28125 consensus_peers.cc:597] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:50.964869  4214 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:14:50.965916 28199 consensus_peers.cc:597] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:50.965309 27983 catalog_manager.cc:5697] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 18 to 24. New cstate: current_term: 24 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:51.009840 28199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111) [suppressed 448 similar messages]
I20260709 13:14:51.045217  4285 raft_consensus.cc:2955] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Committing config change with OpId 23.14: config changed from index -1 to 14, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:14:51.070204 28159 raft_consensus.cc:2955] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index -1 to 14, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:14:51.114238 27983 catalog_manager.cc:5697] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 14, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 23 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:14:51.134948 28125 consensus_peers.cc:597] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:51.147183 28199 consensus_peers.cc:597] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:51.176828 28159 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:14:51.201939 28124 consensus_peers.cc:597] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34353a7e5a8346119cf9f0836165a1b7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.203233 28233 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:14:51.206149  4280 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:14:51.210032 28125 consensus_peers.cc:597] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:51.216840 28125 consensus_peers.cc:597] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:51.234301 28199 consensus_peers.cc:597] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:51.238046  4280 raft_consensus.cc:2955] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Committing config change with OpId 25.16: config changed from index -1 to 16, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:14:51.250279 28125 consensus_peers.cc:597] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:51.255033 28728 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:51.267501 28235 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:51.279198 27982 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 16, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 25 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:51.299489 28233 raft_consensus.cc:2955] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Committing config change with OpId 25.16: config changed from index -1 to 16, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:14:51.334177 28198 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 888e64754b8d47aebb2c7c3aba7b0ee2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.345494 28153 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:51.349977 28153 raft_consensus.cc:1275] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:14:51.357720  4271 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:14:51.358402 28125 consensus_peers.cc:597] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:51.362268  4271 consensus_queue.cc:1048] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:14:51.364984 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:51.413272 28125 consensus_peers.cc:597] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:51.416469  4271 raft_consensus.cc:2955] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index -1 to 14, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:14:51.435647 28728 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:51.437032 28159 raft_consensus.cc:2955] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index -1 to 14, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:14:51.480832 27982 catalog_manager.cc:5697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 14, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 21 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:14:51.486485  4298 ts_tablet_manager.cc:933] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:51.499979 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 48993.2us
W20260709 13:14:51.524262 28199 consensus_peers.cc:597] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:51.524816 28199 consensus_peers.cc:597] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.528357  4298 tablet_copy_client.cc:323] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:14:51.534564 28245 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet 7e272db0632a4262a90eaacd6e6d4378 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:14:51.534548 28159 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:51.535514 28245 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet 7e272db0632a4262a90eaacd6e6d4378 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-7e272db0632a4262a90eaacd6e6d4378
I20260709 13:14:51.545645 28245 tablet_copy_source_session.cc:215] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:14:51.550406 28124 consensus_peers.cc:597] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 70996712c3c942468a6006602de6477f (127.26.83.196:46855): Couldn't send request to peer 70996712c3c942468a6006602de6477f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 25d46d07733d450e897e3a72d765e511. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.554271  4298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e272db0632a4262a90eaacd6e6d4378. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:51.576381 28125 consensus_peers.cc:597] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:51.578382  4298 tablet_copy_client.cc:806] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:51.578471 28235 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:14:51.580351  4298 tablet_copy_client.cc:670] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
W20260709 13:14:51.586844 28125 consensus_peers.cc:597] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:51.588307 28199 consensus_peers.cc:597] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:51.588939 28199 consensus_peers.cc:597] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:51.590193  4294 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:51.593384  4298 tablet_copy_client.cc:538] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:51.619160  4294 raft_consensus.cc:2955] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
I20260709 13:14:51.630118  4298 tablet_bootstrap.cc:492] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:14:51.639045 28235 raft_consensus.cc:2955] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } }
W20260709 13:14:51.674778 26959 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 28107)
I20260709 13:14:51.705461  4305 ts_tablet_manager.cc:933] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:51.738590 28728 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:51.746762 27982 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index -1 to 13, NON_VOTER 70996712c3c942468a6006602de6477f (127.26.83.196) added. New cstate: current_term: 21 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:51.767581 28728 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:51.779220  4305 tablet_copy_client.cc:323] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:14:51.784941 28170 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet 34353a7e5a8346119cf9f0836165a1b7 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:14:51.785671 28170 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet 34353a7e5a8346119cf9f0836165a1b7 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-34353a7e5a8346119cf9f0836165a1b7
I20260709 13:14:51.804164 28170 tablet_copy_source_session.cc:215] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:51.817489  4298 tablet_bootstrap.cc:492] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=3 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:51.817905  4308 ts_tablet_manager.cc:933] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:51.825407  4305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34353a7e5a8346119cf9f0836165a1b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:14:51.826176  4298 tablet_bootstrap.cc:492] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:14:51.829002  4298 ts_tablet_manager.cc:1403] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.199s	user 0.071s	sys 0.036s
I20260709 13:14:51.833262  4298 raft_consensus.cc:359] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:51.834172  4298 raft_consensus.cc:740] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
W20260709 13:14:51.834774 28199 consensus_peers.cc:597] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:51.834911  4298 consensus_queue.cc:260] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:51.838259  4298 ts_tablet_manager.cc:1434] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:14:51.840680 28245 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-7e272db0632a4262a90eaacd6e6d4378 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:14:51.841178 28245 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-7e272db0632a4262a90eaacd6e6d4378 on tablet 7e272db0632a4262a90eaacd6e6d4378 with peer 70996712c3c942468a6006602de6477f
I20260709 13:14:51.858263  4308 tablet_copy_client.cc:323] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.195:46875
I20260709 13:14:51.860098 28245 tablet_copy_service.cc:140] P 11f10c12c03e47dcac88b050d4c1431b: Received BeginTabletCopySession request for tablet 888e64754b8d47aebb2c7c3aba7b0ee2 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:44070)
I20260709 13:14:51.860698 28245 tablet_copy_service.cc:161] P 11f10c12c03e47dcac88b050d4c1431b: Beginning new tablet copy session on tablet 888e64754b8d47aebb2c7c3aba7b0ee2 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:44070: session id = 70996712c3c942468a6006602de6477f-888e64754b8d47aebb2c7c3aba7b0ee2
W20260709 13:14:51.861142 28125 consensus_peers.cc:597] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:51.865469  4305 tablet_copy_client.cc:806] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:51.866151  4305 tablet_copy_client.cc:670] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:51.878572 28245 tablet_copy_source_session.cc:215] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:51.882331  4308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888e64754b8d47aebb2c7c3aba7b0ee2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:51.884258 28199 consensus_peers.cc:597] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:51.888891  4305 tablet_copy_client.cc:538] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:51.899523  4305 tablet_bootstrap.cc:492] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:14:51.924225  4308 tablet_copy_client.cc:806] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:51.927107  4308 tablet_copy_client.cc:670] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
W20260709 13:14:51.927392 28199 consensus_peers.cc:597] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:51.939476  4308 tablet_copy_client.cc:538] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:51.972074  4308 tablet_bootstrap.cc:492] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
I20260709 13:14:52.008754 28758 raft_consensus.cc:1217] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 LEARNER]: Deduplicated request from leader. Original: 23.13->[23.14-23.14]   Dedup: 23.14->[]
W20260709 13:14:52.134374 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:52.166363  4308 tablet_bootstrap.cc:492] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=3 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:52.173643  4308 tablet_bootstrap.cc:492] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:14:52.181440  4308 ts_tablet_manager.cc:1403] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.210s	user 0.047s	sys 0.012s
I20260709 13:14:52.185032  4308 raft_consensus.cc:359] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:52.192955  4308 raft_consensus.cc:740] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:14:52.186394  4298 ts_tablet_manager.cc:933] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: Initiating tablet copy from peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:52.193810  4308 consensus_queue.cc:260] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
W20260709 13:14:52.203438  4305 log.cc:927] Time spent T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:14:52.206645  4308 ts_tablet_manager.cc:1434] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:14:52.209015 28245 tablet_copy_service.cc:342] P 11f10c12c03e47dcac88b050d4c1431b: Request end of tablet copy session 70996712c3c942468a6006602de6477f-888e64754b8d47aebb2c7c3aba7b0ee2 received from {username='slave'} at 127.0.0.1:44070
I20260709 13:14:52.209782 28245 tablet_copy_service.cc:434] P 11f10c12c03e47dcac88b050d4c1431b: ending tablet copy session 70996712c3c942468a6006602de6477f-888e64754b8d47aebb2c7c3aba7b0ee2 on tablet 888e64754b8d47aebb2c7c3aba7b0ee2 with peer 70996712c3c942468a6006602de6477f
I20260709 13:14:52.237511  4298 tablet_copy_client.cc:323] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: tablet copy: Beginning tablet copy session from remote peer at address 127.26.83.194:38757
I20260709 13:14:52.264223  4305 tablet_bootstrap.cc:492] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:52.265254  4305 tablet_bootstrap.cc:492] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:14:52.266057  4305 ts_tablet_manager.cc:1403] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.367s	user 0.044s	sys 0.047s
I20260709 13:14:52.273003 28170 tablet_copy_service.cc:140] P 611ec64ec58244fe82058d6130e4e6f3: Received BeginTabletCopySession request for tablet 25d46d07733d450e897e3a72d765e511 from peer 70996712c3c942468a6006602de6477f ({username='slave'} at 127.0.0.1:43184)
I20260709 13:14:52.273694 28170 tablet_copy_service.cc:161] P 611ec64ec58244fe82058d6130e4e6f3: Beginning new tablet copy session on tablet 25d46d07733d450e897e3a72d765e511 from peer 70996712c3c942468a6006602de6477f at {username='slave'} at 127.0.0.1:43184: session id = 70996712c3c942468a6006602de6477f-25d46d07733d450e897e3a72d765e511
I20260709 13:14:52.292526  4305 raft_consensus.cc:359] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:52.296144  4305 raft_consensus.cc:740] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
W20260709 13:14:52.303118 28199 consensus_peers.cc:597] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:52.303849 28170 tablet_copy_source_session.cc:215] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:14:52.302098  4305 consensus_queue.cc:260] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:52.320196  4305 ts_tablet_manager.cc:1434] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.003s
I20260709 13:14:52.322098  4298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25d46d07733d450e897e3a72d765e511. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:14:52.326541 28728 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:52.328600 28170 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-34353a7e5a8346119cf9f0836165a1b7 received from {username='slave'} at 127.0.0.1:43184
I20260709 13:14:52.329193 28170 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-34353a7e5a8346119cf9f0836165a1b7 on tablet 34353a7e5a8346119cf9f0836165a1b7 with peer 70996712c3c942468a6006602de6477f
I20260709 13:14:52.353516  4298 tablet_copy_client.cc:806] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 0 data blocks...
I20260709 13:14:52.354192  4298 tablet_copy_client.cc:670] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: tablet copy: Starting download of 1 WAL segments...
I20260709 13:14:52.361877  4298 tablet_copy_client.cc:538] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:14:52.387854 28758 raft_consensus.cc:1217] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [term 21 LEARNER]: Deduplicated request from leader. Original: 21.13->[21.14-21.14]   Dedup: 21.14->[]
W20260709 13:14:52.402869 28125 consensus_peers.cc:597] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:52.413859  4298 tablet_bootstrap.cc:492] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: Bootstrap starting.
W20260709 13:14:52.559242 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:52.572286  4298 tablet_bootstrap.cc:492] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:14:52.576095  4298 tablet_bootstrap.cc:492] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: Bootstrap complete.
I20260709 13:14:52.577888  4298 ts_tablet_manager.cc:1403] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: Time spent bootstrapping tablet: real 0.164s	user 0.037s	sys 0.008s
I20260709 13:14:52.582737  4298 raft_consensus.cc:359] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:52.583649  4298 raft_consensus.cc:740] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 70996712c3c942468a6006602de6477f, State: Initialized, Role: LEARNER
I20260709 13:14:52.584275  4298 consensus_queue.cc:260] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } }
I20260709 13:14:52.586967  4298 ts_tablet_manager.cc:1434] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:14:52.613096 28758 raft_consensus.cc:1217] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 LEARNER]: Deduplicated request from leader. Original: 25.15->[25.16-25.16]   Dedup: 25.16->[]
I20260709 13:14:52.620142 28170 tablet_copy_service.cc:342] P 611ec64ec58244fe82058d6130e4e6f3: Request end of tablet copy session 70996712c3c942468a6006602de6477f-25d46d07733d450e897e3a72d765e511 received from {username='slave'} at 127.0.0.1:43184
I20260709 13:14:52.620609 28170 tablet_copy_service.cc:434] P 611ec64ec58244fe82058d6130e4e6f3: ending tablet copy session 70996712c3c942468a6006602de6477f-25d46d07733d450e897e3a72d765e511 on tablet 25d46d07733d450e897e3a72d765e511 with peer 70996712c3c942468a6006602de6477f
I20260709 13:14:52.621529  4214 raft_consensus.cc:1064] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:14:52.624328  4214 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 12, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:52.646961 28199 consensus_peers.cc:597] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:52.661679 28155 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:14:52.663923 28758 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:14:52.665952  4271 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:52.666841  4300 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:14:52.675302 26959 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 28107)
W20260709 13:14:52.679690 28199 consensus_peers.cc:597] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:52.724967  4316 log.cc:927] Time spent T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: Append to log took a long time: real 0.067s	user 0.019s	sys 0.010s
W20260709 13:14:52.728197 28199 consensus_peers.cc:597] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:52.764796 28757 raft_consensus.cc:1217] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 LEARNER]: Deduplicated request from leader. Original: 21.12->[21.13-21.13]   Dedup: 21.13->[]
W20260709 13:14:52.771446 28728 consensus_peers.cc:597] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:52.772111 28728 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:52.839583 28125 consensus_peers.cc:597] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:52.855969  4319 raft_consensus.cc:2955] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index 14 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:52.860025 28153 raft_consensus.cc:2955] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index 14 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:52.875174 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 58721.8us
I20260709 13:14:52.895464 27982 catalog_manager.cc:5697] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 14 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:52.918591 28125 consensus_peers.cc:597] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:52.946208 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 59920.8us
I20260709 13:14:52.962366 28757 raft_consensus.cc:2955] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index 14 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:52.968662 28728 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:52.993211  4271 raft_consensus.cc:1064] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:14:52.997292  4271 consensus_queue.cc:237] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:53.000129 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:53.021451 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:53.045915 28757 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [term 21 LEARNER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:14:53.053018  4320 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:53.059190 28153 raft_consensus.cc:1275] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:14:53.074216  4271 consensus_queue.cc:1048] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:14:53.076351 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:53.080320 28235 consensus_queue.cc:237] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:53.085950 28728 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:53.088353 28758 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:14:53.090507  4271 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:53.099948 28159 raft_consensus.cc:1275] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:14:53.111248  4294 raft_consensus.cc:1064] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:14:53.113968  4294 consensus_queue.cc:237] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 14, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:53.117646  4319 raft_consensus.cc:2955] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 15 to 16, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:53.125939  4302 consensus_queue.cc:1048] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:14:53.131412 28757 raft_consensus.cc:2955] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:53.137678  4302 raft_consensus.cc:2955] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index 14 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:53.139578 28728 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:53.146682 28153 raft_consensus.cc:2955] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:53.151242 28199 consensus_peers.cc:597] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:53.152328 28159 raft_consensus.cc:2955] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:53.167363 28199 consensus_peers.cc:597] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:53.162492 27971 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7e272db0632a4262a90eaacd6e6d4378 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:53.180302 27983 catalog_manager.cc:5697] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 15 to 16, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 23 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:53.229972 27982 catalog_manager.cc:5697] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 14 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:53.249171 28757 raft_consensus.cc:2955] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:53.258507 28125 consensus_peers.cc:597] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:53.261317 27969 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 49740.6us
I20260709 13:14:53.262079 28233 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:14:53.264375  4210 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:14:53.266400 27969 catalog_manager.cc:4755] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 7e272db0632a4262a90eaacd6e6d4378: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:53.269186 28125 consensus_peers.cc:597] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:53.274746 28758 raft_consensus.cc:1275] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:14:53.294760  4280 consensus_queue.cc:1048] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:14:53.305116  4280 raft_consensus.cc:1064] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: attempting to promote NON_VOTER 70996712c3c942468a6006602de6477f to VOTER
I20260709 13:14:53.308696  4280 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:53.325378  4294 raft_consensus.cc:2955] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Committing config change with OpId 25.17: config changed from index 16 to 17, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:53.327308 28233 raft_consensus.cc:2955] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Committing config change with OpId 25.17: config changed from index 16 to 17, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:53.371601 28728 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:53.390558 27977 catalog_manager.cc:5697] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: config changed from index 16 to 17, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 25 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:53.401580 28124 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:46855, user_credentials={real_user=slave}} blocked reactor thread for 45071.3us
W20260709 13:14:53.401006 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45646.1us
I20260709 13:14:53.407241 28758 raft_consensus.cc:2955] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Committing config change with OpId 25.17: config changed from index 16 to 17, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:53.450398 28125 consensus_peers.cc:597] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:53.452077 28233 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:53.453997  4340 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:53.457525 28758 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 LEARNER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:14:53.461284  4280 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
W20260709 13:14:53.475797 28125 consensus_peers.cc:597] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:53.493708 28125 consensus_peers.cc:597] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:53.494352 28125 consensus_peers.cc:597] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:53.528131 28199 consensus_peers.cc:597] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:53.531607 28125 consensus_peers.cc:597] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:53.545569  4338 raft_consensus.cc:2955] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:53.550586 28199 consensus_peers.cc:597] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:53.557070 28728 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:53.562494 28758 raft_consensus.cc:2955] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:53.567756 28227 raft_consensus.cc:2955] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:53.573607 27977 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 reported cstate change: config changed from index 13 to 14, 70996712c3c942468a6006602de6477f (127.26.83.196) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:53.623430 28159 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:53.636199 28227 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:14:53.644999  4338 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:53.650087 28757 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Refusing update from remote peer 611ec64ec58244fe82058d6130e4e6f3: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:14:53.658784  4338 raft_consensus.cc:2955] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index 14 to 15, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:53.661603  4280 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:53.668447 28758 raft_consensus.cc:2955] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:53.675777 26959 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 28107)
I20260709 13:14:53.677536 28233 raft_consensus.cc:2955] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
I20260709 13:14:53.687515 27969 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 25d46d07733d450e897e3a72d765e511 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:53.702956 27977 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f reported cstate change: config changed from index 14 to 15, VOTER 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193) evicted. New cstate: current_term: 21 leader_uuid: "611ec64ec58244fe82058d6130e4e6f3" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } } }
W20260709 13:14:53.760964 28728 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:53.882146 28199 consensus_peers.cc:597] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:53.900092 28199 consensus_peers.cc:597] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:53.908684 28199 consensus_peers.cc:597] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:53.925321 28728 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:53.933925 28125 consensus_peers.cc:597] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:53.943157 28199 consensus_peers.cc:597] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:53.966147 28728 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:53.974234 28125 consensus_peers.cc:597] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:53.995112 28125 consensus_peers.cc:597] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:54.001240 28125 consensus_peers.cc:597] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:54.021088 28728 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:54.064237 28125 consensus_peers.cc:597] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:54.187531 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 60291.3us
W20260709 13:14:54.307854 28125 consensus_peers.cc:597] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:54.374434 28199 consensus_peers.cc:597] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:54.432056 28199 consensus_peers.cc:597] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:54.545112 27969 catalog_manager.cc:4755] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 25d46d07733d450e897e3a72d765e511: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:54.635195 28199 consensus_peers.cc:597] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:54.676309 26959 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 28107)
W20260709 13:14:54.932945 28728 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:55.010217 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:55.076040 28125 consensus_peers.cc:597] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:55.135732 28728 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:55.130636 28199 consensus_peers.cc:597] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:55.137845 28199 consensus_peers.cc:597] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:55.164813 28199 consensus_peers.cc:597] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:55.297580 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:55.358531 28728 consensus_peers.cc:597] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:55.450496 28728 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:55.490993 28199 consensus_peers.cc:597] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:55.586808 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:55.616566 27969 catalog_manager.cc:4755] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 7e272db0632a4262a90eaacd6e6d4378: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:55.664635 28125 consensus_peers.cc:597] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:55.677090 26959 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 28107)
W20260709 13:14:55.701340 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:55.720912 28728 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:55.731483 28125 consensus_peers.cc:597] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:55.732061 28125 consensus_peers.cc:597] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:55.738248 28728 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:55.744518 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:55.856413 28125 consensus_peers.cc:597] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:55.908789 28125 consensus_peers.cc:597] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:55.908950 28728 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:55.922043 28125 consensus_peers.cc:597] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:55.960114 28125 consensus_peers.cc:597] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:55.998875 28728 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:56.029843 28199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111) [suppressed 473 similar messages]
W20260709 13:14:56.082398 28199 consensus_peers.cc:597] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:56.179303 28199 consensus_peers.cc:597] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:56.240592 28125 consensus_peers.cc:597] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:56.273864 28125 consensus_peers.cc:597] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:56.308772 28199 consensus_peers.cc:597] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:56.418648 28125 consensus_peers.cc:597] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:56.441098 28199 consensus_peers.cc:597] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:56.467166 28728 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:56.484220 28199 consensus_peers.cc:597] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:14:56.503546 28199 consensus_peers.cc:597] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:56.506242 28125 consensus_peers.cc:597] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:56.531149 28127 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 50285.1us
W20260709 13:14:56.557508 28728 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:56.560052 28728 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:56.603642 26959 tablet_server.cc:196] TabletServer@127.26.83.193:0 shutdown complete.
W20260709 13:14:56.665897 28199 consensus_peers.cc:597] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:56.740834 28728 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:56.748694 28125 consensus_peers.cc:597] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:56.863245 26959 tablet_server.cc:179] TabletServer@127.26.83.194:0 shutting down...
W20260709 13:14:56.891893 28199 consensus_peers.cc:597] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:56.898974 28728 consensus_peers.cc:597] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:56.905331 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:56.925843 28199 consensus_peers.cc:597] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:56.934978 28199 consensus_peers.cc:597] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:56.999708 28125 consensus_peers.cc:597] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:57.010516 28199 consensus_peers.cc:597] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:57.014528 28125 consensus_peers.cc:597] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:57.023090 28199 consensus_peers.cc:597] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.028143 28728 consensus_peers.cc:597] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.055644 28199 consensus_peers.cc:597] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.056501 28199 consensus_peers.cc:597] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.059976 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.060689 28728 consensus_peers.cc:597] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.062964 26959 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:14:57.071961 26959 tablet_replica.cc:333] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.073179 26959 raft_consensus.cc:2243] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.074378 26959 raft_consensus.cc:2272] T 6f5f1be456de4894af76dffdad9ea6a3 P 611ec64ec58244fe82058d6130e4e6f3 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:57.077068 28728 consensus_peers.cc:597] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.085140 28199 consensus_peers.cc:597] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.077746 26959 tablet_replica.cc:333] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
W20260709 13:14:57.112710 28728 consensus_peers.cc:597] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.202771 28199 consensus_peers.cc:597] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.208981 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 96358.4us
I20260709 13:14:57.209055 26959 raft_consensus.cc:2243] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:14:57.209707 28728 consensus_peers.cc:597] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.210522 26959 raft_consensus.cc:2272] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.280764 26959 tablet_replica.cc:333] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.282207 26959 raft_consensus.cc:2243] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Raft consensus shutting down.
W20260709 13:14:57.284893 28125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 70371.2us
I20260709 13:14:57.301574 26959 raft_consensus.cc:2272] T 6fbb28a09242477abe442ecc8a938738 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:57.290769 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.300288 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:38757, user_credentials={real_user=slave}} blocked reactor thread for 89674.7us
W20260709 13:14:57.320077 28125 consensus_peers.cc:597] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:57.319594 28199 consensus_peers.cc:597] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:57.322072 26959 tablet_replica.cc:333] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.355474 26959 raft_consensus.cc:2243] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.371268 26959 raft_consensus.cc:2272] T 66540b865ca24721910c667ccd790b77 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:57.380255 28199 consensus_peers.cc:597] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.381228 28199 consensus_peers.cc:597] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.384351 28199 consensus_peers.cc:597] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.381718 26959 tablet_replica.cc:333] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.386495 26959 raft_consensus.cc:2243] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.392927 26959 raft_consensus.cc:2272] T ceb515ffdfa04f2f977a4c5aa1aecbee P 611ec64ec58244fe82058d6130e4e6f3 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.396296 26959 tablet_replica.cc:333] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.408862 26959 raft_consensus.cc:2243] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:14:57.399017 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.413533 26959 raft_consensus.cc:2272] T c20222bc64224781bd97e8b75f2ddfa1 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:57.413347 28728 consensus_peers.cc:597] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.437445 26959 tablet_replica.cc:333] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
W20260709 13:14:57.412163 28199 consensus_peers.cc:597] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:57.444962 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:57.449082 28728 consensus_peers.cc:597] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.449726 28728 consensus_peers.cc:597] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.446148 26959 raft_consensus.cc:2243] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:14:57.454058 26959 raft_consensus.cc:2272] T 17fbe81a71fd4f8c83a65145cf2f0682 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.458287 26959 tablet_replica.cc:333] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.466145 26959 raft_consensus.cc:2243] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:14:57.467525 26959 raft_consensus.cc:2272] T 0720cab120de4047b423cb89ed85154a P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:57.460227 28728 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:14:57.480000 28199 consensus_peers.cc:597] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.482883 28199 consensus_peers.cc:597] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.483737 28199 consensus_peers.cc:597] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.485872 28199 consensus_peers.cc:597] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.495170 28728 consensus_peers.cc:597] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:57.509385 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.499729 26959 tablet_replica.cc:333] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.511166 26959 raft_consensus.cc:2243] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:14:57.513087 28199 consensus_peers.cc:597] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:14:57.513687 26959 raft_consensus.cc:2272] T 6ef726aecabb47089dcd217bfb8bfefc P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.518790 26959 tablet_replica.cc:333] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.520985 26959 raft_consensus.cc:2243] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:14:57.528507 26959 raft_consensus.cc:2272] T e09b3a74da19442187d6c447c95018a9 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.539556 26959 tablet_replica.cc:333] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.545334 26959 raft_consensus.cc:2243] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.547251 26959 raft_consensus.cc:2272] T fc1588fd3df04c2c8354baa741ddeca9 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.557745 26959 tablet_replica.cc:333] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.558761 26959 raft_consensus.cc:2243] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:14:57.562158 26959 raft_consensus.cc:2272] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.568480 26959 tablet_replica.cc:333] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.571683 26959 raft_consensus.cc:2243] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.585261 26959 raft_consensus.cc:2272] T aa289960fe9a4d2c8e61deb822de8412 P 611ec64ec58244fe82058d6130e4e6f3 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.588482 26959 tablet_replica.cc:333] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.589407 26959 raft_consensus.cc:2243] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.591320 26959 raft_consensus.cc:2272] T fab37ead976b4378b1298941cec677a8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.594158 26959 tablet_replica.cc:333] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.595161 26959 raft_consensus.cc:2243] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:14:57.596511 26959 raft_consensus.cc:2272] T 1115d5b02e774b14935e3f48488f82ea P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.606534 26959 tablet_replica.cc:333] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.607635 26959 raft_consensus.cc:2243] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:14:57.609062 26959 raft_consensus.cc:2272] T 8d3461aa17604e69b08928a9bc0f8d18 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.623594 26959 tablet_replica.cc:333] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.624514 26959 raft_consensus.cc:2243] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.625561 26959 raft_consensus.cc:2272] T b7ee90e83c2f49509f54dc87ea4d50e6 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.628759 26959 tablet_replica.cc:333] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.629773 26959 raft_consensus.cc:2243] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.630903 26959 raft_consensus.cc:2272] T 4b129d69618f48999f257e73b5b8b1d8 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.634382 26959 tablet_replica.cc:333] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
W20260709 13:14:57.647541 28728 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:14:57.649591 26959 raft_consensus.cc:2243] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.650763 26959 raft_consensus.cc:2272] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.653880 26959 tablet_replica.cc:333] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.654817 26959 raft_consensus.cc:2243] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.655943 26959 raft_consensus.cc:2272] T 798c63c7bb2e4373af20f55bbe872cf2 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.659281 26959 tablet_replica.cc:333] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.673579 26959 raft_consensus.cc:2243] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.674623 26959 raft_consensus.cc:2272] T c0a4a73cb5754c40af548cd4519da5b4 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.681354 26959 tablet_replica.cc:333] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.682677 26959 raft_consensus.cc:2243] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.684091 26959 raft_consensus.cc:2272] T 6a669edb54e3494387ede68c8acd8422 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.687526 26959 tablet_replica.cc:333] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.697750 26959 raft_consensus.cc:2243] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:14:57.699851 26959 raft_consensus.cc:2272] T f703ac7a390a45efae315d629c1dad89 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.703207 26959 tablet_replica.cc:333] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.704142 26959 raft_consensus.cc:2243] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:14:57.713923 26959 raft_consensus.cc:2272] T 2d0ccc53d55d4d73baf657119bf89f1c P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.717893 26959 tablet_replica.cc:333] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.719671 26959 raft_consensus.cc:2243] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.720512 26959 raft_consensus.cc:2272] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.723397 26959 tablet_replica.cc:333] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.728888 26959 raft_consensus.cc:2243] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.731511 26959 raft_consensus.cc:2272] T 8ada0581aa09483e91a300220a129859 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.735249 26959 tablet_replica.cc:333] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.765403 26959 raft_consensus.cc:2243] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:14:57.766676 26959 raft_consensus.cc:2272] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 611ec64ec58244fe82058d6130e4e6f3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.770610 26959 tablet_replica.cc:333] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.771404 26959 raft_consensus.cc:2243] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:14:57.779625 26959 raft_consensus.cc:2272] T 030bc6df8e7e4f2da3b18613d20d9bdc P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.783411 26959 tablet_replica.cc:333] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.793696 26959 raft_consensus.cc:2243] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.795069 26959 raft_consensus.cc:2272] T 33b72fb1e20942d9a3907c50252930bb P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:57.794507 27969 catalog_manager.cc:4755] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 7e272db0632a4262a90eaacd6e6d4378: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:57.802670 26959 tablet_replica.cc:333] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.834656 26959 raft_consensus.cc:2243] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.835170  4363 raft_consensus.cc:493] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:57.838542  4363 raft_consensus.cc:515] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:57.842936 26959 raft_consensus.cc:2272] T 7c0743024f324c14b4c6fc08f70600d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.846122 26959 tablet_replica.cc:333] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.859931 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:57.862207  4363 leader_election.cc:290] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:57.862249 26959 raft_consensus.cc:2243] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 LEADER]: Raft consensus shutting down.
W20260709 13:14:57.865080 28199 consensus_peers.cc:597] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:57.865893 26959 raft_consensus.cc:2272] T 34353a7e5a8346119cf9f0836165a1b7 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.880386 26959 tablet_replica.cc:333] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.888770 26959 raft_consensus.cc:2243] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.894402 26959 raft_consensus.cc:2272] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.895987 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9145e2ecf82c4a03b0034029f10bab74" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:57.912539 28199 consensus_peers.cc:597] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:57.919325 28199 consensus_peers.cc:597] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:57.920310 28124 leader_election.cc:304] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
W20260709 13:14:57.921274 28124 raft_consensus.cc:2686] T 9145e2ecf82c4a03b0034029f10bab74 P 611ec64ec58244fe82058d6130e4e6f3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:14:57.952600  4363 raft_consensus.cc:493] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 70996712c3c942468a6006602de6477f)
I20260709 13:14:57.953419  4363 raft_consensus.cc:515] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:57.955818  4363 leader_election.cc:290] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:57.959967 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:57.952557 26959 tablet_replica.cc:333] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.963316 26959 raft_consensus.cc:2243] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.964437 26959 raft_consensus.cc:2272] T 53c6cdf8c6704ee59d0a736273582a48 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:57.987195 26959 tablet_replica.cc:333] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:57.988109 26959 raft_consensus.cc:2243] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:57.990090 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56e9de7a9c54453bc5fc3aac0fa589c" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:57.992496 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:14:57.995699 26959 raft_consensus.cc:2272] T 552d81eee50848758aae23da5ed8927b P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.020620 28124 leader_election.cc:304] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f
I20260709 13:14:58.022413  4381 raft_consensus.cc:2749] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:58.028569 26959 tablet_replica.cc:333] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.029943 26959 raft_consensus.cc:2243] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:14:58.036387 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:58.036962 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:14:58.042138 26959 raft_consensus.cc:2272] T a8381961442846ab97965dc11b100d75 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.049618 26959 tablet_replica.cc:333] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.050563 26959 raft_consensus.cc:2243] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:58.055251 26959 raft_consensus.cc:2272] T 9e12858e91dd456a9f422bca6d2d6ac3 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.058508 26959 tablet_replica.cc:333] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.059458 26959 raft_consensus.cc:2243] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:58.060371 26959 raft_consensus.cc:2272] T d8d07cbc383b4089b326bc02e8b33622 P 611ec64ec58244fe82058d6130e4e6f3 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.071065 26959 tablet_replica.cc:333] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.072425 26959 raft_consensus.cc:2243] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:58.073357 26959 raft_consensus.cc:2272] T d56e9de7a9c54453bc5fc3aac0fa589c P 611ec64ec58244fe82058d6130e4e6f3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.076747 26959 tablet_replica.cc:333] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.077672 26959 raft_consensus.cc:2243] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:58.078549 26959 raft_consensus.cc:2272] T c71e4401306a4e84a111e5443be7652c P 611ec64ec58244fe82058d6130e4e6f3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.081843 26959 tablet_replica.cc:333] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.082901 26959 raft_consensus.cc:2243] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:58.083696 26959 raft_consensus.cc:2272] T 847d15cbd80f44b18e547291eed25617 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:58.102944 28728 consensus_peers.cc:597] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:58.104637 26959 tablet_replica.cc:333] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.105641 26959 raft_consensus.cc:2243] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:58.110517  4381 raft_consensus.cc:493] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:58.111334  4381 raft_consensus.cc:515] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:58.119158 28125 leader_election.cc:336] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:58.124610  4381 leader_election.cc:290] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:58.125622 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2facf6a0a91849a5a177c2481e42aa47" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
W20260709 13:14:58.126094 28728 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:14:58.127348 28127 leader_election.cc:304] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:58.128813  4381 raft_consensus.cc:2749] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:14:58.130385 28728 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:14:58.131251 26959 raft_consensus.cc:2272] T 4c9b061a7ca84a61ac13d29946a28169 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.168462  4381 raft_consensus.cc:493] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:14:58.169209  4381 raft_consensus.cc:515] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
W20260709 13:14:58.178519 28125 leader_election.cc:336] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:58.180207  4381 leader_election.cc:290] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:58.181526 26959 tablet_replica.cc:333] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.182672 26959 raft_consensus.cc:2243] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:58.184128 26959 raft_consensus.cc:2272] T 7e272db0632a4262a90eaacd6e6d4378 P 611ec64ec58244fe82058d6130e4e6f3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.186940 28235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d65f83feb5bb4b468e71fc5b9b708de4" candidate_uuid: "611ec64ec58244fe82058d6130e4e6f3" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:58.211076 28127 leader_election.cc:304] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611ec64ec58244fe82058d6130e4e6f3; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:14:58.212090  4381 raft_consensus.cc:2749] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:14:58.239917 26959 tablet_replica.cc:333] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.240864 26959 raft_consensus.cc:2243] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:58.241622 26959 raft_consensus.cc:2272] T d65f83feb5bb4b468e71fc5b9b708de4 P 611ec64ec58244fe82058d6130e4e6f3 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.252986 26959 tablet_replica.cc:333] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.253719 26959 raft_consensus.cc:2243] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 LEADER]: Raft consensus shutting down.
W20260709 13:14:58.265211 28728 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:14:58.265892 26959 raft_consensus.cc:2272] T e6b1c27f68974f4a9878c7c155cb43d2 P 611ec64ec58244fe82058d6130e4e6f3 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.269055 26959 tablet_replica.cc:333] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.271490 26959 raft_consensus.cc:2243] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:14:58.274144 26959 raft_consensus.cc:2272] T 8addb79d6158437f8d7a4822a75661ab P 611ec64ec58244fe82058d6130e4e6f3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:58.286283 26959 tablet_replica.cc:333] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.287159 26959 raft_consensus.cc:2243] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:14:58.288341 26959 raft_consensus.cc:2272] T 25d46d07733d450e897e3a72d765e511 P 611ec64ec58244fe82058d6130e4e6f3 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:58.310977 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:14:58.318248 28728 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:14:58.338722 26959 tablet_replica.cc:333] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3: stopping tablet replica
I20260709 13:14:58.339581 26959 raft_consensus.cc:2243] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:58.352471 26959 raft_consensus.cc:2272] T 2facf6a0a91849a5a177c2481e42aa47 P 611ec64ec58244fe82058d6130e4e6f3 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:14:58.357633 28256 debug-util.cc:398] Leaking SignalData structure 0x7b0800370be0 after lost signal to thread 26962
W20260709 13:14:58.358857 28256 debug-util.cc:398] Leaking SignalData structure 0x7b080027ee20 after lost signal to thread 28018
W20260709 13:14:58.359484 28256 debug-util.cc:398] Leaking SignalData structure 0x7b08000f5180 after lost signal to thread 28185
W20260709 13:14:58.360680 28256 debug-util.cc:398] Leaking SignalData structure 0x7b0800348920 after lost signal to thread 28259
I20260709 13:14:58.365338  4375 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:58.366166 28256 debug-util.cc:398] Leaking SignalData structure 0x7b08003639c0 after lost signal to thread 28789
I20260709 13:14:58.365958  4375 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:58.368300  4375 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:58.394546 28199 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:58.405411 28199 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:58.407078 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:58.412228  4375 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:58.549973  4375 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:58.550554  4375 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:58.558140  4375 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:58.598578 28730 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:46875, user_credentials={real_user=slave}} blocked reactor thread for 45770.6us
W20260709 13:14:58.646049 28199 leader_election.cc:336] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:58.648030 28199 leader_election.cc:336] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:58.648538 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:58.652284  4388 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:58.755422 28728 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:14:58.790069  4377 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:58.790729  4377 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:58.791410  4386 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:58.792068  4386 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:58.793207  4377 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:58.794852  4386 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:58.804615 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:58.805325 28728 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:58.809896  4386 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:58.810395 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:58.810519  4386 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:58.811010 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:58.812709  4386 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:58.813102  4377 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:58.815675 28199 consensus_peers.cc:597] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:14:58.817982 28728 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:14:58.818744 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:58.819943  4352 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:14:58.829806 28728 connection.cc:769] client connection to 127.26.83.194:38757 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:14:58.830430 28728 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:14:58.831828 28728 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:58.832373 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:58.833233  4386 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:58.858834 28199 consensus_peers.cc:597] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:58.867620  4388 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:58.871184  4388 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:58.873545  4388 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:58.922082  4352 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:58.922724  4352 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:58.925697  4352 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:58.928331 28199 leader_election.cc:336] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:58.929105 28199 consensus_peers.cc:597] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:14:58.929834 28199 leader_election.cc:336] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:58.930248 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:58.932528  4388 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:14:58.932894 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:58.933872 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:58.934419 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:58.935302  4352 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:14:58.998198  4388 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:58.998888  4388 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:59.001462  4388 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:59.011461 28199 consensus_peers.cc:597] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:59.016530  4388 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.017241  4388 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:59.019742  4388 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:59.034075  4352 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.034811  4352 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:59.043802  4352 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:59.053282  4386 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.053953  4386 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:59.057726 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:59.061175  4386 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:59.066102 28199 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
W20260709 13:14:59.066949 28199 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:59.067360 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:59.069271  4388 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:14:59.074783 28728 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:59.075623 28728 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.076076 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:59.088364  4390 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.089112  4390 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:59.091833  4390 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:14:59.103492  4390 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:14:59.107704 28199 leader_election.cc:336] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
W20260709 13:14:59.109331 28199 leader_election.cc:336] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:59.109808 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:14:59.104940 28728 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.111128 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:59.111958  4390 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:14:59.124435  4388 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:14:59.150794  4375 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.151396  4375 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:59.156461 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:59.157209 28755 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 26.
I20260709 13:14:59.159298 28198 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:59.160487  4388 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:14:59.160998  4388 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.161486  4388 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:59.165272  4375 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:59.177843 28728 leader_election.cc:336] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
W20260709 13:14:59.179637 28199 consensus_peers.cc:597] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:59.186452 28728 leader_election.cc:336] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:14:59.187312 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:59.190496  4390 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:14:59.183696  4388 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:59.195540  4388 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:59.196532 28199 consensus_peers.cc:597] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:14:59.196878 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d0ccc53d55d4d73baf657119bf89f1c" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f"
I20260709 13:14:59.197499 28755 raft_consensus.cc:3060] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:14:59.202611  4386 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.203261  4386 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:59.212369 28199 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.219283 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:59.223220 27973 maintenance_manager.cc:419] P fea5cdb8fcad442a901284de409f4a11: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033364
I20260709 13:14:59.217470  4386 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:14:59.239118 28199 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
W20260709 13:14:59.241339 28728 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 54912.3us
W20260709 13:14:59.242045 28728 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:14:59.242156 28199 consensus_peers.cc:597] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:59.243943 28755 raft_consensus.cc:2468] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 27.
I20260709 13:14:59.248765 28198 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:59.252990  4388 raft_consensus.cc:2804] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:14:59.261963  4388 raft_consensus.cc:697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 27 LEADER]: Becoming Leader. State: Replica: 11f10c12c03e47dcac88b050d4c1431b, State: Running, Role: LEADER
I20260709 13:14:59.263008  4388 consensus_queue.cc:237] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:59.303539  4375 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.304163  4375 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:59.311736  4375 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
I20260709 13:14:59.319304 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
I20260709 13:14:59.320228 28755 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f10c12c03e47dcac88b050d4c1431b in term 25.
I20260709 13:14:59.361157  4388 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.361826  4388 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:59.367115 28755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "11f10c12c03e47dcac88b050d4c1431b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "70996712c3c942468a6006602de6477f" is_pre_election: true
W20260709 13:14:59.375310 28728 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:14:59.374166  4388 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 70996712c3c942468a6006602de6477f (127.26.83.196:46855)
W20260709 13:14:59.385862 28728 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:14:59.390830 27982 catalog_manager.cc:5697] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b reported cstate change: term changed from 26 to 27, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195). New cstate: current_term: 27 leader_uuid: "11f10c12c03e47dcac88b050d4c1431b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:59.403067  4377 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.403730  4377 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:14:59.407354  4377 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:59.408229 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:14:59.409075 28230 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 21.
I20260709 13:14:59.409415  4388 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.410354 28730 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
I20260709 13:14:59.410076  4388 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:59.411530  4377 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:14:59.411975  4377 raft_consensus.cc:493] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.412451  4377 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:59.412791  4388 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:59.393182 28199 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:59.423892 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.424477 28199 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.425104 28199 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:59.426017  4388 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:14:59.433583 28199 consensus_peers.cc:597] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.434202 28199 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.434659 28199 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 611ec64ec58244fe82058d6130e4e6f3, 70996712c3c942468a6006602de6477f
I20260709 13:14:59.435743  4388 raft_consensus.cc:2749] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:59.425809 28728 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.436388 28728 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:59.438198  4386 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:14:59.447177  4377 raft_consensus.cc:515] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:59.446106 28199 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:59.449741 28199 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.450662  4377 leader_election.cc:290] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Requested vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
I20260709 13:14:59.451871 28230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25d46d07733d450e897e3a72d765e511" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b"
I20260709 13:14:59.454211 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:59.454797 28230 raft_consensus.cc:3060] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:14:59.455049  4388 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:14:59.453660 28728 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:59.458247 28728 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:14:59.465952 28230 raft_consensus.cc:2468] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 22.
W20260709 13:14:59.468466 28728 consensus_peers.cc:597] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:59.468643 28730 leader_election.cc:304] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 
W20260709 13:14:59.469228 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:59.469681  4377 raft_consensus.cc:2804] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:14:59.470517  4377 raft_consensus.cc:697] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Becoming Leader. State: Replica: 70996712c3c942468a6006602de6477f, State: Running, Role: LEADER
W20260709 13:14:59.475643 28199 consensus_peers.cc:597] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.476346 28199 consensus_peers.cc:597] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:59.473980  4377 consensus_queue.cc:237] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:59.488780 28728 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.492653  4390 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
W20260709 13:14:59.494593 28182 debug-util.cc:398] Leaking SignalData structure 0x7b08001f35c0 after lost signal to thread 26962
I20260709 13:14:59.494175  4390 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
W20260709 13:14:59.495815 28182 debug-util.cc:398] Leaking SignalData structure 0x7b080025c1e0 after lost signal to thread 28018
W20260709 13:14:59.496486 28182 debug-util.cc:398] Leaking SignalData structure 0x7b080000fde0 after lost signal to thread 28185
I20260709 13:14:59.496591  4390 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:59.497742 28182 debug-util.cc:398] Leaking SignalData structure 0x7b08000fd000 after lost signal to thread 28259
W20260709 13:14:59.499329 28182 debug-util.cc:398] Leaking SignalData structure 0x7b0800320840 after lost signal to thread 28789
W20260709 13:14:59.497777 28728 leader_election.cc:336] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.509454 27972 maintenance_manager.cc:643] P fea5cdb8fcad442a901284de409f4a11: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.285s	user 0.039s	sys 0.007s Metrics: {"bytes_written":7434,"delete_count":0,"lbm_write_time_us":1174,"lbm_writes_lt_1ms":7,"reinsert_count":0,"update_count":293}
I20260709 13:14:59.511866 27973 maintenance_manager.cc:419] P fea5cdb8fcad442a901284de409f4a11: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.374460
W20260709 13:14:59.522228 28728 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:59.525991 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.526608 28728 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.527143 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:59.528053  4390 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:14:59.531651 28199 consensus_peers.cc:597] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:59.538225 27982 catalog_manager.cc:5697] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f reported cstate change: term changed from 21 to 22, leader changed from 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194) to 70996712c3c942468a6006602de6477f (127.26.83.196). New cstate: current_term: 22 leader_uuid: "70996712c3c942468a6006602de6477f" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:59.548764 28199 consensus_peers.cc:597] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:14:59.556401 28728 consensus_peers.cc:597] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.583266 28199 consensus_peers.cc:597] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.594893 28199 consensus_peers.cc:597] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:59.600946  4388 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:14:59.601675  4388 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:14:59.603965  4388 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:14:59.623522 28728 consensus_peers.cc:597] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.644955 28199 leader_election.cc:336] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:14:59.653514 28199 leader_election.cc:336] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:14:59.654067 28199 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:14:59.654959  4388 raft_consensus.cc:2749] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:14:59.674250 28728 consensus_peers.cc:597] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:59.685871 27972 maintenance_manager.cc:643] P fea5cdb8fcad442a901284de409f4a11: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.170s	user 0.112s	sys 0.014s Metrics: {"cfile_cache_miss":10,"cfile_cache_miss_bytes":28823,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":8919,"dirs.run_cpu_time_us":634,"dirs.run_wall_time_us":2700,"lbm_read_time_us":2342,"lbm_reads_lt_1ms":26,"lbm_write_time_us":1755,"lbm_writes_lt_1ms":12,"peak_mem_usage":313454,"reinsert_count":0,"thread_start_us":16443,"threads_started":2,"update_count":315}
I20260709 13:14:59.688892 27973 maintenance_manager.cc:419] P fea5cdb8fcad442a901284de409f4a11: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:14:59.691350 27972 maintenance_manager.cc:643] P fea5cdb8fcad442a901284de409f4a11: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 13:14:59.704420 28728 consensus_peers.cc:597] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.707526 28199 consensus_peers.cc:597] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.728351 28199 consensus_peers.cc:597] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:14:59.731182 28755 raft_consensus.cc:1275] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Refusing update from remote peer 11f10c12c03e47dcac88b050d4c1431b: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
W20260709 13:14:59.733484 28199 consensus_peers.cc:597] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:14:59.740814  4374 consensus_queue.cc:1048] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:14:59.751755 28728 consensus_peers.cc:597] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.775134 28199 consensus_peers.cc:597] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:59.775782 28199 consensus_peers.cc:597] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.788655 28728 consensus_peers.cc:597] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.818748 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.828706 28199 consensus_peers.cc:597] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.827242 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:14:59.838064 26959 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 28182)
W20260709 13:14:59.838016 28199 consensus_peers.cc:597] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.843220 28199 consensus_peers.cc:597] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:14:59.847534 28728 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:14:59.856943 28199 consensus_peers.cc:597] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.863422 28728 consensus_peers.cc:597] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.924934 28728 consensus_peers.cc:597] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.959203 28199 consensus_peers.cc:597] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.960759 28728 consensus_peers.cc:597] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:14:59.975195 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.988529 28728 consensus_peers.cc:597] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:14:59.998133 28199 consensus_peers.cc:597] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:00.026901  4375 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:00.027530  4375 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:15:00.029709  4375 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:00.040570 28199 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:00.046363 28199 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:00.048774 28199 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:00.049600  4375 raft_consensus.cc:2749] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:00.067193 28231 raft_consensus.cc:1275] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Refusing update from remote peer 70996712c3c942468a6006602de6477f: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:15:00.070370  4377 consensus_queue.cc:1048] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:15:00.128774 28199 consensus_peers.cc:597] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:00.254186 28199 consensus_peers.cc:597] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:00.279223 28728 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:00.316179 28199 consensus_peers.cc:597] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:15:00.337575 28199 consensus_peers.cc:597] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:00.399266 28728 consensus_peers.cc:597] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:15:00.450913  4386 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:00.451516  4386 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:00.455448  4386 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:00.463989 28728 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:00.466114 28728 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:00.466741 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:00.468992  4386 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:15:00.484889 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:00.512018 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:00.529175 28199 consensus_peers.cc:597] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:15:00.533829 28728 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:00.582506  4377 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:00.583180  4377 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:00.586128  4377 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:00.599278 28728 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:00.603396 28728 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:00.603866 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:00.604593  4377 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:00.608938 28728 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:00.618947 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:15:00.679759 28728 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:00.698873  4375 raft_consensus.cc:493] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:00.699501  4375 raft_consensus.cc:515] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:15:00.701601  4375 leader_election.cc:290] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:15:00.709779  4375 raft_consensus.cc:493] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:00.710475  4375 raft_consensus.cc:515] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:15:00.713702  4375 leader_election.cc:290] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:00.711486 28199 leader_election.cc:336] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
W20260709 13:15:00.714907 28199 leader_election.cc:336] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:15:00.715446 28199 leader_election.cc:304] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:00.717481  4375 raft_consensus.cc:2749] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:15:00.733403 28199 leader_election.cc:336] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
W20260709 13:15:00.734603 28199 leader_election.cc:336] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:15:00.735136 28199 leader_election.cc:304] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:00.735989  4375 raft_consensus.cc:2749] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:00.821090  4386 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:00.821768  4386 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:00.824393  4386 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:00.833532 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:00.836378 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
W20260709 13:15:00.838596 26959 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 28182)
I20260709 13:15:00.841410 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:00.843187  4386 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:15:00.871438 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:00.959031  4375 raft_consensus.cc:493] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:00.959663  4375 raft_consensus.cc:515] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:15:00.962054  4375 leader_election.cc:290] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:00.967254 28199 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
W20260709 13:15:00.967913 28199 leader_election.cc:336] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:15:00.968313 28199 leader_election.cc:304] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:00.976046  4407 raft_consensus.cc:2749] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:01.020967  4386 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:01.021883  4386 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:01.040077  4386 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:01.045378 28728 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111) [suppressed 653 similar messages]
W20260709 13:15:01.049690 28199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:45587, user_credentials={real_user=slave}} blocked reactor thread for 81793.3us
W20260709 13:15:01.053850 28728 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:01.070760 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:01.071740 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:01.072304 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:01.073355  4377 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:01.088006  4408 raft_consensus.cc:493] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:01.088608  4408 raft_consensus.cc:515] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:15:01.091233  4408 leader_election.cc:290] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:01.097658 28199 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:01.098610 28199 leader_election.cc:336] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:01.099172 28199 leader_election.cc:304] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:01.101083  4408 raft_consensus.cc:2749] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:01.167145  4377 raft_consensus.cc:493] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:01.167779  4377 raft_consensus.cc:515] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } attrs { replace: true } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:01.172024 28231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34353a7e5a8346119cf9f0836165a1b7" candidate_uuid: "70996712c3c942468a6006602de6477f" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "11f10c12c03e47dcac88b050d4c1431b" is_pre_election: true
I20260709 13:15:01.172681 28231 raft_consensus.cc:2468] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70996712c3c942468a6006602de6477f in term 25.
I20260709 13:15:01.181460  4377 leader_election.cc:290] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:15:01.186820 28728 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:01.187788 28728 leader_election.cc:336] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:01.188328 28728 leader_election.cc:304] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b, 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:01.189342  4377 raft_consensus.cc:2749] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:15:01.202736  4386 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:01.203424  4386 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:01.207679  4386 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:01.219837 28728 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:01.230662 28728 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:01.231503 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:01.232764  4377 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:01.254678 28199 consensus_peers.cc:597] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:15:01.265270 28728 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:01.297097  4377 raft_consensus.cc:493] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:01.297747  4377 raft_consensus.cc:515] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:01.300034  4377 leader_election.cc:290] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:01.325718 28728 leader_election.cc:336] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:01.331012 28728 leader_election.cc:336] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:01.337991 28728 leader_election.cc:304] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:01.340073  4375 raft_consensus.cc:493] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:01.340472  4410 raft_consensus.cc:2749] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:01.340811  4375 raft_consensus.cc:515] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:15:01.343052  4375 leader_election.cc:290] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:15:01.344223  4410 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:01.344868  4410 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:01.347399  4410 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:01.353905 28199 leader_election.cc:336] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:01.356487 28199 leader_election.cc:336] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:01.356994 28199 leader_election.cc:304] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:01.357693  4375 raft_consensus.cc:2749] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:01.366211 28728 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:01.367226 28728 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:01.367733 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:01.368643  4410 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:15:01.380908 26959 tablet_server.cc:196] TabletServer@127.26.83.194:0 shutdown complete.
W20260709 13:15:01.402508 28728 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:01.503465 28728 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:15:01.620857 26959 tablet_server.cc:179] TabletServer@127.26.83.195:0 shutting down...
W20260709 13:15:01.629392 28199 consensus_peers.cc:597] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:01.655768 28730 consensus_peers.cc:597] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:01.665405 28730 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:01.679493 28199 consensus_peers.cc:597] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:01.698480 28730 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:01.740181 28730 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.745096 26959 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:01.748443 26959 tablet_replica.cc:333] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.749687 26959 raft_consensus.cc:2243] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.752765 26959 raft_consensus.cc:2272] T 6fbb28a09242477abe442ecc8a938738 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:01.753208 28730 consensus_peers.cc:597] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.756887 26959 tablet_replica.cc:333] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.758338 26959 raft_consensus.cc:2243] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.759063 26959 raft_consensus.cc:2272] T 61f69a3a4f9a410e9fc9d6e9c38de9f1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.762430 26959 tablet_replica.cc:333] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.763450 26959 raft_consensus.cc:2243] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.765939 26959 raft_consensus.cc:2272] T 66540b865ca24721910c667ccd790b77 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.770177 26959 tablet_replica.cc:333] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.771081 26959 raft_consensus.cc:2243] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.772383 26959 raft_consensus.cc:2272] T ceb515ffdfa04f2f977a4c5aa1aecbee P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.775247 26959 tablet_replica.cc:333] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.776058 26959 raft_consensus.cc:2243] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.779220 26959 raft_consensus.cc:2272] T 6ef726aecabb47089dcd217bfb8bfefc P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:01.782517 28730 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.783663  4411 raft_consensus.cc:493] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:01.782598 26959 tablet_replica.cc:333] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.784302  4411 raft_consensus.cc:515] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:15:01.787911  4411 leader_election.cc:290] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:01.788689 28728 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:15:01.789665 26959 raft_consensus.cc:2243] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.792532 26959 raft_consensus.cc:2272] T 60a7d81169af4acc91d8f3b342ac4885 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:01.792798 28199 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:01.797219 26959 tablet_replica.cc:333] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.799392 26959 raft_consensus.cc:2243] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.800071 26959 raft_consensus.cc:2272] T c20222bc64224781bd97e8b75f2ddfa1 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.802807 26959 tablet_replica.cc:333] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.803763 26959 raft_consensus.cc:2243] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.804538 26959 raft_consensus.cc:2272] T 9f16469a2e44408daf3969b51ae89aa6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.809686 26959 tablet_replica.cc:333] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.810590 26959 raft_consensus.cc:2243] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.811823 26959 raft_consensus.cc:2272] T 6810464df4a245f49df908fc877020d7 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.814929 26959 tablet_replica.cc:333] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.815721 26959 raft_consensus.cc:2243] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.816510 26959 raft_consensus.cc:2272] T a732ff5937ea47eaaef6755cac5ad60e P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.819702 26959 tablet_replica.cc:333] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.820629 26959 raft_consensus.cc:2243] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.821744 26959 raft_consensus.cc:2272] T fc1588fd3df04c2c8354baa741ddeca9 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.824958 26959 tablet_replica.cc:333] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.825858 26959 raft_consensus.cc:2243] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.826597 26959 raft_consensus.cc:2272] T b7ee90e83c2f49509f54dc87ea4d50e6 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.829885 26959 tablet_replica.cc:333] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.830957 26959 raft_consensus.cc:2243] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.832679 26959 raft_consensus.cc:2272] T 4b129d69618f48999f257e73b5b8b1d8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:01.834923 28730 consensus_peers.cc:597] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.835538 26959 tablet_replica.cc:333] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.836508 26959 raft_consensus.cc:2243] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.838066 26959 raft_consensus.cc:2272] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.840962 26959 tablet_replica.cc:333] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.841820 26959 raft_consensus.cc:2243] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.842806 26959 raft_consensus.cc:2272] T b99bb21dbacb4d63839f1403ef9b08b9 P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.845669 26959 tablet_replica.cc:333] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.846470 26959 raft_consensus.cc:2243] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.847332 26959 raft_consensus.cc:2272] T 8d3461aa17604e69b08928a9bc0f8d18 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.849905 26959 tablet_replica.cc:333] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.850701 26959 raft_consensus.cc:2243] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.851361 26959 raft_consensus.cc:2272] T fab37ead976b4378b1298941cec677a8 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.854008 26959 tablet_replica.cc:333] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.854763 26959 raft_consensus.cc:2243] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.855602 26959 raft_consensus.cc:2272] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.858170 26959 tablet_replica.cc:333] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.858918 26959 raft_consensus.cc:2243] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.860256 26959 raft_consensus.cc:2272] T 798c63c7bb2e4373af20f55bbe872cf2 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.863094 26959 tablet_replica.cc:333] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.863870 26959 raft_consensus.cc:2243] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.864475 26959 raft_consensus.cc:2272] T c0a4a73cb5754c40af548cd4519da5b4 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.867132 26959 tablet_replica.cc:333] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.867928 26959 raft_consensus.cc:2243] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.868800 26959 raft_consensus.cc:2272] T 33b72fb1e20942d9a3907c50252930bb P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:01.869439 28730 consensus_peers.cc:597] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.871404 26959 tablet_replica.cc:333] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.872306 26959 raft_consensus.cc:2243] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.873090 26959 raft_consensus.cc:2272] T 7c0743024f324c14b4c6fc08f70600d2 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.874141  4411 raft_consensus.cc:493] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:01.874821  4411 raft_consensus.cc:515] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } }
I20260709 13:15:01.876416 26959 tablet_replica.cc:333] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.877393 26959 raft_consensus.cc:2243] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.878408 26959 raft_consensus.cc:2272] T 34353a7e5a8346119cf9f0836165a1b7 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.881333 26959 tablet_replica.cc:333] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.882272 26959 raft_consensus.cc:2243] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.883441 26959 raft_consensus.cc:2272] T 6a669edb54e3494387ede68c8acd8422 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.886958 26959 tablet_replica.cc:333] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.887777 26959 raft_consensus.cc:2243] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.888373  4411 leader_election.cc:290] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:15:01.889180 26959 raft_consensus.cc:2272] T 2d0ccc53d55d4d73baf657119bf89f1c P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:01.891121 28199 leader_election.cc:336] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:01.892839 26959 tablet_replica.cc:333] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.893695 26959 raft_consensus.cc:2243] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.894711 26959 raft_consensus.cc:2272] T 2929b1e07bb74c67863e4c504d8d09a6 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.897367 26959 tablet_replica.cc:333] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.898267 26959 raft_consensus.cc:2243] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.898845 26959 raft_consensus.cc:2272] T 030bc6df8e7e4f2da3b18613d20d9bdc P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.901549 26959 tablet_replica.cc:333] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.902493 26959 raft_consensus.cc:2243] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.903367 26959 raft_consensus.cc:2272] T e53fd092d3dd48c492e04080969cb666 P 11f10c12c03e47dcac88b050d4c1431b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.906064 26959 tablet_replica.cc:333] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.906922 26959 raft_consensus.cc:2243] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.907737 26959 raft_consensus.cc:2272] T 923fa739e01e4f05a05099628c53a6c3 P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.910632 26959 tablet_replica.cc:333] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.911382 26959 raft_consensus.cc:2243] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.912607 26959 raft_consensus.cc:2272] T c71e4401306a4e84a111e5443be7652c P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.915242 26959 tablet_replica.cc:333] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.915963 26959 raft_consensus.cc:2243] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.916880 26959 raft_consensus.cc:2272] T c5b0e8aadf244e7eb41e7e732bc27cf9 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.919378 26959 tablet_replica.cc:333] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.920125 26959 raft_consensus.cc:2243] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.921223 26959 raft_consensus.cc:2272] T 8ada0581aa09483e91a300220a129859 P 11f10c12c03e47dcac88b050d4c1431b [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:01.921658 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:01.923511 28730 consensus_peers.cc:597] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:01.925441 28730 consensus_peers.cc:597] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.923993 26959 tablet_replica.cc:333] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
W20260709 13:15:01.928254 28728 consensus_peers.cc:597] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:01.929366 26959 raft_consensus.cc:2243] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.930650 26959 raft_consensus.cc:2272] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.934592 26959 tablet_replica.cc:333] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.935874 26959 raft_consensus.cc:2243] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.937402 26959 raft_consensus.cc:2272] T 7e272db0632a4262a90eaacd6e6d4378 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.940673 26959 tablet_replica.cc:333] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.941512 26959 raft_consensus.cc:2243] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.942598 26959 raft_consensus.cc:2272] T 5c654c99561449cb823642a1b85879ac P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.945430 26959 tablet_replica.cc:333] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
W20260709 13:15:01.957739 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:01.962143 28730 consensus_peers.cc:597] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.959704 26959 raft_consensus.cc:2243] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 24 LEADER]: Raft consensus shutting down.
W20260709 13:15:01.965960 28728 consensus_peers.cc:597] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:01.967549 26959 raft_consensus.cc:2272] T d65f83feb5bb4b468e71fc5b9b708de4 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.970911 26959 tablet_replica.cc:333] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.971750 26959 raft_consensus.cc:2243] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.973032 26959 raft_consensus.cc:2272] T 25d46d07733d450e897e3a72d765e511 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.975646 26959 tablet_replica.cc:333] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
W20260709 13:15:01.975816 28730 consensus_peers.cc:597] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:01.978663 27969 catalog_manager.cc:4755] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 7e272db0632a4262a90eaacd6e6d4378: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:15:01.976471 26959 raft_consensus.cc:2243] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:15:01.981499 26959 raft_consensus.cc:2272] T 2facf6a0a91849a5a177c2481e42aa47 P 11f10c12c03e47dcac88b050d4c1431b [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:01.982698 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.984412 26959 tablet_replica.cc:333] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
W20260709 13:15:01.984516 28730 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:01.985352 28730 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.985373 26959 raft_consensus.cc:2243] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.986505 26959 raft_consensus.cc:2272] T 6b9a6ec9a3a1422b8723b26150db560f P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.989389 26959 tablet_replica.cc:333] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.990235 26959 raft_consensus.cc:2243] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:15:01.990651 28730 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.991024 26959 raft_consensus.cc:2272] T 9145e2ecf82c4a03b0034029f10bab74 P 11f10c12c03e47dcac88b050d4c1431b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:01.993928 26959 tablet_replica.cc:333] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.994702 26959 raft_consensus.cc:2243] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.995416 26959 raft_consensus.cc:2272] T 53c6cdf8c6704ee59d0a736273582a48 P 11f10c12c03e47dcac88b050d4c1431b [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:01.995937 28730 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:01.998308 26959 tablet_replica.cc:333] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:01.999156 26959 raft_consensus.cc:2243] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:01.999912 26959 raft_consensus.cc:2272] T d56e9de7a9c54453bc5fc3aac0fa589c P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:02.000061 28730 consensus_peers.cc:597] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:02.002725 26959 tablet_replica.cc:333] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:02.003481 26959 raft_consensus.cc:2243] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:02.004196 26959 raft_consensus.cc:2272] T 4c9b061a7ca84a61ac13d29946a28169 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:02.007025 26959 tablet_replica.cc:333] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:02.007740 26959 raft_consensus.cc:2243] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:02.008512 26959 raft_consensus.cc:2272] T 552d81eee50848758aae23da5ed8927b P 11f10c12c03e47dcac88b050d4c1431b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:02.011161 26959 tablet_replica.cc:333] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:02.011880 26959 raft_consensus.cc:2243] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:02.012984 26959 raft_consensus.cc:2272] T 9e12858e91dd456a9f422bca6d2d6ac3 P 11f10c12c03e47dcac88b050d4c1431b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:02.015692 26959 tablet_replica.cc:333] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:02.016392 26959 raft_consensus.cc:2243] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:02.017216 26959 raft_consensus.cc:2272] T a8381961442846ab97965dc11b100d75 P 11f10c12c03e47dcac88b050d4c1431b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:02.020035 26959 tablet_replica.cc:333] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:02.020654 26959 raft_consensus.cc:2243] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:02.021245 26959 raft_consensus.cc:2272] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 11f10c12c03e47dcac88b050d4c1431b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:02.023850 26959 tablet_replica.cc:333] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b: stopping tablet replica
I20260709 13:15:02.024462 26959 raft_consensus.cc:2243] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:02.025277 26959 raft_consensus.cc:2272] T dd7077e3c7d94184913764cd90b92757 P 11f10c12c03e47dcac88b050d4c1431b [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:02.035926 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.039842 28728 consensus_peers.cc:597] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:02.043962  4410 raft_consensus.cc:493] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:02.044687  4410 raft_consensus.cc:515] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:02.047997  4410 leader_election.cc:290] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:02.059661 28728 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:02.067061 28728 leader_election.cc:336] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:02.068594 28728 leader_election.cc:304] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:02.071643  4410 raft_consensus.cc:2749] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:15:02.076205 28730 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:02.078997 28730 consensus_peers.cc:597] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:02.082289 28730 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:02.095688 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.097997 28730 consensus_peers.cc:597] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:02.100502 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.100327 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.110423 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.146441 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.201573 28730 consensus_peers.cc:597] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:02.205523 28728 consensus_peers.cc:597] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.206496 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.217928 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.218977 28730 consensus_peers.cc:597] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f -> Peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Couldn't send request to peer 11f10c12c03e47dcac88b050d4c1431b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:02.226701 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.232980 28728 consensus_peers.cc:597] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.240767 28198 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.252058 28728 consensus_peers.cc:597] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.257314 28728 consensus_peers.cc:597] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.278045 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.289474 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.306916 28201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.351272 28728 consensus_peers.cc:597] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.351853 28728 consensus_peers.cc:597] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.366825 28728 consensus_peers.cc:597] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:02.369390 28728 consensus_peers.cc:597] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:02.399632 28728 consensus_peers.cc:597] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.415597 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.502142 28199 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.517903 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.526883 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.546411  4418 leader_election.cc:336] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Service unavailable: reactor is shutting down (error 108)
I20260709 13:15:02.547132  4418 leader_election.cc:304] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:15:02.547933  4418 raft_consensus.cc:2686] T 8addb79d6158437f8d7a4822a75661ab P 11f10c12c03e47dcac88b050d4c1431b: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:15:02.555490 28200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.554188  4418 leader_election.cc:336] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Service unavailable: reactor is shutting down (error 108)
I20260709 13:15:02.564555  4418 leader_election.cc:304] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11f10c12c03e47dcac88b050d4c1431b; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
W20260709 13:15:02.565713  4418 raft_consensus.cc:2686] T 17fbe81a71fd4f8c83a65145cf2f0682 P 11f10c12c03e47dcac88b050d4c1431b: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:15:02.572510 28728 consensus_peers.cc:597] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:15:02.577884  4417 raft_consensus.cc:493] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:02.578516  4417 raft_consensus.cc:515] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:02.580878  4417 leader_election.cc:290] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:02.598481 28728 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:02.599860 28728 leader_election.cc:336] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:02.607688 28728 leader_election.cc:304] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:02.608466  4420 raft_consensus.cc:2749] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:02.639537 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:15:02.670912 28728 consensus_peers.cc:597] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f -> Peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Couldn't send request to peer 611ec64ec58244fe82058d6130e4e6f3. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:02.729547 28728 consensus_peers.cc:597] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:02.769980  4377 raft_consensus.cc:493] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:02.770552  4377 raft_consensus.cc:515] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:02.772629  4377 leader_election.cc:290] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:02.803274 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:02.804345 28728 leader_election.cc:336] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:02.804939 28728 leader_election.cc:304] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:02.805946  4417 raft_consensus.cc:2749] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:15:02.896919  4417 raft_consensus.cc:493] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:15:02.897595  4417 raft_consensus.cc:515] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } attrs { replace: true } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:02.900679  4417 leader_election.cc:290] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:15:02.906589 28730 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Network error: Client connection negotiation failed: client connection to 127.26.83.195:46875: connect: Connection refused (error 111)
W20260709 13:15:02.931767 28728 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:15:02.932349 28728 leader_election.cc:304] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 11f10c12c03e47dcac88b050d4c1431b
I20260709 13:15:02.941339  4417 raft_consensus.cc:2749] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:02.942934 28728 leader_election.cc:336] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:02.991948  4377 raft_consensus.cc:493] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:15:02.992450  4377 raft_consensus.cc:515] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:02.994401  4377 leader_election.cc:290] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:15:03.002962 28730 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Network error: Client connection negotiation failed: client connection to 127.26.83.195:46875: connect: Connection refused (error 111)
W20260709 13:15:03.031203 28728 consensus_peers.cc:597] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:03.035681 28728 leader_election.cc:336] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:03.036155 28728 leader_election.cc:304] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:03.036952  4377 raft_consensus.cc:2749] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:15:03.051707 28728 consensus_peers.cc:597] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:03.054085  4377 raft_consensus.cc:493] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 11f10c12c03e47dcac88b050d4c1431b)
I20260709 13:15:03.054621  4377 raft_consensus.cc:515] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:03.057165  4377 leader_election.cc:290] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757), 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875)
W20260709 13:15:03.062902 28730 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 11f10c12c03e47dcac88b050d4c1431b (127.26.83.195:46875): Network error: Client connection negotiation failed: client connection to 127.26.83.195:46875: connect: Connection refused (error 111)
W20260709 13:15:03.071353 28728 consensus_peers.cc:597] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:15:03.071888 28728 consensus_peers.cc:597] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:03.077281 28728 leader_election.cc:336] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:03.077752 28728 leader_election.cc:304] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 11f10c12c03e47dcac88b050d4c1431b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:03.078616  4377 raft_consensus.cc:2749] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:15:03.081813 28728 consensus_peers.cc:597] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:03.145749 28728 consensus_peers.cc:597] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:15:03.155298 28728 consensus_peers.cc:597] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:15:03.368867 28728 consensus_peers.cc:597] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:15:03.535784 26959 tablet_server.cc:196] TabletServer@127.26.83.195:0 shutdown complete.
W20260709 13:15:03.598217 28728 consensus_peers.cc:597] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:15:03.640609  4421 raft_consensus.cc:493] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:03.641235  4421 raft_consensus.cc:515] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:03.643492  4421 leader_election.cc:290] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:03.665948 28728 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:03.666857 28728 leader_election.cc:336] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:03.667366 28728 leader_election.cc:304] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:03.668195  4377 raft_consensus.cc:2749] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:03.702952 26959 tablet_server.cc:179] TabletServer@127.26.83.196:0 shutting down...
I20260709 13:15:03.721174  4377 raft_consensus.cc:493] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:03.721726  4377 raft_consensus.cc:515] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:03.723963  4377 leader_election.cc:290] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
W20260709 13:15:03.730170 28728 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:03.732478 28728 leader_election.cc:336] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:03.732971 28728 leader_election.cc:304] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:03.733850  4377 raft_consensus.cc:2749] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:15:03.787966 26959 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:03.789166 26959 tablet_replica.cc:333] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.790390 26959 raft_consensus.cc:2243] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.791298 26959 raft_consensus.cc:2272] T 7e272db0632a4262a90eaacd6e6d4378 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:03.793427 28728 consensus_peers.cc:597] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:15:03.794351  4421 raft_consensus.cc:493] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 611ec64ec58244fe82058d6130e4e6f3)
I20260709 13:15:03.795158 26959 tablet_replica.cc:333] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.794888  4421 raft_consensus.cc:515] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: false } }
I20260709 13:15:03.796243 26959 raft_consensus.cc:2243] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.797201  4421 leader_election.cc:290] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587), 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757)
I20260709 13:15:03.797621 26959 raft_consensus.cc:2272] T 25d46d07733d450e897e3a72d765e511 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.801136 26959 tablet_replica.cc:333] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.802372 26959 raft_consensus.cc:2243] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.803942 26959 raft_consensus.cc:2272] T 5c654c99561449cb823642a1b85879ac P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.807242 26959 tablet_replica.cc:333] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.808213 26959 raft_consensus.cc:2243] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.809283 26959 raft_consensus.cc:2272] T b99bb21dbacb4d63839f1403ef9b08b9 P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.812572 26959 tablet_replica.cc:333] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.813618 26959 raft_consensus.cc:2243] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f [term 4 LEARNER]: Raft consensus shutting down.
I20260709 13:15:03.814366 26959 pending_rounds.cc:70] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Trying to abort 2 pending ops.
I20260709 13:15:03.814704 26959 pending_rounds.cc:77] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305636679895769088 op_type: NO_OP noop_request { }
I20260709 13:15:03.815162 26959 raft_consensus.cc:2889] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:03.815523 26959 pending_rounds.cc:77] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f: Aborting op as it isn't in flight: id { term: 2 index: 3 } timestamp: 7305636646613749760 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "030bc6df8e7e4f2da3b18613d20d9bdc" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0faaace2b36c4bcebf2fa6835235e92b" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 45587 } attrs { replace: true } } peers { permanent_uuid: "611ec64ec58244fe82058d6130e4e6f3" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 38757 } } peers { permanent_uuid: "11f10c12c03e47dcac88b050d4c1431b" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 46875 } } peers { permanent_uuid: "70996712c3c942468a6006602de6477f" member_type: NON_VOTER last_known_addr { host: "127.26.83.196" port: 46855 } attrs { promote: true } } } }
I20260709 13:15:03.816757 26959 raft_consensus.cc:2917] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f [term 4 LEARNER]: Aborting config change with OpId 2.3: Aborted: Op aborted
I20260709 13:15:03.817236 26959 raft_consensus.cc:2889] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f [term 4 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:15:03.817664 26959 raft_consensus.cc:2272] T 030bc6df8e7e4f2da3b18613d20d9bdc P 70996712c3c942468a6006602de6477f [term 4 NON_PARTICIPANT]: Raft consensus is shut down!
I20260709 13:15:03.820902 26959 tablet_replica.cc:333] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.822053 26959 raft_consensus.cc:2243] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.823104 26959 raft_consensus.cc:2272] T 888e64754b8d47aebb2c7c3aba7b0ee2 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:03.824472 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
W20260709 13:15:03.825740 28728 leader_election.cc:336] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 611ec64ec58244fe82058d6130e4e6f3 (127.26.83.194:38757): Network error: Client connection negotiation failed: client connection to 127.26.83.194:38757: connect: Connection refused (error 111)
I20260709 13:15:03.826220 28728 leader_election.cc:304] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70996712c3c942468a6006602de6477f; no voters: 0faaace2b36c4bcebf2fa6835235e92b, 611ec64ec58244fe82058d6130e4e6f3
I20260709 13:15:03.826594 26959 tablet_replica.cc:333] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.827049  4421 raft_consensus.cc:2749] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:03.827741 26959 raft_consensus.cc:2243] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.829052 26959 raft_consensus.cc:2272] T b7ee90e83c2f49509f54dc87ea4d50e6 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.832546 26959 tablet_replica.cc:333] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.833519 26959 raft_consensus.cc:2243] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.834273 26959 raft_consensus.cc:2272] T e6b1c27f68974f4a9878c7c155cb43d2 P 70996712c3c942468a6006602de6477f [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.837828 26959 tablet_replica.cc:333] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.838816 26959 raft_consensus.cc:2243] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.840150 26959 raft_consensus.cc:2272] T 2929b1e07bb74c67863e4c504d8d09a6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.843472 26959 tablet_replica.cc:333] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.844477 26959 raft_consensus.cc:2243] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.845899 26959 raft_consensus.cc:2272] T 847d15cbd80f44b18e547291eed25617 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.849321 26959 tablet_replica.cc:333] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.850384 26959 raft_consensus.cc:2243] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.851897 26959 raft_consensus.cc:2272] T a732ff5937ea47eaaef6755cac5ad60e P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.855414 26959 tablet_replica.cc:333] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.856379 26959 raft_consensus.cc:2243] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.857604 26959 raft_consensus.cc:2272] T d952e9d07d9e4c5ba47d0f8a9601dc73 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.860920 26959 tablet_replica.cc:333] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.861910 26959 raft_consensus.cc:2243] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.863197 26959 raft_consensus.cc:2272] T dd7077e3c7d94184913764cd90b92757 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.866336 26959 tablet_replica.cc:333] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.867336 26959 raft_consensus.cc:2243] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.868562 26959 raft_consensus.cc:2272] T c20222bc64224781bd97e8b75f2ddfa1 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.871798 26959 tablet_replica.cc:333] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.872861 26959 raft_consensus.cc:2243] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.873649 26959 raft_consensus.cc:2272] T f703ac7a390a45efae315d629c1dad89 P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.876822 26959 tablet_replica.cc:333] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.877763 26959 raft_consensus.cc:2243] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.878489 26959 raft_consensus.cc:2272] T 2d0ccc53d55d4d73baf657119bf89f1c P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.881758 26959 tablet_replica.cc:333] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.882745 26959 raft_consensus.cc:2243] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.884209 26959 raft_consensus.cc:2272] T c0a4a73cb5754c40af548cd4519da5b4 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.887512 26959 tablet_replica.cc:333] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.888652 26959 raft_consensus.cc:2243] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.889621 26959 raft_consensus.cc:2272] T 1ce6e6dfcbee43d5b0550ce7fc634c38 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.893128 26959 tablet_replica.cc:333] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.894207 26959 raft_consensus.cc:2243] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.895937 26959 raft_consensus.cc:2272] T aa289960fe9a4d2c8e61deb822de8412 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.899720 26959 tablet_replica.cc:333] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.900807 26959 raft_consensus.cc:2243] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.902139 26959 raft_consensus.cc:2272] T d8d07cbc383b4089b326bc02e8b33622 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:03.902419 28728 consensus_peers.cc:597] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f -> Peer 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): Couldn't send request to peer 0faaace2b36c4bcebf2fa6835235e92b. Status: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:15:03.905946 26959 tablet_replica.cc:333] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.906993 26959 raft_consensus.cc:2243] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.908432 26959 raft_consensus.cc:2272] T 60a7d81169af4acc91d8f3b342ac4885 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.911835 26959 tablet_replica.cc:333] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.912949 26959 raft_consensus.cc:2243] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.913832 26959 raft_consensus.cc:2272] T 34353a7e5a8346119cf9f0836165a1b7 P 70996712c3c942468a6006602de6477f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.917678 26959 tablet_replica.cc:333] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.918777 26959 raft_consensus.cc:2243] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.920135 26959 raft_consensus.cc:2272] T 33b72fb1e20942d9a3907c50252930bb P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.924162 26959 tablet_replica.cc:333] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.925185 26959 raft_consensus.cc:2243] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.926431 26959 raft_consensus.cc:2272] T 552d81eee50848758aae23da5ed8927b P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.929679 26959 tablet_replica.cc:333] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.930598 26959 raft_consensus.cc:2243] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.931463 26959 raft_consensus.cc:2272] T cd81a684ef6b4e0f9cc48bc3bcc280e8 P 70996712c3c942468a6006602de6477f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.934597 26959 tablet_replica.cc:333] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.935560 26959 raft_consensus.cc:2243] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.936827 26959 raft_consensus.cc:2272] T fab37ead976b4378b1298941cec677a8 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.939862 26959 tablet_replica.cc:333] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.940788 26959 raft_consensus.cc:2243] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.941444 26959 raft_consensus.cc:2272] T 1115d5b02e774b14935e3f48488f82ea P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.944247 26959 tablet_replica.cc:333] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.945128 26959 raft_consensus.cc:2243] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.945921 26959 raft_consensus.cc:2272] T 0720cab120de4047b423cb89ed85154a P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.949313 26959 tablet_replica.cc:333] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.950227 26959 raft_consensus.cc:2243] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.951546 26959 raft_consensus.cc:2272] T 6b9a6ec9a3a1422b8723b26150db560f P 70996712c3c942468a6006602de6477f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.954718 26959 tablet_replica.cc:333] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.955601 26959 raft_consensus.cc:2243] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.956925 26959 raft_consensus.cc:2272] T 9145e2ecf82c4a03b0034029f10bab74 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.960070 26959 tablet_replica.cc:333] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.960970 26959 raft_consensus.cc:2243] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.962230 26959 raft_consensus.cc:2272] T 6810464df4a245f49df908fc877020d7 P 70996712c3c942468a6006602de6477f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.965427 26959 tablet_replica.cc:333] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.966324 26959 raft_consensus.cc:2243] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.967476 26959 raft_consensus.cc:2272] T d56e9de7a9c54453bc5fc3aac0fa589c P 70996712c3c942468a6006602de6477f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.971174 26959 tablet_replica.cc:333] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.972003 26959 raft_consensus.cc:2243] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.973246 26959 raft_consensus.cc:2272] T 6f5f1be456de4894af76dffdad9ea6a3 P 70996712c3c942468a6006602de6477f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.976234 26959 tablet_replica.cc:333] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.977149 26959 raft_consensus.cc:2243] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:03.977895 26959 raft_consensus.cc:2272] T e09b3a74da19442187d6c447c95018a9 P 70996712c3c942468a6006602de6477f [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.980993 26959 tablet_replica.cc:333] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.981870 26959 raft_consensus.cc:2243] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.983171 26959 raft_consensus.cc:2272] T e53fd092d3dd48c492e04080969cb666 P 70996712c3c942468a6006602de6477f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.986389 26959 tablet_replica.cc:333] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.987222 26959 raft_consensus.cc:2243] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.988456 26959 raft_consensus.cc:2272] T 923fa739e01e4f05a05099628c53a6c3 P 70996712c3c942468a6006602de6477f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.991425 26959 tablet_replica.cc:333] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.992242 26959 raft_consensus.cc:2243] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.993643 26959 raft_consensus.cc:2272] T 53c6cdf8c6704ee59d0a736273582a48 P 70996712c3c942468a6006602de6477f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:03.996881 26959 tablet_replica.cc:333] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:03.997611 26959 raft_consensus.cc:2243] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:15:03.998742 26959 raft_consensus.cc:2272] T 7c0743024f324c14b4c6fc08f70600d2 P 70996712c3c942468a6006602de6477f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:04.001735 26959 tablet_replica.cc:333] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f: stopping tablet replica
I20260709 13:15:04.002413 26959 raft_consensus.cc:2243] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:15:04.003683 26959 raft_consensus.cc:2272] T 9f16469a2e44408daf3969b51ae89aa6 P 70996712c3c942468a6006602de6477f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:04.930979 26959 tablet_server.cc:196] TabletServer@127.26.83.196:0 shutdown complete.
I20260709 13:15:05.146447 26959 master.cc:562] Master@127.26.83.254:42913 shutting down...
W20260709 13:15:05.218344 28032 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:15:05.218672 28032 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:15:05.218952 28032 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:15:05.219660 28032 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:15:05.219925 28032 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:15:05.220122 28032 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:15:06.652612 28015 debug-util.cc:398] Leaking SignalData structure 0x7b0800340120 after lost signal to thread 26962
W20260709 13:15:06.653352 28015 debug-util.cc:398] Leaking SignalData structure 0x7b080023b340 after lost signal to thread 28018
W20260709 13:15:10.210134 27969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111) [suppressed 376 similar messages]
W20260709 13:15:10.214787 27969 catalog_manager.cc:4755] TS 0faaace2b36c4bcebf2fa6835235e92b (127.26.83.193:45587): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 7e272db0632a4262a90eaacd6e6d4378: Network error: Client connection negotiation failed: client connection to 127.26.83.193:45587: connect: Connection refused (error 111)
I20260709 13:15:10.841872 26959 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:10.842998 26959 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:10.843436 26959 tablet_replica.cc:333] T 00000000000000000000000000000000 P fea5cdb8fcad442a901284de409f4a11: stopping tablet replica
I20260709 13:15:11.051084 26959 master.cc:584] Master@127.26.83.254:42913 shutdown complete.
I20260709 13:15:11.100040 26959 test_util.cc:182] -----------------------------------------------
I20260709 13:15:11.100291 26959 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602621086139-26959-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (265777 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:15:11.104180 26959 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.83.254:38601
I20260709 13:15:11.105466 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:11.112222  4427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:11.112370  4426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:11.113286  4429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:11.113888 26959 server_base.cc:1061] running on GCE node
I20260709 13:15:11.117673 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:11.117967 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:11.118112 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602911118099 us; error 0 us; skew 500 ppm
I20260709 13:15:11.118618 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:11.121680 26959 webserver.cc:533] Webserver started at http://127.26.83.254:39881/ using document root <none> and password file <none>
I20260709 13:15:11.122213 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:11.122391 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:11.122633 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:11.124045 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/master-0-root/instance:
uuid: "76a355c1f71f4850a9d06c919f2b5bee"
format_stamp: "Formatted at 2026-07-09 13:15:11 on dist-test-slave-329l"
I20260709 13:15:11.129438 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:15:11.133666  4434 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.134547 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:15:11.134907 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/master-0-root
uuid: "76a355c1f71f4850a9d06c919f2b5bee"
format_stamp: "Formatted at 2026-07-09 13:15:11 on dist-test-slave-329l"
I20260709 13:15:11.135236 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:11.158545 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:11.159942 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:11.204571 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.254:38601
I20260709 13:15:11.204813  4485 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.254:38601 every 8 connection(s)
I20260709 13:15:11.209023  4486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:11.220707  4486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee: Bootstrap starting.
I20260709 13:15:11.225502  4486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:11.230244  4486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee: No bootstrap required, opened a new log
I20260709 13:15:11.232465  4486 raft_consensus.cc:359] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a355c1f71f4850a9d06c919f2b5bee" member_type: VOTER }
I20260709 13:15:11.232971  4486 raft_consensus.cc:385] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:11.233204  4486 raft_consensus.cc:740] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76a355c1f71f4850a9d06c919f2b5bee, State: Initialized, Role: FOLLOWER
I20260709 13:15:11.233841  4486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a355c1f71f4850a9d06c919f2b5bee" member_type: VOTER }
I20260709 13:15:11.234272  4486 raft_consensus.cc:399] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:15:11.234565  4486 raft_consensus.cc:493] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:15:11.234833  4486 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:11.239981  4486 raft_consensus.cc:515] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a355c1f71f4850a9d06c919f2b5bee" member_type: VOTER }
I20260709 13:15:11.240567  4486 leader_election.cc:304] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76a355c1f71f4850a9d06c919f2b5bee; no voters: 
I20260709 13:15:11.241910  4486 leader_election.cc:290] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:15:11.242297  4489 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:11.244005  4489 raft_consensus.cc:697] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [term 1 LEADER]: Becoming Leader. State: Replica: 76a355c1f71f4850a9d06c919f2b5bee, State: Running, Role: LEADER
I20260709 13:15:11.244845  4489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a355c1f71f4850a9d06c919f2b5bee" member_type: VOTER }
I20260709 13:15:11.245398  4486 sys_catalog.cc:565] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:15:11.251757  4491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76a355c1f71f4850a9d06c919f2b5bee. Latest consensus state: current_term: 1 leader_uuid: "76a355c1f71f4850a9d06c919f2b5bee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a355c1f71f4850a9d06c919f2b5bee" member_type: VOTER } }
I20260709 13:15:11.252820  4491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:11.254874  4490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76a355c1f71f4850a9d06c919f2b5bee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76a355c1f71f4850a9d06c919f2b5bee" member_type: VOTER } }
I20260709 13:15:11.255537  4490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:11.259390  4499 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:15:11.264261  4499 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:15:11.265830 26959 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:15:11.273715  4499 catalog_manager.cc:1383] Generated new cluster ID: 5a1c09161aba49f7b7dd60b936014334
I20260709 13:15:11.274026  4499 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:15:11.291101  4499 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:15:11.292536  4499 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:15:11.306730  4499 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 76a355c1f71f4850a9d06c919f2b5bee: Generated new TSK 0
I20260709 13:15:11.307368  4499 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:15:11.333012 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:11.340126  4507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:11.341566  4508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:11.344611 26959 server_base.cc:1061] running on GCE node
W20260709 13:15:11.345009  4510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:11.347401 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:11.347671 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:11.347873 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602911347849 us; error 0 us; skew 500 ppm
I20260709 13:15:11.348553 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:11.351297 26959 webserver.cc:533] Webserver started at http://127.26.83.193:34901/ using document root <none> and password file <none>
I20260709 13:15:11.351855 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:11.352077 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:11.352388 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:11.353744 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/instance:
uuid: "978db1e31e16405182e65e88f3b9bebc"
format_stamp: "Formatted at 2026-07-09 13:15:11 on dist-test-slave-329l"
I20260709 13:15:11.359045 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:15:11.362977  4515 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.364082 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:15:11.364570 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-0-root
uuid: "978db1e31e16405182e65e88f3b9bebc"
format_stamp: "Formatted at 2026-07-09 13:15:11 on dist-test-slave-329l"
I20260709 13:15:11.365029 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:11.394594 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:11.396268 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:11.398242 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:11.401109 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:11.401383 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.401680 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:11.401886 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.455418 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.193:36159
I20260709 13:15:11.455542  4577 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.193:36159 every 8 connection(s)
I20260709 13:15:11.461652 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:11.470822  4582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:11.472179  4583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:11.473510  4585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:11.473699 26959 server_base.cc:1061] running on GCE node
I20260709 13:15:11.476662 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:15:11.476861  4578 heartbeater.cc:344] Connected to a master server at 127.26.83.254:38601
W20260709 13:15:11.477068 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:11.477313 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602911477293 us; error 0 us; skew 500 ppm
I20260709 13:15:11.477360  4578 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:11.478286 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:11.478531  4578 heartbeater.cc:507] Master 127.26.83.254:38601 requested a full tablet report, sending...
I20260709 13:15:11.481380 26959 webserver.cc:533] Webserver started at http://127.26.83.194:37225/ using document root <none> and password file <none>
I20260709 13:15:11.481986  4451 ts_manager.cc:194] Registered new tserver with Master: 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:11.482317 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:11.482645 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:11.483011 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:11.484871  4451 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42670
I20260709 13:15:11.484947 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/instance:
uuid: "fb69c26178814cacb4f3abd0750d4b82"
format_stamp: "Formatted at 2026-07-09 13:15:11 on dist-test-slave-329l"
I20260709 13:15:11.491267 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:15:11.495836  4590 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.496701 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:15:11.497190 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-1-root
uuid: "fb69c26178814cacb4f3abd0750d4b82"
format_stamp: "Formatted at 2026-07-09 13:15:11 on dist-test-slave-329l"
I20260709 13:15:11.497495 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:11.513633 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:11.514942 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:11.516666 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:11.519270 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:11.519520 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.519788 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:11.519961 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.568697 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.194:35607
I20260709 13:15:11.568840  4652 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.194:35607 every 8 connection(s)
I20260709 13:15:11.573954 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:11.582762  4656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:11.584487  4657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:11.586469  4660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:11.586679 26959 server_base.cc:1061] running on GCE node
I20260709 13:15:11.588559 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:11.588935 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:11.589105  4653 heartbeater.cc:344] Connected to a master server at 127.26.83.254:38601
I20260709 13:15:11.589195 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602911589175 us; error 0 us; skew 500 ppm
I20260709 13:15:11.589744  4653 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:11.590359 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:11.590911  4653 heartbeater.cc:507] Master 127.26.83.254:38601 requested a full tablet report, sending...
I20260709 13:15:11.593711  4451 ts_manager.cc:194] Registered new tserver with Master: fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:11.593719 26959 webserver.cc:533] Webserver started at http://127.26.83.195:45607/ using document root <none> and password file <none>
I20260709 13:15:11.594632 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:11.594892 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:11.595218 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:11.595467  4451 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42678
I20260709 13:15:11.596668 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/instance:
uuid: "117be35051554b3b837c42a5dfcfa38a"
format_stamp: "Formatted at 2026-07-09 13:15:11 on dist-test-slave-329l"
I20260709 13:15:11.602128 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:15:11.606225  4664 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.607455 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:15:11.607950 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-2-root
uuid: "117be35051554b3b837c42a5dfcfa38a"
format_stamp: "Formatted at 2026-07-09 13:15:11 on dist-test-slave-329l"
I20260709 13:15:11.608440 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:11.630986 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:11.632372 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:11.634006 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:11.636966 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:11.637249 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.637537 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:11.637737 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.687747 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.195:42995
I20260709 13:15:11.687851  4726 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.195:42995 every 8 connection(s)
I20260709 13:15:11.693603 26959 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:11.702224  4730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:11.703897  4731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:11.707837  4727 heartbeater.cc:344] Connected to a master server at 127.26.83.254:38601
I20260709 13:15:11.708396  4727 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:11.708799 26959 server_base.cc:1061] running on GCE node
W20260709 13:15:11.709110  4733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:11.709581  4727 heartbeater.cc:507] Master 127.26.83.254:38601 requested a full tablet report, sending...
I20260709 13:15:11.710368 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:11.710752 26959 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:11.711011 26959 hybrid_clock.cc:648] HybridClock initialized: now 1783602911710983 us; error 0 us; skew 500 ppm
I20260709 13:15:11.711714 26959 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:11.711903  4451 ts_manager.cc:194] Registered new tserver with Master: 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:11.713696  4451 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42684
I20260709 13:15:11.714799 26959 webserver.cc:533] Webserver started at http://127.26.83.196:36581/ using document root <none> and password file <none>
I20260709 13:15:11.715390 26959 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:11.715624 26959 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:11.715894 26959 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:11.717332 26959 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-3-root/instance:
uuid: "098b8827550f475e8c8fa8a20de8f512"
format_stamp: "Formatted at 2026-07-09 13:15:11 on dist-test-slave-329l"
I20260709 13:15:11.722486 26959 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:15:11.726464  4738 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.727415 26959 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:15:11.727689 26959 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-3-root
uuid: "098b8827550f475e8c8fa8a20de8f512"
format_stamp: "Formatted at 2026-07-09 13:15:11 on dist-test-slave-329l"
I20260709 13:15:11.727941 26959 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task3vUH32/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602621086139-26959-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:11.748991 26959 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:11.750204 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:11.751791 26959 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:11.754241 26959 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:11.754451 26959 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.754726 26959 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:11.755061 26959 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:11.800904 26959 rpc_server.cc:307] RPC server started. Bound to: 127.26.83.196:36999
I20260709 13:15:11.801036  4800 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.83.196:36999 every 8 connection(s)
I20260709 13:15:11.816469  4801 heartbeater.cc:344] Connected to a master server at 127.26.83.254:38601
I20260709 13:15:11.817111  4801 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:11.817926  4801 heartbeater.cc:507] Master 127.26.83.254:38601 requested a full tablet report, sending...
I20260709 13:15:11.820053  4451 ts_manager.cc:194] Registered new tserver with Master: 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:11.820878 26959 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016423732s
I20260709 13:15:11.822140  4451 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42686
I20260709 13:15:11.848677  4451 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42688:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:15:12.122936  4618 tablet_service.cc:1511] Processing CreateTablet for tablet 40db247913f74803b44a6878860081b7 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:12.125969  4617 tablet_service.cc:1511] Processing CreateTablet for tablet fb2de72a119a4bb2b88f82a3713d8f7c (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:12.129818  4616 tablet_service.cc:1511] Processing CreateTablet for tablet 498cf28e26b0420fa93d10f75cedb4b4 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:12.131476  4766 tablet_service.cc:1511] Processing CreateTablet for tablet fb2de72a119a4bb2b88f82a3713d8f7c (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:12.124590  4618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40db247913f74803b44a6878860081b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.132156  4542 tablet_service.cc:1511] Processing CreateTablet for tablet fb2de72a119a4bb2b88f82a3713d8f7c (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:12.132447  4765 tablet_service.cc:1511] Processing CreateTablet for tablet 5171793d105d4f9b9070c7bf79d17a74 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:12.133206  4617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb2de72a119a4bb2b88f82a3713d8f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.133299  4766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb2de72a119a4bb2b88f82a3713d8f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.133700  4542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb2de72a119a4bb2b88f82a3713d8f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.135272  4764 tablet_service.cc:1511] Processing CreateTablet for tablet 498cf28e26b0420fa93d10f75cedb4b4 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:12.137084  4764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 498cf28e26b0420fa93d10f75cedb4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.137686  4616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 498cf28e26b0420fa93d10f75cedb4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.136909  4541 tablet_service.cc:1511] Processing CreateTablet for tablet 5171793d105d4f9b9070c7bf79d17a74 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:12.138154  4543 tablet_service.cc:1511] Processing CreateTablet for tablet 40db247913f74803b44a6878860081b7 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:12.139377  4541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5171793d105d4f9b9070c7bf79d17a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.140166  4543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40db247913f74803b44a6878860081b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.141538  4765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5171793d105d4f9b9070c7bf79d17a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.143761  4540 tablet_service.cc:1511] Processing CreateTablet for tablet a153a8986132450aa579bc00e57dd1c7 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:12.165683  4540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a153a8986132450aa579bc00e57dd1c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.167873  4615 tablet_service.cc:1511] Processing CreateTablet for tablet a153a8986132450aa579bc00e57dd1c7 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:12.169536  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a153a8986132450aa579bc00e57dd1c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.171065  4763 tablet_service.cc:1511] Processing CreateTablet for tablet b5e2e6b7b0af4d39b62cb855c6bfd81a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:12.172701  4763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e2e6b7b0af4d39b62cb855c6bfd81a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.176141  4539 tablet_service.cc:1511] Processing CreateTablet for tablet b5e2e6b7b0af4d39b62cb855c6bfd81a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:12.176718  4614 tablet_service.cc:1511] Processing CreateTablet for tablet b5e2e6b7b0af4d39b62cb855c6bfd81a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:12.177997  4539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e2e6b7b0af4d39b62cb855c6bfd81a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.178319  4614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e2e6b7b0af4d39b62cb855c6bfd81a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.186627  4762 tablet_service.cc:1511] Processing CreateTablet for tablet c88b6c7c1b3345b297004b802687432b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:12.188375  4762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c88b6c7c1b3345b297004b802687432b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.193320  4536 tablet_service.cc:1511] Processing CreateTablet for tablet e2c78af3c856414e866ff06aa637ada1 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:12.194832  4536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c78af3c856414e866ff06aa637ada1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.188263  4538 tablet_service.cc:1511] Processing CreateTablet for tablet c88b6c7c1b3345b297004b802687432b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:12.197670  4538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c88b6c7c1b3345b297004b802687432b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.190270  4537 tablet_service.cc:1511] Processing CreateTablet for tablet 1a4681c16b4942faaea1ca21bc374bee (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:12.199750  4537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a4681c16b4942faaea1ca21bc374bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.199234  4760 tablet_service.cc:1511] Processing CreateTablet for tablet e2c78af3c856414e866ff06aa637ada1 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:12.190577  4761 tablet_service.cc:1511] Processing CreateTablet for tablet 463b2b829b9b474b969be53a8778e42b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:12.202708  4760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c78af3c856414e866ff06aa637ada1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.203490  4613 tablet_service.cc:1511] Processing CreateTablet for tablet 463b2b829b9b474b969be53a8778e42b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:12.205084  4761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463b2b829b9b474b969be53a8778e42b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.205688  4613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463b2b829b9b474b969be53a8778e42b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.210271  4535 tablet_service.cc:1511] Processing CreateTablet for tablet e4e79648a69946a0b61b094c6737070a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:12.212230  4535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4e79648a69946a0b61b094c6737070a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.219277  4612 tablet_service.cc:1511] Processing CreateTablet for tablet 1a4681c16b4942faaea1ca21bc374bee (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:12.221437  4612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a4681c16b4942faaea1ca21bc374bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.228575  4611 tablet_service.cc:1511] Processing CreateTablet for tablet e2c78af3c856414e866ff06aa637ada1 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:12.229249  4534 tablet_service.cc:1511] Processing CreateTablet for tablet 8d23781a09a142888526ed9cbcd30d04 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:12.233157  4611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c78af3c856414e866ff06aa637ada1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.235144  4759 tablet_service.cc:1511] Processing CreateTablet for tablet e4e79648a69946a0b61b094c6737070a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:12.237143  4759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4e79648a69946a0b61b094c6737070a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.238673  4610 tablet_service.cc:1511] Processing CreateTablet for tablet a4d49f8bbce5438badd7576f93fa30fd (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:12.239248  4609 tablet_service.cc:1511] Processing CreateTablet for tablet 8d23781a09a142888526ed9cbcd30d04 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:12.240280  4610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4d49f8bbce5438badd7576f93fa30fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.241153  4609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d23781a09a142888526ed9cbcd30d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.252770  4758 tablet_service.cc:1511] Processing CreateTablet for tablet a4d49f8bbce5438badd7576f93fa30fd (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:12.254511  4758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4d49f8bbce5438badd7576f93fa30fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.258133  4757 tablet_service.cc:1511] Processing CreateTablet for tablet 26a8a7249f2c46db84e421ff58ed474e (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:12.259797  4757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a8a7249f2c46db84e421ff58ed474e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:12.263464  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:12.287756  4691 tablet_service.cc:1511] Processing CreateTablet for tablet 5171793d105d4f9b9070c7bf79d17a74 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:12.320379  4691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5171793d105d4f9b9070c7bf79d17a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.314620  4686 tablet_service.cc:1511] Processing CreateTablet for tablet 1a4681c16b4942faaea1ca21bc374bee (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:12.326257  4686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a4681c16b4942faaea1ca21bc374bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.286442  4692 tablet_service.cc:1511] Processing CreateTablet for tablet 40db247913f74803b44a6878860081b7 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:12.330286  4685 tablet_service.cc:1511] Processing CreateTablet for tablet e4e79648a69946a0b61b094c6737070a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:12.330883  4692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40db247913f74803b44a6878860081b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.302749  4690 tablet_service.cc:1511] Processing CreateTablet for tablet 498cf28e26b0420fa93d10f75cedb4b4 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:12.335000  4690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 498cf28e26b0420fa93d10f75cedb4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.335965  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4e79648a69946a0b61b094c6737070a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.306792  4688 tablet_service.cc:1511] Processing CreateTablet for tablet c88b6c7c1b3345b297004b802687432b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:12.337555  4684 tablet_service.cc:1511] Processing CreateTablet for tablet a4d49f8bbce5438badd7576f93fa30fd (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:12.338263  4688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c88b6c7c1b3345b297004b802687432b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.282961  4534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d23781a09a142888526ed9cbcd30d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.304665  4689 tablet_service.cc:1511] Processing CreateTablet for tablet a153a8986132450aa579bc00e57dd1c7 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:12.308791  4687 tablet_service.cc:1511] Processing CreateTablet for tablet 463b2b829b9b474b969be53a8778e42b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:12.339653  4684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4d49f8bbce5438badd7576f93fa30fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.294838  4616 tablet_service.cc:1511] Processing CreateTablet for tablet 26a8a7249f2c46db84e421ff58ed474e (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:12.345958  4616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a8a7249f2c46db84e421ff58ed474e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.353341  4689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a153a8986132450aa579bc00e57dd1c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.355612  4687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463b2b829b9b474b969be53a8778e42b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.374122  4683 tablet_service.cc:1511] Processing CreateTablet for tablet 8d23781a09a142888526ed9cbcd30d04 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:12.384126  4763 tablet_service.cc:1511] Processing CreateTablet for tablet 290a017cf81042b6a4e64319563864c5 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:12.387645  4763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 290a017cf81042b6a4e64319563864c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.400151  4822 tablet_bootstrap.cc:492] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:12.400406  4615 tablet_service.cc:1511] Processing CreateTablet for tablet 4b2841fd64294d6b9762df32ed1bb63e (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:12.402563  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b2841fd64294d6b9762df32ed1bb63e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.405088  4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d23781a09a142888526ed9cbcd30d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.408892  4613 tablet_service.cc:1511] Processing CreateTablet for tablet 3b05eadf0e794d249a3ee1e016745baf (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:12.410485  4613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b05eadf0e794d249a3ee1e016745baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.321733  4618 tablet_service.cc:1511] Processing CreateTablet for tablet bd4347b508bf4002a95a198ba9bcf072 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:12.441258  4540 tablet_service.cc:1511] Processing CreateTablet for tablet 290a017cf81042b6a4e64319563864c5 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:12.453670  4821 tablet_bootstrap.cc:492] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:12.458894  4822 tablet_bootstrap.cc:654] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:12.467892  4823 tablet_bootstrap.cc:492] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:12.473567  4542 tablet_service.cc:1511] Processing CreateTablet for tablet 4b2841fd64294d6b9762df32ed1bb63e (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:12.477478  4618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd4347b508bf4002a95a198ba9bcf072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.481534  4543 tablet_service.cc:1511] Processing CreateTablet for tablet 26a8a7249f2c46db84e421ff58ed474e (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:12.488720  4578 heartbeater.cc:499] Master 127.26.83.254:38601 was elected leader, sending a full tablet report...
I20260709 13:15:12.490940  4764 tablet_service.cc:1511] Processing CreateTablet for tablet bd4347b508bf4002a95a198ba9bcf072 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:12.492878  4765 tablet_service.cc:1511] Processing CreateTablet for tablet d356c2066af241e4a89fad36578a7f89 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:12.491017  4761 tablet_service.cc:1511] Processing CreateTablet for tablet 3b05eadf0e794d249a3ee1e016745baf (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:12.508661  4764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd4347b508bf4002a95a198ba9bcf072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.510879  4765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d356c2066af241e4a89fad36578a7f89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.513859  4688 tablet_service.cc:1511] Processing CreateTablet for tablet 290a017cf81042b6a4e64319563864c5 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:12.517057  4540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 290a017cf81042b6a4e64319563864c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.518201  4761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b05eadf0e794d249a3ee1e016745baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.518362  4821 tablet_bootstrap.cc:654] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:12.517220  4823 tablet_bootstrap.cc:654] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:12.526726  4691 tablet_service.cc:1511] Processing CreateTablet for tablet bd4347b508bf4002a95a198ba9bcf072 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:12.531350  4686 tablet_service.cc:1511] Processing CreateTablet for tablet 4b2841fd64294d6b9762df32ed1bb63e (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:12.533111  4824 tablet_bootstrap.cc:492] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:12.534066  4535 tablet_service.cc:1511] Processing CreateTablet for tablet 3b05eadf0e794d249a3ee1e016745baf (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:12.534116  4539 tablet_service.cc:1511] Processing CreateTablet for tablet d356c2066af241e4a89fad36578a7f89 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:12.535508  4535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b05eadf0e794d249a3ee1e016745baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.536206  4543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a8a7249f2c46db84e421ff58ed474e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.536924  4539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d356c2066af241e4a89fad36578a7f89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.537269  4688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 290a017cf81042b6a4e64319563864c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.545219  4689 tablet_service.cc:1511] Processing CreateTablet for tablet d356c2066af241e4a89fad36578a7f89 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:12.547557  4615 tablet_service.cc:1511] Processing CreateTablet for tablet 0c7d2adc93614f2ea7a7e5f63f4efab7 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:12.549006  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c7d2adc93614f2ea7a7e5f63f4efab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.553364  4542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b2841fd64294d6b9762df32ed1bb63e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.563979  4687 tablet_service.cc:1511] Processing CreateTablet for tablet 0c7d2adc93614f2ea7a7e5f63f4efab7 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:12.565470  4686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b2841fd64294d6b9762df32ed1bb63e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.569617  4687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c7d2adc93614f2ea7a7e5f63f4efab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.567637  4757 tablet_service.cc:1511] Processing CreateTablet for tablet 0c7d2adc93614f2ea7a7e5f63f4efab7 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:12.573104  4757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c7d2adc93614f2ea7a7e5f63f4efab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.571067  4689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d356c2066af241e4a89fad36578a7f89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.560621  4824 tablet_bootstrap.cc:654] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:12.600513  4653 heartbeater.cc:499] Master 127.26.83.254:38601 was elected leader, sending a full tablet report...
I20260709 13:15:12.624274  4691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd4347b508bf4002a95a198ba9bcf072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.653805  4536 tablet_service.cc:1511] Processing CreateTablet for tablet 5e52269d0b2241c6b96a578f3ab61d72 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:12.654076  4611 tablet_service.cc:1511] Processing CreateTablet for tablet 5e52269d0b2241c6b96a578f3ab61d72 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:12.655315  4536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e52269d0b2241c6b96a578f3ab61d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.655478  4611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e52269d0b2241c6b96a578f3ab61d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.662277  4822 tablet_bootstrap.cc:492] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:12.663142  4822 ts_tablet_manager.cc:1403] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.264s	user 0.016s	sys 0.006s
I20260709 13:15:12.673516  4685 tablet_service.cc:1511] Processing CreateTablet for tablet 5e52269d0b2241c6b96a578f3ab61d72 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:12.674149  4824 tablet_bootstrap.cc:492] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:12.674661  4824 ts_tablet_manager.cc:1403] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.142s	user 0.022s	sys 0.022s
I20260709 13:15:12.675012  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e52269d0b2241c6b96a578f3ab61d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.678421  4824 raft_consensus.cc:359] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:12.679366  4824 raft_consensus.cc:385] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:12.679750  4824 raft_consensus.cc:740] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:12.680655  4824 consensus_queue.cc:260] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:12.692230  4539 tablet_service.cc:1511] Processing CreateTablet for tablet 6f4fb667ee0546d7b1b1a8a6ca7519a6 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:12.694188  4823 tablet_bootstrap.cc:492] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:12.695065  4823 ts_tablet_manager.cc:1403] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.259s	user 0.012s	sys 0.018s
I20260709 13:15:12.694244  4539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f4fb667ee0546d7b1b1a8a6ca7519a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.699389  4616 tablet_service.cc:1511] Processing CreateTablet for tablet 6f4fb667ee0546d7b1b1a8a6ca7519a6 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:12.708544  4616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f4fb667ee0546d7b1b1a8a6ca7519a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.702085  4683 tablet_service.cc:1511] Processing CreateTablet for tablet 5b40eb24527a4f8bbc5da80339a74c26 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:12.710639  4821 tablet_bootstrap.cc:492] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:12.716674  4727 heartbeater.cc:499] Master 127.26.83.254:38601 was elected leader, sending a full tablet report...
I20260709 13:15:12.699626  4822 raft_consensus.cc:359] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:12.721825  4822 raft_consensus.cc:385] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:12.721374  4761 tablet_service.cc:1511] Processing CreateTablet for tablet 5b40eb24527a4f8bbc5da80339a74c26 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:12.711799  4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b40eb24527a4f8bbc5da80339a74c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.723173  4761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b40eb24527a4f8bbc5da80339a74c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.716971  4823 raft_consensus.cc:359] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:12.722263  4822 raft_consensus.cc:740] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:12.726799  4822 consensus_queue.cc:260] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:12.727146  4765 tablet_service.cc:1511] Processing CreateTablet for tablet 6f4fb667ee0546d7b1b1a8a6ca7519a6 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:12.728554  4765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f4fb667ee0546d7b1b1a8a6ca7519a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.739544  4537 tablet_service.cc:1511] Processing CreateTablet for tablet 5b40eb24527a4f8bbc5da80339a74c26 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:12.741062  4537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b40eb24527a4f8bbc5da80339a74c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.741345  4618 tablet_service.cc:1511] Processing CreateTablet for tablet 4d318f5fdde647228f87449d2cf0201d (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:12.742713  4618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d318f5fdde647228f87449d2cf0201d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.725809  4823 raft_consensus.cc:385] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:12.760670  4824 ts_tablet_manager.cc:1434] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.086s	user 0.014s	sys 0.008s
I20260709 13:15:12.761870  4824 tablet_bootstrap.cc:492] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:12.761986  4822 ts_tablet_manager.cc:1434] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.098s	user 0.002s	sys 0.007s
I20260709 13:15:12.768201  4822 tablet_bootstrap.cc:492] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:12.735271  4821 ts_tablet_manager.cc:1403] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.282s	user 0.018s	sys 0.039s
I20260709 13:15:12.785418  4830 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.785965  4830 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:12.798760  4686 tablet_service.cc:1511] Processing CreateTablet for tablet 4d318f5fdde647228f87449d2cf0201d (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:12.800418  4686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d318f5fdde647228f87449d2cf0201d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.800034  4763 tablet_service.cc:1511] Processing CreateTablet for tablet 4d318f5fdde647228f87449d2cf0201d (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:12.783126  4822 tablet_bootstrap.cc:654] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:12.791194  4830 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:12.756608  4823 raft_consensus.cc:740] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:12.824133  4763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d318f5fdde647228f87449d2cf0201d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.825387  4801 heartbeater.cc:499] Master 127.26.83.254:38601 was elected leader, sending a full tablet report...
I20260709 13:15:12.825250  4536 tablet_service.cc:1511] Processing CreateTablet for tablet 556306e5b2024ff493e111028a819954 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:12.827270  4536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 556306e5b2024ff493e111028a819954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.828549  4821 raft_consensus.cc:359] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:12.829906  4823 consensus_queue.cc:260] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:12.837581  4654 tablet.cc:2406] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:15:12.833220  4821 raft_consensus.cc:385] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:12.851183  4691 tablet_service.cc:1511] Processing CreateTablet for tablet 556306e5b2024ff493e111028a819954 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:12.852705  4691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 556306e5b2024ff493e111028a819954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.861505  4758 tablet_service.cc:1511] Processing CreateTablet for tablet ae70b5bde8ec47b1931014ebfe03ea2a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:12.863042  4758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae70b5bde8ec47b1931014ebfe03ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.842156  4821 raft_consensus.cc:740] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:12.872402  4685 tablet_service.cc:1511] Processing CreateTablet for tablet 54dd8f3fe16647c5b841897841d7206b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:12.874038  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54dd8f3fe16647c5b841897841d7206b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.879968  4539 tablet_service.cc:1511] Processing CreateTablet for tablet ae70b5bde8ec47b1931014ebfe03ea2a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:12.881507  4539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae70b5bde8ec47b1931014ebfe03ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.881860  4611 tablet_service.cc:1511] Processing CreateTablet for tablet 556306e5b2024ff493e111028a819954 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:12.883231  4611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 556306e5b2024ff493e111028a819954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.863116  4824 tablet_bootstrap.cc:654] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:12.888840  4764 tablet_service.cc:1511] Processing CreateTablet for tablet 54dd8f3fe16647c5b841897841d7206b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:12.890311  4764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54dd8f3fe16647c5b841897841d7206b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.890188  4616 tablet_service.cc:1511] Processing CreateTablet for tablet ae70b5bde8ec47b1931014ebfe03ea2a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:12.891633  4616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae70b5bde8ec47b1931014ebfe03ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.902077  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:12.903488  4553 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:12.906044  4592 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:12.907523  4831 raft_consensus.cc:2804] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:12.908051  4831 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:12.908483  4831 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:12.910144  4823 ts_tablet_manager.cc:1434] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.215s	user 0.009s	sys 0.010s
I20260709 13:15:12.916994  4831 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:12.922134  4821 consensus_queue.cc:260] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:12.923519  4543 tablet_service.cc:1511] Processing CreateTablet for tablet 54dd8f3fe16647c5b841897841d7206b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:12.926566  4543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54dd8f3fe16647c5b841897841d7206b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.928218  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:12.928946  4553 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:12.931178  4823 tablet_bootstrap.cc:492] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:12.939765  4829 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.940361  4829 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:12.942621  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:12.943435  4776 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:12.946986  4829 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:12.947494  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:12.948215  4775 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:12.949276  4615 tablet_service.cc:1511] Processing CreateTablet for tablet cd411637f5c84d50b6c9ba5cae086a07 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:12.950685  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd411637f5c84d50b6c9ba5cae086a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.954519  4822 tablet_bootstrap.cc:492] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:12.955245  4822 ts_tablet_manager.cc:1403] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.187s	user 0.007s	sys 0.023s
I20260709 13:15:12.958614  4822 raft_consensus.cc:359] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:12.958667  4775 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:12.959462  4822 raft_consensus.cc:385] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:12.959859  4822 raft_consensus.cc:740] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:12.945856  4831 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:12.960601  4594 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:12.967227  4831 raft_consensus.cc:2804] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:12.968921  4831 raft_consensus.cc:697] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:12.968514  4553 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:12.984270  4831 consensus_queue.cc:237] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:12.988451  4822 consensus_queue.cc:260] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:12.990146  4765 tablet_service.cc:1511] Processing CreateTablet for tablet cd411637f5c84d50b6c9ba5cae086a07 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:12.991129  4822 ts_tablet_manager.cc:1434] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 13:15:12.991600  4765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd411637f5c84d50b6c9ba5cae086a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.992223  4822 tablet_bootstrap.cc:492] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:12.993250  4618 tablet_service.cc:1511] Processing CreateTablet for tablet f21085005d7842b9aaa3c27849f36d63 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:12.994676  4618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f21085005d7842b9aaa3c27849f36d63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.994467  4537 tablet_service.cc:1511] Processing CreateTablet for tablet f21085005d7842b9aaa3c27849f36d63 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:12.995744  4537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f21085005d7842b9aaa3c27849f36d63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:12.987020  4683 tablet_service.cc:1511] Processing CreateTablet for tablet cd411637f5c84d50b6c9ba5cae086a07 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:13.001751  4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd411637f5c84d50b6c9ba5cae086a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.002921  4821 ts_tablet_manager.cc:1434] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.210s	user 0.012s	sys 0.008s
I20260709 13:15:13.004062  4821 tablet_bootstrap.cc:492] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:13.009470  4823 tablet_bootstrap.cc:654] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.009569  4822 tablet_bootstrap.cc:654] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.017751  4691 tablet_service.cc:1511] Processing CreateTablet for tablet f21085005d7842b9aaa3c27849f36d63 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:13.020025  4691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f21085005d7842b9aaa3c27849f36d63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.029268  4535 tablet_service.cc:1511] Processing CreateTablet for tablet eeb7a8868de44466ab91f3972a95eb22 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:13.030691  4535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb7a8868de44466ab91f3972a95eb22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.043599  4611 tablet_service.cc:1511] Processing CreateTablet for tablet eeb7a8868de44466ab91f3972a95eb22 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:13.052067  4685 tablet_service.cc:1511] Processing CreateTablet for tablet 8be6a3412d5a45a3a84b4b39e54c977a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:13.053630  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8be6a3412d5a45a3a84b4b39e54c977a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.057005  4611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb7a8868de44466ab91f3972a95eb22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.063858  4824 tablet_bootstrap.cc:492] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:13.064682  4824 ts_tablet_manager.cc:1403] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.303s	user 0.020s	sys 0.011s
I20260709 13:15:13.068033  4761 tablet_service.cc:1511] Processing CreateTablet for tablet eeb7a8868de44466ab91f3972a95eb22 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:13.069880  4761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb7a8868de44466ab91f3972a95eb22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.071336  4824 raft_consensus.cc:359] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.072495  4824 raft_consensus.cc:385] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.072947  4824 raft_consensus.cc:740] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.074319  4824 consensus_queue.cc:260] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.076416  4763 tablet_service.cc:1511] Processing CreateTablet for tablet 8be6a3412d5a45a3a84b4b39e54c977a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:13.085695  4763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8be6a3412d5a45a3a84b4b39e54c977a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.105222  4536 tablet_service.cc:1511] Processing CreateTablet for tablet 8be6a3412d5a45a3a84b4b39e54c977a (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:13.105042  4450 catalog_manager.cc:5697] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:13.106580  4536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8be6a3412d5a45a3a84b4b39e54c977a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.107350  4822 tablet_bootstrap.cc:492] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:13.107924  4822 ts_tablet_manager.cc:1403] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.116s	user 0.043s	sys 0.021s
I20260709 13:15:13.112558  4822 raft_consensus.cc:359] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.113392  4822 raft_consensus.cc:385] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.113711  4822 raft_consensus.cc:740] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.114571  4822 consensus_queue.cc:260] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.116917  4822 ts_tablet_manager.cc:1434] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:13.117688  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:13.118036  4822 tablet_bootstrap.cc:492] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
W20260709 13:15:13.119481  4666 leader_election.cc:343] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.121160  4824 ts_tablet_manager.cc:1434] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.003s
I20260709 13:15:13.124686  4821 tablet_bootstrap.cc:654] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.132694  4823 tablet_bootstrap.cc:492] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:13.133339  4823 ts_tablet_manager.cc:1403] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.203s	user 0.013s	sys 0.002s
I20260709 13:15:13.124879  4822 tablet_bootstrap.cc:654] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.126461  4824 tablet_bootstrap.cc:492] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:13.129050  4615 tablet_service.cc:1511] Processing CreateTablet for tablet 746d9c187500488796f0879f81bf74f3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:13.135385  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746d9c187500488796f0879f81bf74f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.143209  4824 tablet_bootstrap.cc:654] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.143445  4821 tablet_bootstrap.cc:492] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:13.144062  4821 ts_tablet_manager.cc:1403] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.140s	user 0.010s	sys 0.006s
I20260709 13:15:13.146966  4823 raft_consensus.cc:359] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.148021  4823 raft_consensus.cc:385] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.150467  4821 raft_consensus.cc:359] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:13.151265  4821 raft_consensus.cc:385] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.151574  4821 raft_consensus.cc:740] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.148417  4823 raft_consensus.cc:740] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.158538  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:13.160305  4666 leader_election.cc:343] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.160868  4666 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:13.161684  4829 raft_consensus.cc:2749] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:13.162961  4765 tablet_service.cc:1511] Processing CreateTablet for tablet 746d9c187500488796f0879f81bf74f3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:13.160157  4691 tablet_service.cc:1511] Processing CreateTablet for tablet 746d9c187500488796f0879f81bf74f3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:13.164324  4765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746d9c187500488796f0879f81bf74f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.171633  4618 tablet_service.cc:1511] Processing CreateTablet for tablet e34aff2d174a4f3c8092c452a40c94a3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:13.173080  4618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e34aff2d174a4f3c8092c452a40c94a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.173115  4691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746d9c187500488796f0879f81bf74f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.172955  4683 tablet_service.cc:1511] Processing CreateTablet for tablet e34aff2d174a4f3c8092c452a40c94a3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:13.174502  4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e34aff2d174a4f3c8092c452a40c94a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.152387  4821 consensus_queue.cc:260] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:13.175755  4823 consensus_queue.cc:260] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.183303  4821 ts_tablet_manager.cc:1434] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.003s
I20260709 13:15:13.184233  4821 tablet_bootstrap.cc:492] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:13.185341  4535 tablet_service.cc:1511] Processing CreateTablet for tablet e34aff2d174a4f3c8092c452a40c94a3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:13.186803  4535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e34aff2d174a4f3c8092c452a40c94a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.190272  4823 ts_tablet_manager.cc:1434] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 13:15:13.191383  4823 tablet_bootstrap.cc:492] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:13.193025  4821 tablet_bootstrap.cc:654] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.196270  4761 tablet_service.cc:1511] Processing CreateTablet for tablet 672896bdac10415b85afc981d3c6eac9 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:13.197839  4761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 672896bdac10415b85afc981d3c6eac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.208575  4537 tablet_service.cc:1511] Processing CreateTablet for tablet 672896bdac10415b85afc981d3c6eac9 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:13.210047  4537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 672896bdac10415b85afc981d3c6eac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.210881  4839 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.211604  4839 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:13.215348  4823 tablet_bootstrap.cc:654] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.231918  4831 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.232566  4831 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.232826  4685 tablet_service.cc:1511] Processing CreateTablet for tablet 92fea640a4c24068bf9e2805a83ee8b6 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:13.233615  4616 tablet_service.cc:1511] Processing CreateTablet for tablet 672896bdac10415b85afc981d3c6eac9 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:13.234378  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92fea640a4c24068bf9e2805a83ee8b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.234916  4616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 672896bdac10415b85afc981d3c6eac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.238339  4824 tablet_bootstrap.cc:492] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:13.239019  4824 ts_tablet_manager.cc:1403] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.113s	user 0.023s	sys 0.008s
I20260709 13:15:13.240854  4831 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:13.245363  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:13.246124  4553 raft_consensus.cc:2468] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:13.250833  4592 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:13.251806  4830 raft_consensus.cc:493] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.252920  4822 tablet_bootstrap.cc:492] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:13.253489  4822 ts_tablet_manager.cc:1403] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.136s	user 0.007s	sys 0.013s
I20260709 13:15:13.254868  4831 raft_consensus.cc:2804] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:13.255350  4831 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.255723  4831 raft_consensus.cc:3060] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:13.252303  4830 raft_consensus.cc:515] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:13.250676  4758 tablet_service.cc:1511] Processing CreateTablet for tablet 92fea640a4c24068bf9e2805a83ee8b6 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:13.268474  4758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92fea640a4c24068bf9e2805a83ee8b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.269485  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:13.271546  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:13.272488  4536 tablet_service.cc:1511] Processing CreateTablet for tablet 92fea640a4c24068bf9e2805a83ee8b6 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:13.275378  4536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92fea640a4c24068bf9e2805a83ee8b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.262434  4839 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:13.277940  4611 tablet_service.cc:1511] Processing CreateTablet for tablet e7c8faa6496c4c3dbd2e25d9617b0ffc (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:13.279452  4611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7c8faa6496c4c3dbd2e25d9617b0ffc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:13.280896  4594 leader_election.cc:343] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.283367  4824 raft_consensus.cc:359] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.284164  4824 raft_consensus.cc:385] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.263142  4822 raft_consensus.cc:359] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.285138  4822 raft_consensus.cc:385] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.285470  4822 raft_consensus.cc:740] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.286351  4822 consensus_queue.cc:260] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.273303  4830 leader_election.cc:290] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:13.291316  4686 tablet_service.cc:1511] Processing CreateTablet for tablet e7c8faa6496c4c3dbd2e25d9617b0ffc (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:13.293081  4686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7c8faa6496c4c3dbd2e25d9617b0ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.284607  4824 raft_consensus.cc:740] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.301592  4757 tablet_service.cc:1511] Processing CreateTablet for tablet e7c8faa6496c4c3dbd2e25d9617b0ffc (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:13.314633  4824 consensus_queue.cc:260] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.315528  4757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7c8faa6496c4c3dbd2e25d9617b0ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.316021  4539 tablet_service.cc:1511] Processing CreateTablet for tablet bd2cefb52b43430cbf06fb122ce9c1d8 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:13.317607  4539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2cefb52b43430cbf06fb122ce9c1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.318303  4821 tablet_bootstrap.cc:492] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:13.318897  4821 ts_tablet_manager.cc:1403] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.135s	user 0.015s	sys 0.015s
I20260709 13:15:13.335995  4615 tablet_service.cc:1511] Processing CreateTablet for tablet bd2cefb52b43430cbf06fb122ce9c1d8 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:13.337481  4824 ts_tablet_manager.cc:1434] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.093s	user 0.005s	sys 0.002s
I20260709 13:15:13.337880  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2cefb52b43430cbf06fb122ce9c1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.338858  4824 tablet_bootstrap.cc:492] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:13.337111  4831 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.347503  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:13.348253  4553 raft_consensus.cc:3060] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:13.346558  4831 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:13.361443  4683 tablet_service.cc:1511] Processing CreateTablet for tablet bd2cefb52b43430cbf06fb122ce9c1d8 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:13.362915  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:13.365590  4740 leader_election.cc:343] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.366869  4821 raft_consensus.cc:359] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:13.367800  4821 raft_consensus.cc:385] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.368124  4821 raft_consensus.cc:740] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.369176  4821 consensus_queue.cc:260] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:13.371578  4821 ts_tablet_manager.cc:1434] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.002s
I20260709 13:15:13.374789  4823 tablet_bootstrap.cc:492] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:13.375465  4823 ts_tablet_manager.cc:1403] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.184s	user 0.005s	sys 0.016s
I20260709 13:15:13.380016  4823 raft_consensus.cc:359] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.381027  4823 raft_consensus.cc:385] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.381402  4823 raft_consensus.cc:740] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.382354  4823 consensus_queue.cc:260] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.384984  4823 ts_tablet_manager.cc:1434] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.001s
I20260709 13:15:13.387395  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:13.389415  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
W20260709 13:15:13.392072  4593 leader_election.cc:343] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:13.393260  4593 leader_election.cc:343] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.395285  4823 tablet_bootstrap.cc:492] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:13.395648  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:13.397558  4593 leader_election.cc:343] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.398115  4593 leader_election.cc:304] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:13.399729  4856 raft_consensus.cc:2749] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:13.401259  4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2cefb52b43430cbf06fb122ce9c1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.402582  4823 tablet_bootstrap.cc:654] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.402460  4553 raft_consensus.cc:2468] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:13.404105  4592 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:13.407240  4856 raft_consensus.cc:2804] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:13.407874  4856 raft_consensus.cc:697] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:13.408522  4822 ts_tablet_manager.cc:1434] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.155s	user 0.009s	sys 0.011s
I20260709 13:15:13.408946  4856 consensus_queue.cc:237] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.409902  4822 tablet_bootstrap.cc:492] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:13.410352  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:13.392637  4824 tablet_bootstrap.cc:654] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:13.417013  4740 leader_election.cc:343] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.417755  4740 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:13.419296  4839 raft_consensus.cc:2749] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:13.420840  4821 tablet_bootstrap.cc:492] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:13.421471  4616 tablet_service.cc:1511] Processing CreateTablet for tablet 4693f763e2c548f8a49643afa022a5f5 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:13.423018  4616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4693f763e2c548f8a49643afa022a5f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.420636  4822 tablet_bootstrap.cc:654] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.425153  4829 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.425791  4829 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.417096  4535 tablet_service.cc:1511] Processing CreateTablet for tablet 4693f763e2c548f8a49643afa022a5f5 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:13.434201  4535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4693f763e2c548f8a49643afa022a5f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.437994  4761 tablet_service.cc:1511] Processing CreateTablet for tablet 4693f763e2c548f8a49643afa022a5f5 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:13.440665  4761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4693f763e2c548f8a49643afa022a5f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.441771  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:13.444064  4666 leader_election.cc:343] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.438999  4829 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:13.463088  4685 tablet_service.cc:1511] Processing CreateTablet for tablet c59401c42fe8430aaed4b49ef7df0372 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:13.465485  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59401c42fe8430aaed4b49ef7df0372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.475155  4757 tablet_service.cc:1511] Processing CreateTablet for tablet c59401c42fe8430aaed4b49ef7df0372 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:13.476681  4757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59401c42fe8430aaed4b49ef7df0372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.478350  4539 tablet_service.cc:1511] Processing CreateTablet for tablet c59401c42fe8430aaed4b49ef7df0372 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:13.481288  4539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59401c42fe8430aaed4b49ef7df0372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.485968  4822 tablet_bootstrap.cc:492] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:13.486732  4822 ts_tablet_manager.cc:1403] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.077s	user 0.021s	sys 0.020s
I20260709 13:15:13.479676  4823 tablet_bootstrap.cc:492] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:13.482975  4616 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0c53efa0ee4dfbba4c3f1922027f50 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:13.484546  4821 tablet_bootstrap.cc:654] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.484122  4683 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0c53efa0ee4dfbba4c3f1922027f50 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:13.492427  4823 ts_tablet_manager.cc:1403] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.097s	user 0.015s	sys 0.007s
I20260709 13:15:13.497217  4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0c53efa0ee4dfbba4c3f1922027f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.503223  4822 raft_consensus.cc:359] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.504088  4822 raft_consensus.cc:385] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.504401  4822 raft_consensus.cc:740] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.506398  4758 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0c53efa0ee4dfbba4c3f1922027f50 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:13.507925  4758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0c53efa0ee4dfbba4c3f1922027f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.511909  4822 consensus_queue.cc:260] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.514394  4822 ts_tablet_manager.cc:1434] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:15:13.515527  4822 tablet_bootstrap.cc:492] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:13.517705  4450 catalog_manager.cc:5697] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:13.519485  4616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0c53efa0ee4dfbba4c3f1922027f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.523133  4823 raft_consensus.cc:359] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.524080  4823 raft_consensus.cc:385] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.526702  4535 tablet_service.cc:1511] Processing CreateTablet for tablet 3fb194bf63a4451f8c4d28502acdad60 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:13.527202  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:13.530658  4535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fb194bf63a4451f8c4d28502acdad60. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:13.531658  4668 leader_election.cc:343] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.532387  4668 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:13.533798  4829 raft_consensus.cc:2749] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:13.539088  4829 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.540166  4829 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.542294  4829 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:13.529747  4823 raft_consensus.cc:740] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.542730  4856 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.545761  4856 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.546559  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:13.546713  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:13.547308  4628 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:13.548759  4856 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:13.549083  4683 tablet_service.cc:1511] Processing CreateTablet for tablet 3fb194bf63a4451f8c4d28502acdad60 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:13.549424  4615 tablet_service.cc:1511] Processing CreateTablet for tablet 3fb194bf63a4451f8c4d28502acdad60 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:13.550489  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:13.550896  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fb194bf63a4451f8c4d28502acdad60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.551348  4553 raft_consensus.cc:2468] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:13.552943  4592 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:13.553881  4856 raft_consensus.cc:2804] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:13.553923  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:13.554481  4856 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.554935  4856 raft_consensus.cc:3060] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:13.554785  4765 tablet_service.cc:1511] Processing CreateTablet for tablet 84197a75d2854ab286a8a19349bf7cc9 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:13.550812  4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fb194bf63a4451f8c4d28502acdad60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.556192  4765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84197a75d2854ab286a8a19349bf7cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.554824  4701 raft_consensus.cc:2468] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:13.559242  4822 tablet_bootstrap.cc:654] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:13.555109  4666 leader_election.cc:343] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.560380  4666 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:13.563145  4829 raft_consensus.cc:2804] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:13.563586  4829 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.563970  4829 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:13.564481  4821 tablet_bootstrap.cc:492] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:13.565099  4821 ts_tablet_manager.cc:1403] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.192s	user 0.006s	sys 0.008s
I20260709 13:15:13.567241  4824 tablet_bootstrap.cc:492] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:13.567931  4824 ts_tablet_manager.cc:1403] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.229s	user 0.014s	sys 0.014s
I20260709 13:15:13.569231  4823 consensus_queue.cc:260] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.573192  4536 tablet_service.cc:1511] Processing CreateTablet for tablet 84197a75d2854ab286a8a19349bf7cc9 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:13.574620  4536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84197a75d2854ab286a8a19349bf7cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.566897  4856 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.580587  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:13.581343  4553 raft_consensus.cc:3060] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:13.583637  4824 raft_consensus.cc:359] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.584452  4611 tablet_service.cc:1511] Processing CreateTablet for tablet 84197a75d2854ab286a8a19349bf7cc9 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:13.586115  4611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84197a75d2854ab286a8a19349bf7cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.589371  4553 raft_consensus.cc:2468] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:13.590883  4592 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:13.591931  4831 raft_consensus.cc:2804] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:13.592469  4831 raft_consensus.cc:697] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:13.593397  4831 consensus_queue.cc:237] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.600574  4856 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:13.602020  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:13.602734  4701 raft_consensus.cc:3060] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:13.603349  4829 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.604576  4823 ts_tablet_manager.cc:1434] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.112s	user 0.006s	sys 0.002s
I20260709 13:15:13.607131  4821 raft_consensus.cc:359] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.608119  4821 raft_consensus.cc:385] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.618695  4823 tablet_bootstrap.cc:492] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:13.618937  4830 consensus_queue.cc:1048] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:13.620246  4829 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:13.622301  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:13.623024  4628 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:13.608461  4821 raft_consensus.cc:740] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.628883  4822 tablet_bootstrap.cc:492] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:13.584542  4824 raft_consensus.cc:385] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.639011  4701 raft_consensus.cc:2468] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:13.644613  4856 consensus_queue.cc:1048] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:15:13.641006  4822 ts_tablet_manager.cc:1403] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.126s	user 0.008s	sys 0.008s
I20260709 13:15:13.651130  4685 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9d0edf26f04c6eaef32e7dcc96eec6 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:13.652747  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9d0edf26f04c6eaef32e7dcc96eec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.654799  4821 consensus_queue.cc:260] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.642161  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
W20260709 13:15:13.659606  4666 leader_election.cc:343] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.660842  4628 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:13.662603  4666 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:13.664189  4829 raft_consensus.cc:2804] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:13.639971  4824 raft_consensus.cc:740] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.671407  4823 tablet_bootstrap.cc:654] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.681102  4821 ts_tablet_manager.cc:1434] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.116s	user 0.006s	sys 0.004s
I20260709 13:15:13.682494  4821 tablet_bootstrap.cc:492] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:13.686513  4535 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9d0edf26f04c6eaef32e7dcc96eec6 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:13.688083  4535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9d0edf26f04c6eaef32e7dcc96eec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.676409  4765 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9d0edf26f04c6eaef32e7dcc96eec6 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:13.687304  4451 catalog_manager.cc:5697] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:13.690161  4765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9d0edf26f04c6eaef32e7dcc96eec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.653609  4822 raft_consensus.cc:359] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:13.696434  4829 raft_consensus.cc:697] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:13.700246  4829 consensus_queue.cc:237] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:13.716113  4821 tablet_bootstrap.cc:654] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.707718  4822 raft_consensus.cc:385] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.726994  4822 raft_consensus.cc:740] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.731163  4822 consensus_queue.cc:260] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:13.732257  4615 tablet_service.cc:1511] Processing CreateTablet for tablet 6beeb8c8bf7343c297daf017c542c67b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:13.732257  4683 tablet_service.cc:1511] Processing CreateTablet for tablet 6beeb8c8bf7343c297daf017c542c67b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:13.733875  4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beeb8c8bf7343c297daf017c542c67b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.733875  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beeb8c8bf7343c297daf017c542c67b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.736665  4758 tablet_service.cc:1511] Processing CreateTablet for tablet 6beeb8c8bf7343c297daf017c542c67b (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:13.738164  4758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6beeb8c8bf7343c297daf017c542c67b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.741236  4835 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.753270  4835 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.728624  4824 consensus_queue.cc:260] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.760424  4835 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:13.768805  4616 tablet_service.cc:1511] Processing CreateTablet for tablet 179cf9478a1642929379f46e4311d9a5 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:13.784358  4616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 179cf9478a1642929379f46e4311d9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.808280  4539 tablet_service.cc:1511] Processing CreateTablet for tablet 179cf9478a1642929379f46e4311d9a5 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:13.809747  4539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 179cf9478a1642929379f46e4311d9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.813737  4685 tablet_service.cc:1511] Processing CreateTablet for tablet 179cf9478a1642929379f46e4311d9a5 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:13.814241  4536 tablet_service.cc:1511] Processing CreateTablet for tablet 1bcbea185b8c48d7989bb050470948a0 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:13.815193  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 179cf9478a1642929379f46e4311d9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.815505  4536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bcbea185b8c48d7989bb050470948a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.817304  4757 tablet_service.cc:1511] Processing CreateTablet for tablet 1bcbea185b8c48d7989bb050470948a0 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:13.820365  4824 ts_tablet_manager.cc:1434] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.252s	user 0.006s	sys 0.004s
I20260709 13:15:13.828573  4757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bcbea185b8c48d7989bb050470948a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.831681  4839 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.832749  4839 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:13.838286  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:13.838513  4450 catalog_manager.cc:5697] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 0 to 1, leader changed from <none> to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 1 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:13.840582  4740 leader_election.cc:343] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.841534  4839 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:13.845819  4821 tablet_bootstrap.cc:492] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:13.846494  4821 ts_tablet_manager.cc:1403] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.164s	user 0.015s	sys 0.017s
I20260709 13:15:13.849146  4831 consensus_queue.cc:1048] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:13.864146  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:13.865772  4824 tablet_bootstrap.cc:492] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:13.866043  4611 tablet_service.cc:1511] Processing CreateTablet for tablet 1bcbea185b8c48d7989bb050470948a0 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:13.867556  4611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bcbea185b8c48d7989bb050470948a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.868863  4821 raft_consensus.cc:359] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:13.869598  4740 leader_election.cc:343] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.870085  4821 raft_consensus.cc:385] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:13.870195  4740 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:13.870674  4821 raft_consensus.cc:740] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:13.871321  4839 raft_consensus.cc:2749] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:13.871814  4822 ts_tablet_manager.cc:1434] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.222s	user 0.008s	sys 0.002s
I20260709 13:15:13.873458  4822 tablet_bootstrap.cc:492] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:13.877475  4821 consensus_queue.cc:260] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:13.880987  4691 tablet_service.cc:1511] Processing CreateTablet for tablet 9885b7c627e24d63a651c2cbc9e0e24d (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:13.882751  4691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9885b7c627e24d63a651c2cbc9e0e24d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.890240  4824 tablet_bootstrap.cc:654] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.900486  4761 tablet_service.cc:1511] Processing CreateTablet for tablet 9885b7c627e24d63a651c2cbc9e0e24d (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:13.900499  4821 ts_tablet_manager.cc:1434] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.053s	user 0.011s	sys 0.001s
I20260709 13:15:13.902251  4761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9885b7c627e24d63a651c2cbc9e0e24d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.906783  4535 tablet_service.cc:1511] Processing CreateTablet for tablet 9885b7c627e24d63a651c2cbc9e0e24d (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:13.908195  4535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9885b7c627e24d63a651c2cbc9e0e24d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.926895  4615 tablet_service.cc:1511] Processing CreateTablet for tablet f2f0c73f991a42e4817d8916fa0a7c30 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:13.928225  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2f0c73f991a42e4817d8916fa0a7c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.930615  4822 tablet_bootstrap.cc:654] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:13.923722  4821 tablet_bootstrap.cc:492] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:13.934674  4686 tablet_service.cc:1511] Processing CreateTablet for tablet f2f0c73f991a42e4817d8916fa0a7c30 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:13.936231  4686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2f0c73f991a42e4817d8916fa0a7c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.937978  4763 tablet_service.cc:1511] Processing CreateTablet for tablet f2f0c73f991a42e4817d8916fa0a7c30 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:13.939608  4763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2f0c73f991a42e4817d8916fa0a7c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.940600  4537 tablet_service.cc:1511] Processing CreateTablet for tablet 0be07b1d9e584cabb47a7e6a71d9cd44 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:13.941964  4537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0be07b1d9e584cabb47a7e6a71d9cd44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:13.947758  4839 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.948330  4839 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:13.951004  4839 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:13.982932  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:13.984773  4740 leader_election.cc:343] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:13.989197  4823 tablet_bootstrap.cc:492] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:13.989846  4823 ts_tablet_manager.cc:1403] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.371s	user 0.052s	sys 0.021s
I20260709 13:15:13.992905  4839 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.993502  4839 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:13.996948  4821 tablet_bootstrap.cc:654] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.000345  4543 tablet_service.cc:1511] Processing CreateTablet for tablet b34f76f541974e5490d222ce8856de10 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:13.996817  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:14.001845  4543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b34f76f541974e5490d222ce8856de10. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:14.002554  4740 leader_election.cc:343] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.005887  4839 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:14.006188  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:14.007501  4685 tablet_service.cc:1511] Processing CreateTablet for tablet 0be07b1d9e584cabb47a7e6a71d9cd44 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
W20260709 13:15:14.007895  4518 leader_election.cc:343] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.008821  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0be07b1d9e584cabb47a7e6a71d9cd44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.012254  4615 tablet_service.cc:1511] Processing CreateTablet for tablet b34f76f541974e5490d222ce8856de10 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:14.013159  4823 raft_consensus.cc:359] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.020938  4823 raft_consensus.cc:385] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.021308  4823 raft_consensus.cc:740] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.021577  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b34f76f541974e5490d222ce8856de10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.022209  4823 consensus_queue.cc:260] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.015767  4691 tablet_service.cc:1511] Processing CreateTablet for tablet 007ae9d1d319436394e39965fef13375 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:14.033308  4823 ts_tablet_manager.cc:1434] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.001s
I20260709 13:15:14.033736  4691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 007ae9d1d319436394e39965fef13375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.040302  4616 tablet_service.cc:1511] Processing CreateTablet for tablet 0be07b1d9e584cabb47a7e6a71d9cd44 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:14.042176  4616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0be07b1d9e584cabb47a7e6a71d9cd44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.034488  4823 tablet_bootstrap.cc:492] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:14.077636  4824 tablet_bootstrap.cc:492] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:14.078272  4824 ts_tablet_manager.cc:1403] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.213s	user 0.017s	sys 0.008s
I20260709 13:15:14.081638  4536 tablet_service.cc:1511] Processing CreateTablet for tablet 007ae9d1d319436394e39965fef13375 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:14.083186  4536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 007ae9d1d319436394e39965fef13375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.083097  4758 tablet_service.cc:1511] Processing CreateTablet for tablet b34f76f541974e5490d222ce8856de10 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:14.085795  4764 tablet_service.cc:1511] Processing CreateTablet for tablet 0059f848a34f47b5a3fe29efe38914b3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:14.086470  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:14.088279  4519 leader_election.cc:343] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.088850  4519 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:14.090472  4878 raft_consensus.cc:2749] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:14.099803  4760 tablet_service.cc:1511] Processing CreateTablet for tablet 86791c9adcc64babb9560e224fd9abd9 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:14.101841  4762 tablet_service.cc:1511] Processing CreateTablet for tablet 74cc2914707b4e1586aff19b228e92c3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:14.102398  4758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b34f76f541974e5490d222ce8856de10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.103107  4760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86791c9adcc64babb9560e224fd9abd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.104656  4822 tablet_bootstrap.cc:492] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:14.105285  4822 ts_tablet_manager.cc:1403] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.232s	user 0.020s	sys 0.007s
W20260709 13:15:14.110420  4593 consensus_peers.cc:597] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 -> Peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Couldn't send request to peer 117be35051554b3b837c42a5dfcfa38a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:14.111390  4764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0059f848a34f47b5a3fe29efe38914b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.111979  4618 tablet_service.cc:1511] Processing CreateTablet for tablet 181238be5f794f7188fdf4eb7f368b71 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:14.113561  4618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181238be5f794f7188fdf4eb7f368b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.085744  4765 tablet_service.cc:1511] Processing CreateTablet for tablet 007ae9d1d319436394e39965fef13375 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:14.117086  4685 tablet_service.cc:1511] Processing CreateTablet for tablet 0059f848a34f47b5a3fe29efe38914b3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:14.118170  4765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 007ae9d1d319436394e39965fef13375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.118525  4685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0059f848a34f47b5a3fe29efe38914b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.117831  4683 tablet_service.cc:1511] Processing CreateTablet for tablet 82b6a1da7d62472aa62e9be6aa348f2e (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:14.120951  4688 tablet_service.cc:1511] Processing CreateTablet for tablet 86791c9adcc64babb9560e224fd9abd9 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:14.121573  4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82b6a1da7d62472aa62e9be6aa348f2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.121640  4535 tablet_service.cc:1511] Processing CreateTablet for tablet 86791c9adcc64babb9560e224fd9abd9 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:14.122923  4535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86791c9adcc64babb9560e224fd9abd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.122991  4537 tablet_service.cc:1511] Processing CreateTablet for tablet 74cc2914707b4e1586aff19b228e92c3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:14.111979  4611 tablet_service.cc:1511] Processing CreateTablet for tablet 74cc2914707b4e1586aff19b228e92c3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:14.124240  4537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74cc2914707b4e1586aff19b228e92c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.124647  4611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74cc2914707b4e1586aff19b228e92c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.125101  4688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86791c9adcc64babb9560e224fd9abd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.128805  4821 tablet_bootstrap.cc:492] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:14.129172  4762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74cc2914707b4e1586aff19b228e92c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.129387  4821 ts_tablet_manager.cc:1403] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.214s	user 0.018s	sys 0.022s
I20260709 13:15:14.136539  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:14.137346  4701 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:14.137925  4539 tablet_service.cc:1511] Processing CreateTablet for tablet 82b6a1da7d62472aa62e9be6aa348f2e (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:14.138397  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:14.139467  4539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82b6a1da7d62472aa62e9be6aa348f2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.142256  4823 tablet_bootstrap.cc:654] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.142069  4875 consensus_queue.cc:1048] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:14.143043  4741 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:14.143999  4839 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:14.144392  4839 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.144719  4839 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:14.150594  4741 leader_election.cc:343] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.151188  4741 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:14.153723  4616 tablet_service.cc:1511] Processing CreateTablet for tablet 0059f848a34f47b5a3fe29efe38914b3 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:14.155154  4616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0059f848a34f47b5a3fe29efe38914b3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:14.157425  4666 consensus_peers.cc:597] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a -> Peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Couldn't send request to peer 978db1e31e16405182e65e88f3b9bebc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:14.157323  4822 raft_consensus.cc:359] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.158403  4822 raft_consensus.cc:385] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.158762  4822 raft_consensus.cc:740] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.159649  4822 consensus_queue.cc:260] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.165781  4615 tablet_service.cc:1511] Processing CreateTablet for tablet 82b6a1da7d62472aa62e9be6aa348f2e (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:14.165906  4821 raft_consensus.cc:359] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.166828  4821 raft_consensus.cc:385] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.167244  4615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82b6a1da7d62472aa62e9be6aa348f2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.166838  4839 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.173244  4691 tablet_service.cc:1511] Processing CreateTablet for tablet 181238be5f794f7188fdf4eb7f368b71 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:14.175593  4691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181238be5f794f7188fdf4eb7f368b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.167161  4821 raft_consensus.cc:740] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.212918  4821 consensus_queue.cc:260] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.217936  4822 ts_tablet_manager.cc:1434] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.112s	user 0.001s	sys 0.008s
I20260709 13:15:14.219247  4822 tablet_bootstrap.cc:492] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:14.220184  4830 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.220755  4830 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.198395  4878 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.222218  4878 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.222833  4830 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:14.224466  4878 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:14.238322  4887 raft_consensus.cc:2749] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:14.241358  4822 tablet_bootstrap.cc:654] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:14.249917  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 99347us
I20260709 13:15:14.250664  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:14.257910  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:14.259431  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:14.261090  4594 leader_election.cc:343] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.262102  4824 raft_consensus.cc:359] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.262936  4824 raft_consensus.cc:385] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.263248  4824 raft_consensus.cc:740] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.263957  4824 consensus_queue.cc:260] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.236852  4839 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
W20260709 13:15:14.262166  4740 leader_election.cc:343] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.274026  4830 consensus_queue.cc:1048] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:14.276149  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:14.277077  4830 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.277379  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:14.277686  4830 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.278122  4700 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.281237  4830 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:14.281463  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:14.282205  4775 raft_consensus.cc:2468] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:14.283520  4594 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:14.284381  4830 raft_consensus.cc:2804] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:14.284898  4824 ts_tablet_manager.cc:1434] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.205s	user 0.004s	sys 0.003s
I20260709 13:15:14.284843  4830 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.285360  4830 raft_consensus.cc:3060] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.286091  4824 tablet_bootstrap.cc:492] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:14.289849  4764 tablet_service.cc:1511] Processing CreateTablet for tablet 181238be5f794f7188fdf4eb7f368b71 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:14.291313  4764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181238be5f794f7188fdf4eb7f368b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.293566  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:14.293715  4593 leader_election.cc:343] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.294333  4775 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:14.293756  4830 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.297510  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:14.298229  4775 raft_consensus.cc:3060] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.298655  4519 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:14.300037  4878 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:14.300575  4878 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.301075  4878 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.306670  4775 raft_consensus.cc:2468] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:14.307219  4824 tablet_bootstrap.cc:654] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.308300  4594 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:14.309329  4875 raft_consensus.cc:2804] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:14.309132  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:14.309864  4875 raft_consensus.cc:697] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:14.310710  4875 consensus_queue.cc:237] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.294914  4593 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:14.322263  4831 raft_consensus.cc:2749] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:15:14.324834  4593 leader_election.cc:343] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.309365  4830 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:14.326875  4869 consensus_queue.cc:1048] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:14.333294  4438 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 76653.8us
I20260709 13:15:14.336575  4821 ts_tablet_manager.cc:1434] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.207s	user 0.026s	sys 0.044s
I20260709 13:15:14.340381  4821 tablet_bootstrap.cc:492] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:14.347460  4821 tablet_bootstrap.cc:654] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.340047  4700 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
W20260709 13:15:14.365913  4436 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 103356us
I20260709 13:15:14.371008  4537 tablet_service.cc:1511] Processing CreateTablet for tablet 779c16a6075e4893824178771ade71a1 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:14.353636  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:14.403002  4823 tablet_bootstrap.cc:492] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:14.410004  4537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 779c16a6075e4893824178771ade71a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.414269  4691 tablet_service.cc:1511] Processing CreateTablet for tablet 779c16a6075e4893824178771ade71a1 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260709 13:15:14.403739  4593 leader_election.cc:343] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.415834  4691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 779c16a6075e4893824178771ade71a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.414767  4823 ts_tablet_manager.cc:1403] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.381s	user 0.018s	sys 0.011s
I20260709 13:15:14.403237  4741 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:14.425984  4839 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:14.426525  4839 raft_consensus.cc:697] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:14.421312  4618 tablet_service.cc:1511] Processing CreateTablet for tablet 779c16a6075e4893824178771ade71a1 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:14.416496  4889 raft_consensus.cc:493] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.427369  4839 consensus_queue.cc:237] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.427537  4889 raft_consensus.cc:515] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.429158  4618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 779c16a6075e4893824178771ade71a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.431676  4824 tablet_bootstrap.cc:492] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:14.432258  4824 ts_tablet_manager.cc:1403] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.146s	user 0.018s	sys 0.000s
I20260709 13:15:14.424264  4878 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.436616  4889 leader_election.cc:290] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:14.437332  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:14.439148  4517 leader_election.cc:343] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.439705  4611 tablet_service.cc:1511] Processing CreateTablet for tablet 3079151438004a9a975fdbe1b5b54378 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:14.440833  4829 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.441401  4829 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.442137  4539 tablet_service.cc:1511] Processing CreateTablet for tablet 3079151438004a9a975fdbe1b5b54378 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:14.443424  4539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3079151438004a9a975fdbe1b5b54378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.443691  4829 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:14.440990  4611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3079151438004a9a975fdbe1b5b54378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.445472  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:14.440395  4823 raft_consensus.cc:359] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.447058  4823 raft_consensus.cc:385] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:15:14.447345  4666 leader_election.cc:343] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.447372  4823 raft_consensus.cc:740] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.448599  4823 consensus_queue.cc:260] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.449154  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:14.450368  4627 raft_consensus.cc:2468] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:14.451942  4740 leader_election.cc:304] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:14.453143  4889 raft_consensus.cc:2804] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:14.453612  4889 raft_consensus.cc:493] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.454026  4889 raft_consensus.cc:3060] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.456403  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:14.464156  4765 tablet_service.cc:1511] Processing CreateTablet for tablet 3079151438004a9a975fdbe1b5b54378 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:14.466070  4765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3079151438004a9a975fdbe1b5b54378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.469864  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:14.475322  4821 tablet_bootstrap.cc:492] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:14.476092  4821 ts_tablet_manager.cc:1403] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.136s	user 0.015s	sys 0.009s
W20260709 13:15:14.478229  4666 leader_election.cc:343] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.478781  4666 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:14.480928  4829 raft_consensus.cc:2749] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:14.483198  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:14.483865  4775 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.484449  4889 raft_consensus.cc:515] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.484967  4451 catalog_manager.cc:5697] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:14.488428  4688 tablet_service.cc:1511] Processing CreateTablet for tablet c4e396207b384d2aac3bf87ded2fd828 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:14.489352  4878 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:14.489811  4688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e396207b384d2aac3bf87ded2fd828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.440790  4822 tablet_bootstrap.cc:492] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:14.490473  4822 ts_tablet_manager.cc:1403] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.271s	user 0.043s	sys 0.026s
I20260709 13:15:14.490675  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:14.490782  4824 raft_consensus.cc:359] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.491587  4824 raft_consensus.cc:385] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.491910  4824 raft_consensus.cc:740] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
W20260709 13:15:14.493186  4517 leader_election.cc:343] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.493361  4822 raft_consensus.cc:359] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.494107  4822 raft_consensus.cc:385] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.494421  4822 raft_consensus.cc:740] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.495180  4822 consensus_queue.cc:260] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.497628  4775 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:14.498513  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:14.500844  4627 raft_consensus.cc:3060] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:14.499289  4741 leader_election.cc:343] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.507431  4821 raft_consensus.cc:359] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.508255  4821 raft_consensus.cc:385] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.508769  4889 leader_election.cc:290] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:14.509125  4519 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:14.509295  4824 consensus_queue.cc:260] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.510521  4878 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:14.511309  4878 raft_consensus.cc:697] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:14.512187  4878 consensus_queue.cc:237] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.519985  4824 ts_tablet_manager.cc:1434] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.068s	user 0.004s	sys 0.003s
I20260709 13:15:14.521184  4824 tablet_bootstrap.cc:492] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:14.519795  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
W20260709 13:15:14.523332  4741 leader_election.cc:343] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.508615  4821 raft_consensus.cc:740] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.533674  4821 consensus_queue.cc:260] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:14.525558  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 47368.4us
I20260709 13:15:14.478215  4823 ts_tablet_manager.cc:1434] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.058s	user 0.003s	sys 0.005s
I20260709 13:15:14.536072  4823 tablet_bootstrap.cc:492] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:14.530833  4822 ts_tablet_manager.cc:1434] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:15:14.540077  4448 catalog_manager.cc:5697] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:14.541765  4822 tablet_bootstrap.cc:492] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:14.542433  4821 ts_tablet_manager.cc:1434] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 13:15:14.548089  4824 tablet_bootstrap.cc:654] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.544668  4821 tablet_bootstrap.cc:492] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:14.555361  4543 tablet_service.cc:1511] Processing CreateTablet for tablet c4e396207b384d2aac3bf87ded2fd828 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:14.558585  4543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e396207b384d2aac3bf87ded2fd828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.556674  4764 tablet_service.cc:1511] Processing CreateTablet for tablet c4e396207b384d2aac3bf87ded2fd828 (DEFAULT_TABLE table=test-workload [id=a4b690bde17245419f655f8ee4ad32e2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:14.569557  4764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e396207b384d2aac3bf87ded2fd828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:14.567040  4823 tablet_bootstrap.cc:654] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.573750  4822 tablet_bootstrap.cc:654] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.556294  4627 raft_consensus.cc:2468] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:14.597812  4821 tablet_bootstrap.cc:654] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.602769  4740 leader_election.cc:304] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:14.604163  4839 raft_consensus.cc:2804] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:14.604878  4824 tablet_bootstrap.cc:492] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:14.605444  4824 ts_tablet_manager.cc:1403] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.003s
I20260709 13:15:14.604975  4448 catalog_manager.cc:5697] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:14.604681  4839 raft_consensus.cc:697] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:14.607659  4839 consensus_queue.cc:237] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.609599  4824 raft_consensus.cc:359] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.610396  4824 raft_consensus.cc:385] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.610687  4824 raft_consensus.cc:740] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.611719  4824 consensus_queue.cc:260] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.614308  4824 ts_tablet_manager.cc:1434] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:15:14.615341  4889 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.615916  4889 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.615316  4824 tablet_bootstrap.cc:492] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:14.623467  4889 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:14.625998  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:14.626113  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:14.627110  4553 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:14.628455  4824 tablet_bootstrap.cc:654] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.633414  4821 tablet_bootstrap.cc:492] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:14.628901  4740 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:14.634004  4821 ts_tablet_manager.cc:1403] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.090s	user 0.012s	sys 0.000s
W20260709 13:15:14.634266  4741 leader_election.cc:343] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.637697  4889 raft_consensus.cc:2804] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:14.638185  4889 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.638540  4889 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.647045  4821 raft_consensus.cc:359] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.647964  4821 raft_consensus.cc:385] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.648243  4821 raft_consensus.cc:740] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.648962  4821 consensus_queue.cc:260] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.651225  4821 ts_tablet_manager.cc:1434] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:15:14.652243  4821 tablet_bootstrap.cc:492] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:14.655617  4451 catalog_manager.cc:5697] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:14.656690  4822 tablet_bootstrap.cc:492] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:14.657218  4822 ts_tablet_manager.cc:1403] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.116s	user 0.016s	sys 0.012s
I20260709 13:15:14.657639  4821 tablet_bootstrap.cc:654] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.663018  4823 tablet_bootstrap.cc:492] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:14.663776  4823 ts_tablet_manager.cc:1403] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.128s	user 0.036s	sys 0.007s
I20260709 13:15:14.667754  4823 raft_consensus.cc:359] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.668551  4823 raft_consensus.cc:385] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.669518  4823 raft_consensus.cc:740] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.670258  4823 consensus_queue.cc:260] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.672530  4823 ts_tablet_manager.cc:1434] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:15:14.673770  4823 tablet_bootstrap.cc:492] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:14.677990  4889 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.660296  4822 raft_consensus.cc:359] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.680024  4824 tablet_bootstrap.cc:492] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:14.681986  4889 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:14.682394  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:14.682617  4824 ts_tablet_manager.cc:1403] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.068s	user 0.022s	sys 0.005s
I20260709 13:15:14.683185  4553 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.680696  4823 tablet_bootstrap.cc:654] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.686257  4824 raft_consensus.cc:359] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.687119  4824 raft_consensus.cc:385] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.687431  4824 raft_consensus.cc:740] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.688364  4824 consensus_queue.cc:260] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.690569  4821 tablet_bootstrap.cc:492] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:14.691320  4821 ts_tablet_manager.cc:1403] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.006s
I20260709 13:15:14.695546  4821 raft_consensus.cc:359] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.696278  4553 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:14.696586  4821 raft_consensus.cc:385] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.697098  4821 raft_consensus.cc:740] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.697641  4898 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.697992  4740 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:14.698271  4898 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.681519  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:14.698038  4821 consensus_queue.cc:260] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.700387  4889 raft_consensus.cc:2804] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:14.681327  4822 raft_consensus.cc:385] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:15:14.701627  4741 leader_election.cc:343] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.701932  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:14.703510  4889 raft_consensus.cc:697] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:14.703902  4898 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:14.702051  4822 raft_consensus.cc:740] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.704475  4889 consensus_queue.cc:237] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.705400  4823 tablet_bootstrap.cc:492] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:14.706079  4823 ts_tablet_manager.cc:1403] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.013s
I20260709 13:15:14.706230  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:14.706426  4822 consensus_queue.cc:260] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.708344  4666 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:14.708971  4823 raft_consensus.cc:359] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.709712  4823 raft_consensus.cc:385] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.709988  4823 raft_consensus.cc:740] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.710777  4823 consensus_queue.cc:260] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.711112  4898 raft_consensus.cc:2749] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:14.711704  4821 ts_tablet_manager.cc:1434] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.002s
I20260709 13:15:14.712849  4821 tablet_bootstrap.cc:492] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:14.712800  4823 ts_tablet_manager.cc:1434] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:15:14.714202  4823 tablet_bootstrap.cc:492] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:14.714417  4824 ts_tablet_manager.cc:1434] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:15:14.719314  4824 tablet_bootstrap.cc:492] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:14.720095  4823 tablet_bootstrap.cc:654] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.720188  4821 tablet_bootstrap.cc:654] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.713044  4822 ts_tablet_manager.cc:1434] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.000s
I20260709 13:15:14.726449  4822 tablet_bootstrap.cc:492] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:14.733273  4821 tablet_bootstrap.cc:492] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:14.733949  4821 ts_tablet_manager.cc:1403] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.006s
I20260709 13:15:14.734233  4824 tablet_bootstrap.cc:654] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.737880  4821 raft_consensus.cc:359] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.739264  4821 raft_consensus.cc:385] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.739418  4822 tablet_bootstrap.cc:654] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.739820  4821 raft_consensus.cc:740] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.741146  4821 consensus_queue.cc:260] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.743911  4887 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.744400  4887 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.746840  4887 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:14.747699  4823 tablet_bootstrap.cc:492] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:14.747628  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:14.748407  4823 ts_tablet_manager.cc:1403] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.011s
I20260709 13:15:14.748523  4701 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:14.749394  4821 ts_tablet_manager.cc:1434] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.001s
I20260709 13:15:14.748579  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:14.750222  4553 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:14.750727  4821 tablet_bootstrap.cc:492] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:14.751318  4823 raft_consensus.cc:359] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.751519  4740 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:14.752507  4887 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:14.752907  4887 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.753260  4887 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.754120  4448 catalog_manager.cc:5697] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:14.756397  4821 tablet_bootstrap.cc:654] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.756896  4823 raft_consensus.cc:385] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.757261  4823 raft_consensus.cc:740] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.758118  4823 consensus_queue.cc:260] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.760964  4887 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.764021  4824 tablet_bootstrap.cc:492] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:14.764012  4823 ts_tablet_manager.cc:1434] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:15:14.764714  4824 ts_tablet_manager.cc:1403] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.001s
I20260709 13:15:14.765738  4823 tablet_bootstrap.cc:492] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:14.766446  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:14.767159  4701 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.772706  4824 raft_consensus.cc:359] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.775972  4824 raft_consensus.cc:385] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.775983  4823 tablet_bootstrap.cc:654] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.776400  4824 raft_consensus.cc:740] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.775022  4822 tablet_bootstrap.cc:492] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:14.777822  4822 ts_tablet_manager.cc:1403] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.000s
I20260709 13:15:14.781096  4822 raft_consensus.cc:359] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.781888  4822 raft_consensus.cc:385] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.782195  4822 raft_consensus.cc:740] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.782956  4822 consensus_queue.cc:260] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.787343  4887 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:14.788311  4824 consensus_queue.cc:260] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.789647  4701 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:14.791734  4824 ts_tablet_manager.cc:1434] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:15:14.793385  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:14.794112  4553 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.795476  4822 ts_tablet_manager.cc:1434] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:15:14.796650  4822 tablet_bootstrap.cc:492] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:14.801285  4875 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.801306  4887 raft_consensus.cc:493] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.802138  4875 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.802305  4887 raft_consensus.cc:515] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.804625  4822 tablet_bootstrap.cc:654] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.806275  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:14.807782  4741 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:14.808413  4823 tablet_bootstrap.cc:492] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:14.808903  4889 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:14.809033  4823 ts_tablet_manager.cc:1403] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.018s
W20260709 13:15:14.809031  4740 leader_election.cc:343] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.809715  4889 raft_consensus.cc:697] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:14.811452  4887 leader_election.cc:290] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:14.811131  4889 consensus_queue.cc:237] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.812067  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:14.813655  4740 leader_election.cc:343] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.814111  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:14.814817  4551 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:14.816080  4592 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:14.816370  4875 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:14.816610  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:14.817028  4830 raft_consensus.cc:2804] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:14.817485  4830 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.817968  4830 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.819970  4823 raft_consensus.cc:359] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.820806  4823 raft_consensus.cc:385] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.821128  4823 raft_consensus.cc:740] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.821998  4823 consensus_queue.cc:260] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.817360  4701 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:14.814107  4740 leader_election.cc:304] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:14.826789  4887 raft_consensus.cc:2749] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:14.828220  4823 ts_tablet_manager.cc:1434] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:15:14.828536  4821 tablet_bootstrap.cc:492] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:14.830086  4821 ts_tablet_manager.cc:1403] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.080s	user 0.026s	sys 0.018s
I20260709 13:15:14.830572  4823 tablet_bootstrap.cc:492] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:14.828276  4830 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.835522  4822 tablet_bootstrap.cc:492] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:14.835366  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:14.836037  4551 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.837443  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:14.838096  4701 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.838786  4823 tablet_bootstrap.cc:654] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.847738  4553 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:14.847880  4701 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:14.849248  4822 ts_tablet_manager.cc:1403] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.015s
I20260709 13:15:14.849567  4830 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:14.852280  4822 raft_consensus.cc:359] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.853096  4822 raft_consensus.cc:385] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.853387  4822 raft_consensus.cc:740] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.854130  4830 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.854055  4822 consensus_queue.cc:260] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.793707  4824 tablet_bootstrap.cc:492] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:14.854807  4830 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.850497  4821 raft_consensus.cc:359] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.858744  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:14.859526  4775 raft_consensus.cc:2468] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:14.861832  4594 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:14.864099  4593 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:14.866513  4551 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:14.866317  4831 raft_consensus.cc:2804] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:14.867657  4822 ts_tablet_manager.cc:1434] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 13:15:14.858929  4821 raft_consensus.cc:385] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.868366  4821 raft_consensus.cc:740] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.867964  4831 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.869297  4824 tablet_bootstrap.cc:654] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.869590  4831 raft_consensus.cc:3060] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.869681  4821 consensus_queue.cc:260] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.866993  4875 raft_consensus.cc:2804] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:14.881547  4821 ts_tablet_manager.cc:1434] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 13:15:14.876705  4824 tablet_bootstrap.cc:492] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:14.878768  4448 catalog_manager.cc:5697] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:14.892027  4821 tablet_bootstrap.cc:492] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:14.873598  4822 tablet_bootstrap.cc:492] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:14.889632  4875 raft_consensus.cc:697] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:14.894176  4875 consensus_queue.cc:237] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:14.898813  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:14.900172  4551 raft_consensus.cc:2468] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:14.902371  4830 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:14.900038  4831 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.905894  4823 tablet_bootstrap.cc:492] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:14.906071  4824 ts_tablet_manager.cc:1403] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.000s
I20260709 13:15:14.906452  4823 ts_tablet_manager.cc:1403] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.076s	user 0.034s	sys 0.029s
I20260709 13:15:14.909298  4824 raft_consensus.cc:359] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.909592  4701 raft_consensus.cc:3060] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.909655  4823 raft_consensus.cc:359] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.910530  4823 raft_consensus.cc:385] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.910845  4823 raft_consensus.cc:740] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.911563  4823 consensus_queue.cc:260] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.912690  4822 tablet_bootstrap.cc:654] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.917181  4824 raft_consensus.cc:385] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.917575  4824 raft_consensus.cc:740] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.921070  4824 consensus_queue.cc:260] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.923400  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:14.925288  4553 raft_consensus.cc:3060] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.926703  4824 ts_tablet_manager.cc:1434] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:15:14.920718  4821 tablet_bootstrap.cc:654] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.929651  4824 tablet_bootstrap.cc:492] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:14.929764  4823 ts_tablet_manager.cc:1434] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:15:14.930730  4823 tablet_bootstrap.cc:492] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:14.921643  4831 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:14.942071  4822 tablet_bootstrap.cc:492] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:14.942835  4822 ts_tablet_manager.cc:1403] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.069s	user 0.011s	sys 0.001s
I20260709 13:15:14.943547  4451 catalog_manager.cc:5697] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:14.946118  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:14.946779  4775 raft_consensus.cc:3060] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.951022  4551 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.954883  4889 consensus_queue.cc:1048] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:14.956398  4775 raft_consensus.cc:2468] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:14.950785  4822 raft_consensus.cc:359] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.959252  4822 raft_consensus.cc:385] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:14.959739  4822 raft_consensus.cc:740] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:14.968371  4822 consensus_queue.cc:260] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:14.969915  4889 consensus_queue.cc:1048] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:14.973762  4822 ts_tablet_manager.cc:1434] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 13:15:14.974438  4594 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:14.975139  4822 tablet_bootstrap.cc:492] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:14.975145  4878 raft_consensus.cc:493] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.976588  4878 raft_consensus.cc:515] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:14.978673  4878 leader_election.cc:290] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:14.979234  4926 consensus_queue.cc:1048] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:15:14.979725  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:14.979951  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:14.973013  4875 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.981130  4875 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:14.981499  4519 leader_election.cc:343] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:14.982056  4518 leader_election.cc:343] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:14.982506  4518 leader_election.cc:304] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:14.983014  4875 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:14.984042  4627 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:14.984937  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:14.986625  4553 raft_consensus.cc:2468] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:14.986945  4824 tablet_bootstrap.cc:654] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.983778  4878 raft_consensus.cc:2749] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:14.990751  4898 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.991310  4823 tablet_bootstrap.cc:654] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:14.991379  4898 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:14.994676  4926 raft_consensus.cc:2804] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:14.999169  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:14.999846  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:15.000192  4776 raft_consensus.cc:2393] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:14.995405  4926 raft_consensus.cc:697] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:15.005414  4898 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:15.008716  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:15.010766  4839 consensus_queue.cc:1048] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:14.994673  4822 tablet_bootstrap.cc:654] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:15.017136  4668 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:15.018519  4869 raft_consensus.cc:2749] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:15.009581  4926 consensus_queue.cc:237] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:15.071501  4594 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:15.100481  4823 tablet_bootstrap.cc:492] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:15.101819  4823 ts_tablet_manager.cc:1403] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.171s	user 0.022s	sys 0.002s
I20260709 13:15:15.108264  4875 raft_consensus.cc:2749] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:15.109802  4821 tablet_bootstrap.cc:492] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:15.111143  4822 tablet_bootstrap.cc:492] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:15.111763  4821 ts_tablet_manager.cc:1403] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.220s	user 0.014s	sys 0.004s
I20260709 13:15:15.112259  4822 ts_tablet_manager.cc:1403] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.138s	user 0.017s	sys 0.000s
I20260709 13:15:15.118693  4821 raft_consensus.cc:359] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:15.120618  4821 raft_consensus.cc:385] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:15.121614  4821 raft_consensus.cc:740] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:15.123329  4821 consensus_queue.cc:260] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:15.130095  4824 tablet_bootstrap.cc:492] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:15.130290  4821 ts_tablet_manager.cc:1434] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.008s
I20260709 13:15:15.131323  4824 ts_tablet_manager.cc:1403] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.202s	user 0.011s	sys 0.004s
I20260709 13:15:15.134101  4821 tablet_bootstrap.cc:492] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:15.105849  4898 raft_consensus.cc:493] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.137547  4898 raft_consensus.cc:515] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:15.158550  4824 raft_consensus.cc:359] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:15.159312  4824 raft_consensus.cc:385] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:15.159528  4824 raft_consensus.cc:740] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:15.160207  4824 consensus_queue.cc:260] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:15.163345  4822 raft_consensus.cc:359] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:15.164443  4822 raft_consensus.cc:385] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:15.167891  4822 raft_consensus.cc:740] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:15.169286  4822 consensus_queue.cc:260] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:15.165222  4821 tablet_bootstrap.cc:654] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:15.173163  4898 leader_election.cc:290] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:15.177496  4822 ts_tablet_manager.cc:1434] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.065s	user 0.012s	sys 0.003s
I20260709 13:15:15.182669  4822 tablet_bootstrap.cc:492] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:15.190027  4839 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.191111  4839 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:15.211127  4839 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:15.212812  4824 ts_tablet_manager.cc:1434] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.004s
I20260709 13:15:15.214622  4824 tablet_bootstrap.cc:492] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:15.224412  4823 raft_consensus.cc:359] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:15.226229  4823 raft_consensus.cc:385] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:15.226646  4823 raft_consensus.cc:740] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:15.227905  4822 tablet_bootstrap.cc:654] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:15.227490  4823 consensus_queue.cc:260] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:15.235563  4823 ts_tablet_manager.cc:1434] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.132s	user 0.013s	sys 0.000s
W20260709 13:15:16.027756  4740 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.029s	sys 0.025s
W20260709 13:15:16.028458  4740 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.029s	sys 0.025s
I20260709 13:15:16.049199  4946 consensus_queue.cc:1048] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:16.058410  4946 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.059137  4946 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:16.060961  4949 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.066942  4946 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:16.086478  4949 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:16.100034  4949 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:16.104063  4949 consensus_queue.cc:1048] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:16.196842  4949 raft_consensus.cc:493] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.197649  4949 raft_consensus.cc:515] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:16.202921  4949 leader_election.cc:290] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
W20260709 13:15:16.242455  4574 debug-util.cc:398] Leaking SignalData structure 0x7b0800211720 after lost signal to thread 26962
I20260709 13:15:16.307310  4946 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:16.308355  4946 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:16.313537  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:16.343324  4946 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:16.425827  4946 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.426723  4946 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:16.535554  4946 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:16.758821  4946 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:16.759727  4946 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:16.761032  4553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51884 (request call id 10) took 1771 ms. Trace:
I20260709 13:15:16.761721  4553 rpcz_store.cc:276] 0709 13:15:14.989183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.990773 (+  1590us) service_pool.cc:224] Handling call
0709 13:15:14.991179 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:14.994310 (+  3131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:14.994351 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:14.994876 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:15:16.758383 (+1763507us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:15:16.758466 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.758627 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:16.760459 (+  1832us) raft_consensus.cc:1685] finished
0709 13:15:16.760489 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.760792 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1384,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":1842,"threads_started":1,"wal-append.queue_time_us":1494}
W20260709 13:15:16.764209  4552 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.007s	sys 0.033s
W20260709 13:15:16.766623  4552 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.007s	sys 0.033s
W20260709 13:15:16.769927  4951 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.001s	sys 0.034s
W20260709 13:15:16.770489  4951 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.001s	sys 0.035s
I20260709 13:15:16.774715  4839 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.775484  4839 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:16.780026  4839 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:16.794785  4956 raft_consensus.cc:493] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.795895  4956 raft_consensus.cc:515] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:16.798987  4956 leader_election.cc:290] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:16.800500  4957 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.801792  4824 tablet_bootstrap.cc:654] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:16.801802  4957 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:16.802433  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:16.803603  4626 raft_consensus.cc:2393] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:16.805357  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:16.807940  4959 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:17.042959  4959 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.046213  4956 raft_consensus.cc:493] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.047361  4956 raft_consensus.cc:515] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.048101  4824 tablet_bootstrap.cc:492] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:17.048707  4824 ts_tablet_manager.cc:1403] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 1.834s	user 0.095s	sys 0.150s
I20260709 13:15:17.050555  4956 leader_election.cc:290] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.053685  4869 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.053481  4824 raft_consensus.cc:359] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.054651  4824 raft_consensus.cc:385] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:17.055186  4824 raft_consensus.cc:740] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:17.048032  4959 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:17.054867  4869 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.058475  4959 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.043180  4957 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:17.059026  4959 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.059633  4869 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.061255  4824 consensus_queue.cc:260] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.062415  4830 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.063249  4830 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.063756  4448 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42684 (request call id 20) took 2131 ms. Trace:
I20260709 13:15:17.064484  4824 ts_tablet_manager.cc:1434] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.006s
I20260709 13:15:17.066797  4824 tablet_bootstrap.cc:492] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:17.064222  4448 rpcz_store.cc:276] 0709 13:15:14.932153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.932332 (+   179us) service_pool.cc:224] Handling call
0709 13:15:17.063470 (+2131138us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:15:17.076136  4830 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:17.077041  4830 raft_consensus.cc:493] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.078032  4830 raft_consensus.cc:515] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.081065  4824 tablet_bootstrap.cc:654] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:17.080446  4959 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.086539  4830 leader_election.cc:290] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:17.087848  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.095001  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:17.101480  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:17.106927  4822 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 1.874s	user 0.024s	sys 0.023s
W20260709 13:15:17.107558  4822 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 1.875s	user 0.024s	sys 0.024s
I20260709 13:15:17.116550  4963 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.117676  4963 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.121788  4963 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
W20260709 13:15:17.130885  4592 leader_election.cc:343] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:17.132095  4592 leader_election.cc:343] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:17.138422  4889 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:17.139340  4889 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.139761  4958 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:17.140481  4958 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.142274  4839 raft_consensus.cc:493] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.143155  4958 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:17.142843  4839 raft_consensus.cc:515] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.143450  4957 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.144595  4957 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.149745  4950 consensus_queue.cc:1048] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:17.151014  4552 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.148362  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.154259  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.155248  4963 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.155632  4839 leader_election.cc:290] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:17.156080  4963 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.150859  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:17.159974  4963 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:17.160522  4822 tablet_bootstrap.cc:492] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:17.161373  4822 ts_tablet_manager.cc:1403] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 1.980s	user 0.042s	sys 0.024s
I20260709 13:15:17.155303  4627 raft_consensus.cc:2393] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:17.148900  4957 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.179996  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.184928  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:17.185251  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:17.186091  4625 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:17.188803  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.190254  4823 tablet_bootstrap.cc:492] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:17.190897  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:17.193387  4592 leader_election.cc:343] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:17.186959  4627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48806 (request call id 4) took 1094 ms. Trace:
I20260709 13:15:17.202553  4666 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:17.202723  4627 rpcz_store.cc:276] 0709 13:15:16.082843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.153770 (+1070927us) service_pool.cc:224] Handling call
0709 13:15:17.176240 (+ 22470us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:17.201656  4961 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.204227  4963 raft_consensus.cc:2804] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:17.204751  4963 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.204377  4961 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.205121  4963 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.205837  4451 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42678 (request call id 22) took 2269 ms. Trace:
I20260709 13:15:17.207325  4961 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:17.208019  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.208307  4821 tablet_bootstrap.cc:492] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:17.210016  4821 ts_tablet_manager.cc:1403] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 2.078s	user 0.010s	sys 0.038s
W20260709 13:15:17.203706  4666 leader_election.cc:343] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:17.214056  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:17.215245  4824 tablet_bootstrap.cc:492] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:17.215838  4824 ts_tablet_manager.cc:1403] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.150s	user 0.025s	sys 0.035s
I20260709 13:15:17.217046  4969 raft_consensus.cc:493] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.224397  4822 raft_consensus.cc:359] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.220439  4969 raft_consensus.cc:515] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.233773  4969 leader_election.cc:290] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:17.235670  4666 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.226260  4822 raft_consensus.cc:385] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:17.222707  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.237473  4775 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:17.230595  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.239481  4776 raft_consensus.cc:2468] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:17.241279  4822 raft_consensus.cc:740] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:17.241115  4594 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:17.242486  4594 leader_election.cc:304] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:17.243292  4959 raft_consensus.cc:2749] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:17.244007  4950 raft_consensus.cc:2804] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:17.244464  4950 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.245057  4950 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.242277  4822 consensus_queue.cc:260] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.244415  4830 raft_consensus.cc:2804] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:17.240350  4551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51874 (request call id 25) took 1158 ms. Trace:
I20260709 13:15:17.247903  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:17.252050  4594 leader_election.cc:343] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:17.250993  4821 raft_consensus.cc:359] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.253724  4821 raft_consensus.cc:385] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:17.254163  4821 raft_consensus.cc:740] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:17.249442  4551 rpcz_store.cc:276] 0709 13:15:16.082385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.239040 (+1156655us) service_pool.cc:224] Handling call
0709 13:15:17.239496 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:17.239790 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:17.239820 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:17.239864 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:17.239936 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.240019 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:17.240169 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:17.246614  4830 raft_consensus.cc:493] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.256125  4830 raft_consensus.cc:3060] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.252625  4594 leader_election.cc:304] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:17.276046  4946 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.277519  4628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48806 (request call id 3) took 1180 ms. Trace:
I20260709 13:15:17.277568  4898 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.278496  4898 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.278112  4628 rpcz_store.cc:276] 0709 13:15:16.082494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.114115 (+1031621us) service_pool.cc:224] Handling call
0709 13:15:17.114552 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:17.114983 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:17.115014 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:17.115524 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:15:17.211993 (+ 96469us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:15:17.212681 (+   688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.212811 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:17.246820 (+ 34009us) raft_consensus.cc:1685] finished
0709 13:15:17.246856 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:17.262785 (+ 15929us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2802,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":84598,"threads_started":1,"wal-append.queue_time_us":120039}
I20260709 13:15:17.281457  4898 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:17.282006  4822 ts_tablet_manager.cc:1434] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.119s	user 0.008s	sys 0.001s
I20260709 13:15:17.281867  4959 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:17.206256  4451 rpcz_store.cc:276] 0709 13:15:14.937113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.937246 (+   133us) service_pool.cc:224] Handling call
0709 13:15:17.205622 (+2268376us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:14.961437 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:14.962229 (+   792us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:14.962277 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:15:14.962305 (+    28us) tablet.cc:662] Decoding operations
0709 13:15:14.962789 (+   484us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:14.962870 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:14.962914 (+    44us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:14.963201 (+   287us) tablet.cc:701] Row locks acquired
0709 13:15:14.963229 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:15:14.963426 (+   197us) write_op.cc:270] Start()
0709 13:15:14.963547 (+   121us) write_op.cc:276] Timestamp: P: 1783602914963367 usec, L: 0
0709 13:15:14.963570 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:15:14.964206 (+   636us) log.cc:844] Serialized 540 byte log entry
0709 13:15:14.972914 (+  8708us) op_driver.cc:464] REPLICATION: finished
0709 13:15:14.974983 (+  2069us) write_op.cc:301] APPLY: starting
0709 13:15:14.975065 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:14.975157 (+    92us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:14.975264 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:14.976245 (+   981us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:14.976420 (+   175us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:15:14.976464 (+    44us) write_op.cc:312] APPLY: finished
0709 13:15:14.976655 (+   191us) log.cc:844] Serialized 22 byte log entry
0709 13:15:14.976891 (+   236us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:14.976969 (+    78us) write_op.cc:454] Released schema lock
0709 13:15:14.977015 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:14.977316 (+   301us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:14.977341 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":237,"apply.run_cpu_time_us":1642,"apply.run_wall_time_us":2305,"num_ops":1,"prepare.queue_time_us":846,"prepare.run_cpu_time_us":3111,"prepare.run_wall_time_us":5526,"replication_time_us":6918}]]}
I20260709 13:15:17.307785  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.289129  4969 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:17.308801  4771 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:17.309499  4969 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.310705  4950 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.313309  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.313699  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.316498  4773 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:17.317203  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.318271  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.318928  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.320591  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.322355  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.324060  4947 raft_consensus.cc:2749] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:17.325420  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:17.324970  4959 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.326373  4549 raft_consensus.cc:2393] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:17.328895  4959 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:17.330152  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.330505  4822 tablet_bootstrap.cc:492] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:17.331010  4702 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:17.338393  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:17.339342  4547 raft_consensus.cc:2468] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:17.289417  4869 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:17.345291  4869 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.347910  4869 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:17.348963  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.353058  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.353104  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.355654  4887 consensus_queue.cc:1048] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:17.356717  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:17.359333  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.361006  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.356858  4519 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.361761  4770 raft_consensus.cc:2468] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:17.362583  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.363792  4946 raft_consensus.cc:2804] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.364468  4946 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:17.364902  4946 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.364996  4950 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.366050  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:17.366485  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:17.367213  4770 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.367973  4666 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.363854  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:17.371129  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.372537  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.373057  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.373296  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.373859  4695 raft_consensus.cc:2468] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:17.374737  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.375208  4740 leader_election.cc:304] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:17.380257  4947 raft_consensus.cc:493] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:17.380578  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.382099  4947 raft_consensus.cc:515] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.380707  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.373453  4694 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:17.381163  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.385792  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.385900  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:17.386554  4770 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:17.387712  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:17.386101  4742 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.205s	user 0.023s	sys 0.024s
I20260709 13:15:17.390093  4594 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:17.389604  4869 raft_consensus.cc:2749] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:17.391396  4875 raft_consensus.cc:2804] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:17.392097  4875 raft_consensus.cc:697] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:17.392776  4824 raft_consensus.cc:359] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.393591  4875 consensus_queue.cc:237] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:17.394811  4574 debug-util.cc:398] Leaking SignalData structure 0x7b08001a7b60 after lost signal to thread 4485
I20260709 13:15:17.395232  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.356104  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:17.396392  4696 raft_consensus.cc:2468] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:17.400842  4948 consensus_queue.cc:1048] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:17.381786  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.403595  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.404340  4628 raft_consensus.cc:2468] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
W20260709 13:15:17.389561  4742 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.209s	user 0.023s	sys 0.024s
I20260709 13:15:17.405646  4698 raft_consensus.cc:3060] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.406800  4824 raft_consensus.cc:385] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:17.312556  4969 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:17.411571  4823 tablet_bootstrap.cc:654] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:17.413029  4574 debug-util.cc:398] Leaking SignalData structure 0x7b08001fe8e0 after lost signal to thread 4577
I20260709 13:15:17.415218  4518 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:17.418298  4947 leader_election.cc:290] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.418557  4889 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:17.419373  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.404603  4776 raft_consensus.cc:2468] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:17.255000  4963 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.407271  4824 raft_consensus.cc:740] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:17.423650  4821 consensus_queue.cc:260] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.433797  4878 raft_consensus.cc:2804] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.396471  4553 raft_consensus.cc:2468] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:17.420852  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:17.421409  4961 raft_consensus.cc:2804] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:17.443220  4878 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:17.453035  4574 debug-util.cc:398] Leaking SignalData structure 0x7b080004ef00 after lost signal to thread 4652
I20260709 13:15:17.465241  4700 raft_consensus.cc:2393] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:17.465968  4668 leader_election.cc:304] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.466063  4830 raft_consensus.cc:515] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:17.467979  4518 leader_election.cc:343] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:17.452448  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 195552us
I20260709 13:15:17.444286  4769 raft_consensus.cc:2468] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:17.422922  4740 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.473328  4824 consensus_queue.cc:260] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:17.427066  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 70444.8us
I20260709 13:15:17.469700  4878 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.471139  4702 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.499456  4821 ts_tablet_manager.cc:1434] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.289s	user 0.008s	sys 0.003s
I20260709 13:15:17.497409  4517 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.495729  4869 raft_consensus.cc:3060] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.505890  4623 raft_consensus.cc:2393] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 1: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:17.502246  4889 raft_consensus.cc:2749] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:17.503099  4574 debug-util.cc:398] Leaking SignalData structure 0x7b0800269ce0 after lost signal to thread 4726
I20260709 13:15:17.513464  4822 tablet_bootstrap.cc:654] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:17.513355  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.543102  4969 consensus_queue.cc:1048] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:17.520985  4956 raft_consensus.cc:2749] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:17.545061  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:17.522006  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.545903  4770 raft_consensus.cc:3060] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.522583  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:17.512269  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 89460.6us
I20260709 13:15:17.500864  4518 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.549085  4824 ts_tablet_manager.cc:1434] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.333s	user 0.005s	sys 0.008s
I20260709 13:15:17.515249  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:17.516758  4963 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.571374  4821 tablet_bootstrap.cc:492] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:17.523236  4519 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:17.573611  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 50455us
I20260709 13:15:17.574714  4519 leader_election.cc:304] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:17.525274  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.518147  4958 consensus_queue.cc:1048] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
W20260709 13:15:17.552860  4741 leader_election.cc:343] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:17.554477  4830 leader_election.cc:290] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:17.558441  4695 raft_consensus.cc:2468] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:17.578979  4969 consensus_queue.cc:1048] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:17.559593  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 65718.2us
I20260709 13:15:17.560326  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:17.561054  4956 raft_consensus.cc:2804] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:17.518620  4593 leader_election.cc:343] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:17.564898  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.581408  4956 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:17.581977  4956 raft_consensus.cc:3060] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.582037  4961 raft_consensus.cc:493] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.582547  4961 raft_consensus.cc:3060] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.583746  4821 tablet_bootstrap.cc:654] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:17.512930  4668 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:17.586050  4668 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 79043.4us
I20260709 13:15:17.586421  4740 leader_election.cc:304] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:17.587283  4969 raft_consensus.cc:2804] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:17.587751  4969 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.588158  4969 raft_consensus.cc:3060] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.582508  4593 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:17.567530  4878 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.565150  4824 tablet_bootstrap.cc:492] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:17.596519  4973 raft_consensus.cc:2749] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:17.577054  4693 raft_consensus.cc:2468] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
W20260709 13:15:17.590945  4668 leader_election.cc:343] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:17.598604  4668 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.591521  4963 raft_consensus.cc:2749] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:17.594386  4592 leader_election.cc:343] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:17.594066  4956 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
W20260709 13:15:17.569757  4574 debug-util.cc:398] Leaking SignalData structure 0x7b080012cc60 after lost signal to thread 4800
I20260709 13:15:17.598768  4770 raft_consensus.cc:2468] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:17.592036  4518 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:17.565101  4946 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.576582  4823 tablet_bootstrap.cc:492] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:17.600709  4958 raft_consensus.cc:2804] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:17.601315  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 82834us
I20260709 13:15:17.603027  4878 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.623247  4823 ts_tablet_manager.cc:1403] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.433s	user 0.029s	sys 0.026s
I20260709 13:15:17.631300  4823 raft_consensus.cc:359] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.632045  4823 raft_consensus.cc:385] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:17.632364  4823 raft_consensus.cc:740] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:17.623708  4981 raft_consensus.cc:2804] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:17.632906  4981 raft_consensus.cc:493] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.633316  4981 raft_consensus.cc:3060] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.641036  4666 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.604115  4959 raft_consensus.cc:2749] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:17.645388  4957 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.641466  4946 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:17.628111  4958 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
W20260709 13:15:17.623957  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 49320.4us
I20260709 13:15:17.623925  4594 leader_election.cc:304] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:17.658402  4593 leader_election.cc:304] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:17.659969  4824 tablet_bootstrap.cc:654] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:17.651684  4898 raft_consensus.cc:2804] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.663060  4823 consensus_queue.cc:260] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.650468  4957 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.655269  4958 raft_consensus.cc:3060] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.663851  4830 raft_consensus.cc:2749] T 498cf28e26b0420fa93d10f75cedb4b4 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:17.643460  4741 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.649636  4668 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.677723  4668 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:17.678107  4963 raft_consensus.cc:2749] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:17.678972  4963 raft_consensus.cc:2749] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:17.681012  4981 raft_consensus.cc:515] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.681571  4869 raft_consensus.cc:2749] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:17.665771  4898 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:17.675173  4956 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
W20260709 13:15:17.682760  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 69103.4us
I20260709 13:15:17.683010  4898 raft_consensus.cc:3060] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.664454  4875 consensus_queue.cc:1048] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:15:17.684926  4981 leader_election.cc:290] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.686071  4889 raft_consensus.cc:2804] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:17.662024  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:17.690203  4822 tablet_bootstrap.cc:492] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:17.691717  4822 ts_tablet_manager.cc:1403] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.362s	user 0.015s	sys 0.020s
I20260709 13:15:17.660826  4973 consensus_queue.cc:1048] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:15:17.667182  4957 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.696969  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:17.700160  4964 raft_consensus.cc:2804] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:17.690567  4889 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
W20260709 13:15:17.686717  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 62920us
I20260709 13:15:17.712246  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:17.707207  4889 raft_consensus.cc:3060] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.715524  4961 raft_consensus.cc:515] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.715910  4518 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:17.717866  4984 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.703855  4964 raft_consensus.cc:697] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:17.695706  4969 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.720625  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:17.725632  4823 ts_tablet_manager.cc:1434] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.097s	user 0.003s	sys 0.004s
I20260709 13:15:17.724655  4821 tablet_bootstrap.cc:492] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:17.725100  4553 raft_consensus.cc:1240] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:17.722700  4984 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.727638  4821 ts_tablet_manager.cc:1403] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.181s	user 0.005s	sys 0.017s
I20260709 13:15:17.728554  4964 consensus_queue.cc:237] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.729672  4823 tablet_bootstrap.cc:492] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:17.720238  4981 raft_consensus.cc:2749] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:17.735965  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:17.738792  4898 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.739905  4628 raft_consensus.cc:2468] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:17.740222  4696 raft_consensus.cc:1240] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:17.741571  4517 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:17.742524  4956 raft_consensus.cc:2804] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:17.742340  4961 leader_election.cc:290] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:17.742962  4956 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.743386  4956 raft_consensus.cc:3060] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.745002  4898 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:17.745276  4973 consensus_queue.cc:1059] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:17.745779  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:17.746798  4973 raft_consensus.cc:3055] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:17.738837  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:17.747315  4973 raft_consensus.cc:740] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:17.748529  4973 consensus_queue.cc:260] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.751690  4973 raft_consensus.cc:3060] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.745924  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:17.754489  4958 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.755744  4553 raft_consensus.cc:1240] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:17.756510  4549 raft_consensus.cc:1240] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:17.757220  4969 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:17.781409  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:17.782323  4695 raft_consensus.cc:3060] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.782955  4518 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.784998  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:17.787340  4875 consensus_queue.cc:1059] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:17.788767  4830 raft_consensus.cc:3055] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:17.788921  4946 raft_consensus.cc:2749] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:17.789156  4830 raft_consensus.cc:740] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:17.790148  4830 consensus_queue.cc:260] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.790748  4889 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.791625  4830 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.794646  4956 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.798111  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
W20260709 13:15:17.798457  4518 leader_election.cc:343] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:17.798758  4627 raft_consensus.cc:3060] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.791464  4822 raft_consensus.cc:359] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:17.736027  4984 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:17.801301  4822 raft_consensus.cc:385] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:17.779953  4990 consensus_queue.cc:1048] T 498cf28e26b0420fa93d10f75cedb4b4 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260709 13:15:17.759202  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 64623.7us
I20260709 13:15:17.853835  4822 raft_consensus.cc:740] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:17.802299  4956 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:17.811611  4547 raft_consensus.cc:1240] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:17.812911  4519 leader_election.cc:304] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:17.813045  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:17.883350  4951 consensus_queue.cc:1048] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:17.802942  4898 raft_consensus.cc:493] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.884562  4898 raft_consensus.cc:515] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:17.885017  4518 leader_election.cc:343] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:17.814910  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:17.888130  4973 consensus_queue.cc:1059] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:17.888448  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:17.888584  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 75862.8us
I20260709 13:15:17.889081  4973 raft_consensus.cc:3055] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:17.889204  4547 raft_consensus.cc:2468] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:17.889804  4973 raft_consensus.cc:740] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:17.808444  4627 raft_consensus.cc:2468] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:17.810935  4958 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:17.841802  4889 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:17.894635  4823 tablet_bootstrap.cc:654] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:17.886932  4898 leader_election.cc:290] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:17.892097  4822 consensus_queue.cc:260] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:17.855156  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 152124us
I20260709 13:15:17.904155  4821 raft_consensus.cc:359] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.907194  4821 raft_consensus.cc:385] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:17.907505  4821 raft_consensus.cc:740] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:17.896184  4956 raft_consensus.cc:2804] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:17.913003  4821 consensus_queue.cc:260] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.916251  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:17.917165  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:17.914250  4973 consensus_queue.cc:260] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:17.918152  4627 raft_consensus.cc:3060] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.918889  4973 raft_consensus.cc:3060] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:17.920598  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:17.920882  4517 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:17.894794  4666 leader_election.cc:304] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:17.895622  4957 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.909655  4956 raft_consensus.cc:493] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.917182  4547 raft_consensus.cc:2393] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:17.932773  4822 ts_tablet_manager.cc:1434] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.241s	user 0.009s	sys 0.004s
I20260709 13:15:17.923154  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:17.934520  4898 raft_consensus.cc:2804] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:17.936880  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:17.937294  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:17.937930  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:17.938128  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:17.947557  4776 raft_consensus.cc:2393] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:17.929152  4956 raft_consensus.cc:3060] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.948343  4824 tablet_bootstrap.cc:492] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:17.948894  4552 raft_consensus.cc:1240] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:17.949213  4824 ts_tablet_manager.cc:1403] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.384s	user 0.008s	sys 0.017s
I20260709 13:15:17.951247  4628 raft_consensus.cc:2468] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:17.953446  4824 raft_consensus.cc:359] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.954404  4824 raft_consensus.cc:385] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:17.954752  4824 raft_consensus.cc:740] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:17.955538  4824 consensus_queue.cc:260] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.958073  4824 ts_tablet_manager.cc:1434] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:15:17.959285  4824 tablet_bootstrap.cc:492] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:17.944365  4898 raft_consensus.cc:493] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.959108  4951 consensus_queue.cc:1048] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:17.959939  4898 raft_consensus.cc:3060] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:17.925232  4958 raft_consensus.cc:2804] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:17.968500  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:17.969771  4666 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:17.970990  4959 raft_consensus.cc:2804] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:17.971524  4959 raft_consensus.cc:697] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:17.972038  4957 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:17.972513  4959 consensus_queue.cc:237] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:17.974360  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:17.977260  4519 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.979545  4519 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:17.981252  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:17.982350  4984 raft_consensus.cc:2749] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:17.981333  4957 raft_consensus.cc:2749] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:17.966583  4824 tablet_bootstrap.cc:654] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:17.987016  4627 raft_consensus.cc:2468] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:17.988583  4822 tablet_bootstrap.cc:492] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:17.989946  4958 raft_consensus.cc:697] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:18.002311  4958 consensus_queue.cc:237] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.002928  4898 raft_consensus.cc:515] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:17.990520  4740 leader_election.cc:343] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:18.005165  4956 raft_consensus.cc:515] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:17.993673  4519 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.028900  4957 raft_consensus.cc:2749] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:18.030014  4740 leader_election.cc:304] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:17.998068  4821 ts_tablet_manager.cc:1434] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.264s	user 0.008s	sys 0.000s
I20260709 13:15:17.995976  4695 raft_consensus.cc:2468] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
W20260709 13:15:17.992904  4741 leader_election.cc:343] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:18.035823  4962 raft_consensus.cc:2749] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:18.039601  4518 leader_election.cc:304] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:18.045018  4957 raft_consensus.cc:2804] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:18.046929  4821 tablet_bootstrap.cc:492] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:18.048616  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:18.050010  4956 leader_election.cc:290] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
W20260709 13:15:18.045087  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 75874.9us
I20260709 13:15:18.053170  4957 raft_consensus.cc:697] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:18.054823  4957 consensus_queue.cc:237] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:18.037273  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 44396.1us
I20260709 13:15:18.058326  4740 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.068984  4695 raft_consensus.cc:2393] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:18.072631  4821 tablet_bootstrap.cc:654] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.084555  4822 tablet_bootstrap.cc:654] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.087483  4451 catalog_manager.cc:5697] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } }
I20260709 13:15:18.091799  4898 leader_election.cc:290] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:18.089630  4451 catalog_manager.cc:5697] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 1 to 2, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 2 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.092797  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:18.095425  4547 raft_consensus.cc:2393] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:18.100975  4824 tablet_bootstrap.cc:492] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:18.101507  4824 ts_tablet_manager.cc:1403] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.143s	user 0.018s	sys 0.021s
I20260709 13:15:18.097566  4887 raft_consensus.cc:2749] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:18.108009  4740 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.109200  4887 raft_consensus.cc:2749] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:18.113933  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:18.114298  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:18.114725  4770 raft_consensus.cc:3060] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.106514  4824 raft_consensus.cc:359] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.115854  4824 raft_consensus.cc:385] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.116164  4824 raft_consensus.cc:740] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.117172  4824 consensus_queue.cc:260] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.118432  4991 consensus_queue.cc:1059] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:18.118813  4740 leader_election.cc:304] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:18.119623  4990 consensus_queue.cc:1059] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:18.119885  4991 raft_consensus.cc:3055] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:18.120273  4991 raft_consensus.cc:740] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:18.120311  4898 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.120816  4889 raft_consensus.cc:3055] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:18.120463  4990 raft_consensus.cc:2749] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:18.121124  4898 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.121354  4991 consensus_queue.cc:260] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.121265  4889 raft_consensus.cc:740] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:18.122941  4991 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:18.123029  4889 consensus_queue.cc:260] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.123974  4898 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:18.124470  4889 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:18.125255  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:18.126020  4824 ts_tablet_manager.cc:1434] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:15:18.126230  4627 raft_consensus.cc:2393] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:18.128142  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:18.129356  4519 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:18.132543  4824 tablet_bootstrap.cc:492] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:18.135565  4666 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.137125  4898 raft_consensus.cc:3060] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.138262  4823 tablet_bootstrap.cc:492] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:18.138720  4823 ts_tablet_manager.cc:1403] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.409s	user 0.016s	sys 0.024s
I20260709 13:15:18.132328  4956 raft_consensus.cc:2749] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:18.143298  4776 raft_consensus.cc:1240] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:18.145485  4973 consensus_queue.cc:1059] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:18.145902  4770 raft_consensus.cc:2468] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:18.146394  4951 raft_consensus.cc:3055] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:18.146747  4951 raft_consensus.cc:740] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:18.147207  4519 leader_election.cc:304] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:18.148231  4956 raft_consensus.cc:2804] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:18.147791  4951 consensus_queue.cc:260] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.148795  4956 raft_consensus.cc:697] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:18.149480  4951 raft_consensus.cc:3060] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:18.149729  4956 consensus_queue.cc:237] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.153846  4898 raft_consensus.cc:2749] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:18.160266  4823 raft_consensus.cc:359] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.162066  4823 raft_consensus.cc:385] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.162268  4823 raft_consensus.cc:740] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.164634  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:18.169577  4823 consensus_queue.cc:260] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.170122  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:18.171204  4770 raft_consensus.cc:2393] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:18.172016  4823 ts_tablet_manager.cc:1434] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.033s	user 0.003s	sys 0.002s
I20260709 13:15:18.173583  4823 tablet_bootstrap.cc:492] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:18.174602  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:18.175603  4776 raft_consensus.cc:2393] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:18.176949  4668 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.178267  4898 raft_consensus.cc:2749] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:18.179311  4668 leader_election.cc:304] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:18.180845  4898 raft_consensus.cc:2749] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:15:18.183243  4592 consensus_peers.cc:597] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 -> Peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Couldn't send request to peer 978db1e31e16405182e65e88f3b9bebc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:18.178946  4824 tablet_bootstrap.cc:654] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.184964  4823 tablet_bootstrap.cc:654] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.185005  4740 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.188905  4889 raft_consensus.cc:2749] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:18.190114  4822 tablet_bootstrap.cc:492] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:18.190268  4740 leader_election.cc:304] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:18.190630  4822 ts_tablet_manager.cc:1403] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.202s	user 0.017s	sys 0.003s
I20260709 13:15:18.191075  4889 raft_consensus.cc:2804] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:18.191824  4889 raft_consensus.cc:697] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:18.192764  4889 consensus_queue.cc:237] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.193523  4822 raft_consensus.cc:359] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.194236  4822 raft_consensus.cc:385] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.194501  4822 raft_consensus.cc:740] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.195314  4822 consensus_queue.cc:260] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.198045  4822 ts_tablet_manager.cc:1434] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:15:18.199198  4822 tablet_bootstrap.cc:492] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:18.212682  4821 tablet_bootstrap.cc:492] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:18.213331  4821 ts_tablet_manager.cc:1403] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.167s	user 0.017s	sys 0.003s
I20260709 13:15:18.214422  4824 tablet_bootstrap.cc:492] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:18.214907  4824 ts_tablet_manager.cc:1403] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.088s	user 0.011s	sys 0.006s
I20260709 13:15:18.217571  4823 tablet_bootstrap.cc:492] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:18.218111  4823 ts_tablet_manager.cc:1403] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.045s	user 0.006s	sys 0.012s
I20260709 13:15:18.217989  4821 raft_consensus.cc:359] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.218781  4821 raft_consensus.cc:385] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.219107  4821 raft_consensus.cc:740] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
W20260709 13:15:18.220438  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:18.221181  4821 consensus_queue.cc:260] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.222401  4822 tablet_bootstrap.cc:654] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.224455  4821 ts_tablet_manager.cc:1434] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:15:18.225919  4821 tablet_bootstrap.cc:492] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:18.234189  4821 tablet_bootstrap.cc:654] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.221218  4823 raft_consensus.cc:359] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.234675  4823 raft_consensus.cc:385] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.234973  4823 raft_consensus.cc:740] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.235307  4824 raft_consensus.cc:359] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.235740  4823 consensus_queue.cc:260] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.236373  4824 raft_consensus.cc:385] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.238093  4823 ts_tablet_manager.cc:1434] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:15:18.236711  4824 raft_consensus.cc:740] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.239392  4823 tablet_bootstrap.cc:492] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:18.239676  4824 consensus_queue.cc:260] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.240810  4448 catalog_manager.cc:5697] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:18.246729  4973 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.247308  4973 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.253937  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:18.254868  4776 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:18.256901  4594 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:18.258155  4951 raft_consensus.cc:2804] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:18.258643  4951 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.258976  4951 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.266681  4824 ts_tablet_manager.cc:1434] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.052s	user 0.011s	sys 0.003s
I20260709 13:15:18.268414  4824 tablet_bootstrap.cc:492] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:18.273151  4973 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:18.275242  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:18.277254  4592 leader_election.cc:343] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:18.283325  4823 tablet_bootstrap.cc:654] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.287061  4824 tablet_bootstrap.cc:654] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.280809  4951 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.292838  4973 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.293700  4973 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.293720  4964 consensus_queue.cc:1048] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:15:18.301192  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:18.302292  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
W20260709 13:15:18.303253  4593 leader_election.cc:343] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:18.299466  4449 catalog_manager.cc:5697] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } }
W20260709 13:15:18.303956  4592 leader_election.cc:343] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:18.303278  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:18.305042  4770 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.305497  4973 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:18.306146  4951 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:18.310088  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:18.312384  4594 leader_election.cc:343] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:18.313062  4594 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:18.315430  4951 raft_consensus.cc:2749] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:18.297544  4822 tablet_bootstrap.cc:492] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:18.323166  4822 ts_tablet_manager.cc:1403] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.124s	user 0.019s	sys 0.004s
I20260709 13:15:18.315284  4770 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:18.329035  4823 tablet_bootstrap.cc:492] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:18.329869  4823 ts_tablet_manager.cc:1403] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.091s	user 0.018s	sys 0.016s
I20260709 13:15:18.330317  4594 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:18.331861  4951 raft_consensus.cc:2804] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:18.332754  4951 raft_consensus.cc:697] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:18.333830  4951 consensus_queue.cc:237] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.335193  4821 tablet_bootstrap.cc:492] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:18.335829  4821 ts_tablet_manager.cc:1403] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.110s	user 0.016s	sys 0.017s
I20260709 13:15:18.339506  4821 raft_consensus.cc:359] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.340391  4821 raft_consensus.cc:385] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.340704  4821 raft_consensus.cc:740] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.341326  4823 raft_consensus.cc:359] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.343366  4823 raft_consensus.cc:385] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.343678  4823 raft_consensus.cc:740] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.344480  4823 consensus_queue.cc:260] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.346920  4823 ts_tablet_manager.cc:1434] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:15:18.347951  4824 tablet_bootstrap.cc:492] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:18.348687  4824 ts_tablet_manager.cc:1403] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.006s
I20260709 13:15:18.348121  4823 tablet_bootstrap.cc:492] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:18.354427  4821 consensus_queue.cc:260] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.357224  4821 ts_tablet_manager.cc:1434] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.004s
I20260709 13:15:18.358544  4821 tablet_bootstrap.cc:492] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:18.365975  4821 tablet_bootstrap.cc:654] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.364528  4823 tablet_bootstrap.cc:654] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.379256  4450 catalog_manager.cc:5697] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.384771  4823 tablet_bootstrap.cc:492] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:18.385337  4821 tablet_bootstrap.cc:492] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:18.385435  4823 ts_tablet_manager.cc:1403] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.010s
I20260709 13:15:18.386082  4821 ts_tablet_manager.cc:1403] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.003s
I20260709 13:15:18.385911  4824 raft_consensus.cc:359] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.386962  4824 raft_consensus.cc:385] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.387322  4824 raft_consensus.cc:740] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.388239  4824 consensus_queue.cc:260] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.389429  4823 raft_consensus.cc:359] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.389469  4898 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.390549  4823 raft_consensus.cc:385] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.390893  4823 raft_consensus.cc:740] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.390733  4898 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.391940  4823 consensus_queue.cc:260] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.392396  4824 ts_tablet_manager.cc:1434] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:15:18.393997  4824 tablet_bootstrap.cc:492] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:18.389429  4959 raft_consensus.cc:493] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.397631  4823 ts_tablet_manager.cc:1434] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:15:18.397487  4959 raft_consensus.cc:515] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.400041  4959 leader_election.cc:290] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:18.402299  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:18.403167  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:18.403937  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:18.404910  4623 raft_consensus.cc:2393] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:18.403270  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:18.406598  4666 leader_election.cc:304] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.403152  4547 raft_consensus.cc:2468] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:18.408694  4959 raft_consensus.cc:3060] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.408962  4666 leader_election.cc:304] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:18.409941  4988 raft_consensus.cc:2804] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:18.410388  4988 raft_consensus.cc:493] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.410753  4988 raft_consensus.cc:3060] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.412350  4824 tablet_bootstrap.cc:654] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.406841  4823 tablet_bootstrap.cc:492] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:18.415753  4898 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:18.394011  4822 raft_consensus.cc:359] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.420717  4822 raft_consensus.cc:385] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.421065  4822 raft_consensus.cc:740] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.423563  4822 consensus_queue.cc:260] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.424630  4627 raft_consensus.cc:2468] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:18.425213  4988 raft_consensus.cc:515] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.428119  4823 tablet_bootstrap.cc:654] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.426594  4821 raft_consensus.cc:359] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.429275  4821 raft_consensus.cc:385] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.429600  4821 raft_consensus.cc:740] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.430805  4821 consensus_queue.cc:260] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.433157  4822 ts_tablet_manager.cc:1434] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.110s	user 0.009s	sys 0.000s
I20260709 13:15:18.434979  4821 ts_tablet_manager.cc:1434] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:15:18.436928  4822 tablet_bootstrap.cc:492] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:18.440639  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:18.446197  4824 tablet_bootstrap.cc:492] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:18.447803  4821 tablet_bootstrap.cc:492] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:18.447634  4448 catalog_manager.cc:5697] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.449295  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:18.450086  4956 consensus_queue.cc:1048] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:18.467072  4547 raft_consensus.cc:3060] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.467811  4623 raft_consensus.cc:3060] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.453397  4988 leader_election.cc:290] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:18.472922  4448 catalog_manager.cc:5697] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.477625  4623 raft_consensus.cc:2468] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:18.466051  4824 ts_tablet_manager.cc:1403] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.072s	user 0.012s	sys 0.008s
I20260709 13:15:18.479516  4666 leader_election.cc:304] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:18.481127  4898 raft_consensus.cc:2804] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:18.473639  4821 tablet_bootstrap.cc:654] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.481719  4898 raft_consensus.cc:697] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:18.486786  4898 consensus_queue.cc:237] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.489029  4552 raft_consensus.cc:1275] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:18.491519  4988 consensus_queue.cc:1048] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:18.496160  4824 raft_consensus.cc:359] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.497155  4824 raft_consensus.cc:385] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.498219  4823 tablet_bootstrap.cc:492] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:18.497471  4824 raft_consensus.cc:740] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.500082  4823 ts_tablet_manager.cc:1403] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.008s
I20260709 13:15:18.505954  4824 consensus_queue.cc:260] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.508373  4878 consensus_queue.cc:1048] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:18.511699  4547 raft_consensus.cc:2468] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:18.512358  4822 tablet_bootstrap.cc:654] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.508967  4824 ts_tablet_manager.cc:1434] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.004s
I20260709 13:15:18.532337  4824 tablet_bootstrap.cc:492] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:18.537933  4959 raft_consensus.cc:2749] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:18.538972  4889 raft_consensus.cc:493] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.547879  4823 raft_consensus.cc:359] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.549103  4823 raft_consensus.cc:385] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.549494  4823 raft_consensus.cc:740] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.550719  4823 consensus_queue.cc:260] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.544445  4889 raft_consensus.cc:515] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.560971  4823 ts_tablet_manager.cc:1434] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.002s
I20260709 13:15:18.562681  4823 tablet_bootstrap.cc:492] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:18.563794  4447 catalog_manager.cc:5697] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.568882  4889 leader_election.cc:290] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:18.571307  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:18.572654  4824 tablet_bootstrap.cc:654] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.572512  4627 raft_consensus.cc:2468] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:18.579885  4823 tablet_bootstrap.cc:654] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.579958  4740 leader_election.cc:304] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:18.582954  4991 raft_consensus.cc:2804] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:18.583416  4991 raft_consensus.cc:493] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.583796  4991 raft_consensus.cc:3060] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.594175  4821 tablet_bootstrap.cc:492] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:18.594710  4821 ts_tablet_manager.cc:1403] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.147s	user 0.013s	sys 0.012s
I20260709 13:15:18.593092  4959 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.598070  4451 catalog_manager.cc:5697] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 0 to 1, leader changed from <none> to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 1 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:18.601830  4959 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.601982  4958 consensus_queue.cc:1048] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:15:18.605413  4627 raft_consensus.cc:1275] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:18.609467  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:18.610240  4770 raft_consensus.cc:2468] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:18.611070  4898 consensus_queue.cc:1048] T 40db247913f74803b44a6878860081b7 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:18.614105  4889 consensus_queue.cc:1048] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:18.614933  4668 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:18.617462  4988 raft_consensus.cc:2804] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:18.617887  4988 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.618249  4988 raft_consensus.cc:3060] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.620682  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:18.623687  4547 raft_consensus.cc:2468] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:18.632959  4878 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.633674  4878 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.636399  4878 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:18.641426  4958 raft_consensus.cc:493] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.642086  4958 raft_consensus.cc:515] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.643355  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:18.645026  4984 consensus_queue.cc:1048] T d356c2066af241e4a89fad36578a7f89 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:18.646673  4984 consensus_queue.cc:1048] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:18.604627  4959 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:18.610000  4821 raft_consensus.cc:359] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.648633  4821 raft_consensus.cc:385] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.648973  4821 raft_consensus.cc:740] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.649896  4821 consensus_queue.cc:260] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.652139  4821 ts_tablet_manager.cc:1434] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 13:15:18.653172  4821 tablet_bootstrap.cc:492] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:18.654670  4958 leader_election.cc:290] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:18.675420  4449 catalog_manager.cc:5697] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:18.703311  5033 log.cc:927] Time spent T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.114s	user 0.002s	sys 0.001s
I20260709 13:15:18.705116  4991 raft_consensus.cc:515] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.707738  4991 leader_election.cc:290] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:18.709470  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:18.710304  4547 raft_consensus.cc:2468] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:18.713487  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:18.714632  4776 raft_consensus.cc:2393] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:18.698428  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:18.715205  4988 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.715752  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:18.715936  4623 raft_consensus.cc:2468] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:18.716637  4769 raft_consensus.cc:2393] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:18.723331  4822 tablet_bootstrap.cc:492] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:18.724000  4822 ts_tablet_manager.cc:1403] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.287s	user 0.018s	sys 0.028s
I20260709 13:15:18.725488  4824 tablet_bootstrap.cc:492] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:18.726140  4824 ts_tablet_manager.cc:1403] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.199s	user 0.043s	sys 0.031s
I20260709 13:15:18.702598  4889 raft_consensus.cc:493] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.742385  4519 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.742744  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:18.745697  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:18.750317  4623 raft_consensus.cc:3060] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.739148  4517 leader_election.cc:304] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:18.753511  5041 raft_consensus.cc:2804] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:18.753983  5041 raft_consensus.cc:493] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.754338  5041 raft_consensus.cc:3060] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.757026  4824 raft_consensus.cc:359] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.757884  4824 raft_consensus.cc:385] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.758164  4824 raft_consensus.cc:740] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.759824  4824 consensus_queue.cc:260] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.775055  4824 ts_tablet_manager.cc:1434] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.002s
I20260709 13:15:18.763079  5041 raft_consensus.cc:515] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.764905  4547 raft_consensus.cc:3060] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.778810  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:18.779371  5041 leader_election.cc:290] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:18.779656  4551 raft_consensus.cc:3060] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.780658  4552 raft_consensus.cc:2393] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:18.781970  4824 tablet_bootstrap.cc:492] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:18.767823  5042 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.780651  4623 raft_consensus.cc:2468] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:18.790905  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:18.791437  4821 tablet_bootstrap.cc:654] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.791834  4776 raft_consensus.cc:2393] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:18.792382  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:18.737465  4889 raft_consensus.cc:515] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:18.776058  4988 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:18.797655  4822 raft_consensus.cc:359] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.801131  4822 raft_consensus.cc:385] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.801620  4822 raft_consensus.cc:740] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.802666  4822 consensus_queue.cc:260] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.792685  5041 consensus_queue.cc:1048] T 290a017cf81042b6a4e64319563864c5 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:15:18.823096  4823 tablet_bootstrap.cc:492] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:18.823930  4823 ts_tablet_manager.cc:1403] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.262s	user 0.017s	sys 0.005s
W20260709 13:15:18.823114  5048 log.cc:927] Time spent T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:15:18.815196  4973 consensus_queue.cc:1048] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:15:18.830240  4823 raft_consensus.cc:359] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.830974  4822 ts_tablet_manager.cc:1434] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.107s	user 0.008s	sys 0.002s
I20260709 13:15:18.831157  4823 raft_consensus.cc:385] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.831593  4823 raft_consensus.cc:740] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.833411  4823 consensus_queue.cc:260] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
W20260709 13:15:18.804467  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 77344.7us
I20260709 13:15:18.828092  4627 raft_consensus.cc:2393] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:18.840714  4823 ts_tablet_manager.cc:1434] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.005s
I20260709 13:15:18.841146  4517 leader_election.cc:304] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.842140  5041 raft_consensus.cc:2749] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:18.843166  4823 tablet_bootstrap.cc:492] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:18.843942  4889 leader_election.cc:290] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:18.859488  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:18.860348  4772 raft_consensus.cc:3060] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.860946  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:18.861726  4695 raft_consensus.cc:2468] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:18.863776  4741 leader_election.cc:304] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:18.853389  4869 raft_consensus.cc:493] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.865967  4869 raft_consensus.cc:515] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.870021  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:18.870895  4553 raft_consensus.cc:2468] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:18.871583  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:18.870890  4822 tablet_bootstrap.cc:492] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
W20260709 13:15:18.874935  4666 leader_election.cc:343] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:18.875432  4823 tablet_bootstrap.cc:654] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.875808  4666 leader_election.cc:304] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.868445  4869 leader_election.cc:290] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:18.882786  5040 raft_consensus.cc:493] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:18.884123  5040 raft_consensus.cc:515] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.894004  5042 raft_consensus.cc:2749] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:18.879458  4959 raft_consensus.cc:2804] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:18.896720  4959 raft_consensus.cc:493] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.897312  4959 raft_consensus.cc:3060] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.897820  4821 tablet_bootstrap.cc:492] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:18.898375  4821 ts_tablet_manager.cc:1403] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.245s	user 0.019s	sys 0.017s
I20260709 13:15:18.901306  5054 consensus_queue.cc:1048] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:18.902562  4740 leader_election.cc:304] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:18.904366  4889 raft_consensus.cc:2804] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:18.906657  4822 tablet_bootstrap.cc:654] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.908001  4973 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.880674  5051 raft_consensus.cc:2804] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:18.915057  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:18.919894  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:18.921957  4889 raft_consensus.cc:697] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:18.905917  4551 raft_consensus.cc:2468] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:18.923157  4889 consensus_queue.cc:237] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:18.910352  5051 raft_consensus.cc:493] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.908751  4973 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.932305  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:18.933085  4701 raft_consensus.cc:2468] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:18.934698  4593 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:18.936380  4964 raft_consensus.cc:2804] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:18.936810  4964 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.937160  4964 raft_consensus.cc:3060] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.937345  5040 leader_election.cc:290] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:18.942320  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:18.945750  4517 leader_election.cc:304] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:18.948433  5057 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.949025  5057 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.950832  4951 consensus_queue.cc:1048] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:18.952522  5057 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:18.953627  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:18.954770  4701 raft_consensus.cc:2468] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:18.956082  4741 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:18.956552  5042 raft_consensus.cc:2749] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:18.957342  5057 raft_consensus.cc:2804] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:18.927583  5051 raft_consensus.cc:3060] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.959504  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:18.960808  4740 leader_election.cc:343] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:18.957813  5057 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.936391  4824 tablet_bootstrap.cc:654] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:18.973227  4821 raft_consensus.cc:359] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.978626  4821 raft_consensus.cc:385] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:18.979158  4821 raft_consensus.cc:740] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:18.966991  4973 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:18.968152  5057 raft_consensus.cc:3060] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:18.984103  4959 raft_consensus.cc:515] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:18.987353  4821 consensus_queue.cc:260] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:18.979329  4772 raft_consensus.cc:2468] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:18.992327  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:19.010183  4959 leader_election.cc:290] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:19.010461  4823 tablet_bootstrap.cc:492] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:19.016237  4823 ts_tablet_manager.cc:1403] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.174s	user 0.010s	sys 0.007s
I20260709 13:15:19.030143  4821 ts_tablet_manager.cc:1434] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.131s	user 0.010s	sys 0.002s
I20260709 13:15:19.024672  4668 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:19.060901  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:19.061755  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:19.064363  4898 raft_consensus.cc:2804] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:19.061769  4553 raft_consensus.cc:3060] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.067363  4898 raft_consensus.cc:697] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:19.068439  4898 consensus_queue.cc:237] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.072257  4822 tablet_bootstrap.cc:492] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:19.072870  4822 ts_tablet_manager.cc:1403] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.202s	user 0.013s	sys 0.013s
I20260709 13:15:19.080837  4821 tablet_bootstrap.cc:492] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:19.097465  4823 raft_consensus.cc:359] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.098484  4823 raft_consensus.cc:385] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.098820  4823 raft_consensus.cc:740] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.099735  4823 consensus_queue.cc:260] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.110562  4822 raft_consensus.cc:359] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.121735  4824 tablet_bootstrap.cc:492] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:19.122466  4824 ts_tablet_manager.cc:1403] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.341s	user 0.021s	sys 0.008s
I20260709 13:15:19.123865  4822 raft_consensus.cc:385] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.124259  4822 raft_consensus.cc:740] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.129310  4823 ts_tablet_manager.cc:1434] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.113s	user 0.013s	sys 0.002s
I20260709 13:15:19.131000  4823 tablet_bootstrap.cc:492] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:19.132273  4822 consensus_queue.cc:260] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.136250  4553 raft_consensus.cc:2468] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:19.123207  4964 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.137398  4824 raft_consensus.cc:359] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.138080  4623 raft_consensus.cc:2393] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 1: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:19.138118  4824 raft_consensus.cc:385] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:15:19.128202  4666 leader_election.cc:343] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:19.138672  4824 raft_consensus.cc:740] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.140146  4824 consensus_queue.cc:260] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.140561  5059 consensus_queue.cc:1048] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:19.142586  4666 leader_election.cc:304] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:19.143941  5065 consensus_queue.cc:1048] T 5e52269d0b2241c6b96a578f3ab61d72 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:19.144169  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:19.146140  4693 raft_consensus.cc:3060] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.142738  4824 ts_tablet_manager.cc:1434] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:15:19.126081  5051 raft_consensus.cc:515] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.145521  5065 raft_consensus.cc:2804] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:19.156127  4824 tablet_bootstrap.cc:492] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:19.176713  4822 ts_tablet_manager.cc:1434] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.104s	user 0.008s	sys 0.001s
I20260709 13:15:19.173301  4973 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.182852  5065 raft_consensus.cc:697] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:19.145632  5057 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.186863  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:19.187819  4627 raft_consensus.cc:2393] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 1: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:19.187728  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:19.189085  4769 raft_consensus.cc:2393] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:19.189936  4740 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:19.191205  4887 raft_consensus.cc:2749] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:19.189881  5057 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:19.192274  4821 tablet_bootstrap.cc:654] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.183524  4973 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:19.149298  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 53202.4us
I20260709 13:15:19.194754  5071 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.195194  5071 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.197214  5071 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:19.201874  4822 tablet_bootstrap.cc:492] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:19.201624  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:19.202996  4776 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:19.204723  4594 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:19.208423  4823 tablet_bootstrap.cc:654] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.209055  4889 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.167543  4693 raft_consensus.cc:2468] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:19.209653  4889 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.210917  4593 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:19.152451  5051 leader_election.cc:290] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:19.155508  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:19.156149  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:19.215229  4898 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:19.215725  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:19.215883  4898 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.216480  4547 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:19.185510  5065 consensus_queue.cc:237] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.233573  4822 tablet_bootstrap.cc:654] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.242844  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:19.251091  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:19.242865  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:19.246618  4898 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:19.247601  4821 tablet_bootstrap.cc:492] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:19.249820  4889 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:19.251055  4824 tablet_bootstrap.cc:654] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.253746  4628 raft_consensus.cc:2468] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:19.257507  4627 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:19.225926  4552 raft_consensus.cc:3060] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.254889  4821 ts_tablet_manager.cc:1403] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.190s	user 0.013s	sys 0.008s
I20260709 13:15:19.227452  4964 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:19.253866  4693 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:19.221984  4701 raft_consensus.cc:3060] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.262264  4740 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:19.255648  4951 raft_consensus.cc:2804] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:19.266465  5076 raft_consensus.cc:2804] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:19.266845  4666 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:19.267108  4973 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:19.265372  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:19.268314  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:19.273458  4451 catalog_manager.cc:5697] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:19.280642  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 76025.7us
I20260709 13:15:19.284173  5076 raft_consensus.cc:697] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:19.285590  4553 raft_consensus.cc:2468] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:19.286299  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:19.286814  4898 raft_consensus.cc:2804] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:19.287230  4769 raft_consensus.cc:2393] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:19.280228  4951 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.287774  4951 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.287824  4592 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:19.287274  4898 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:19.285369  5076 consensus_queue.cc:237] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.289288  4898 raft_consensus.cc:3060] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:19.288407  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:19.281080  4889 raft_consensus.cc:2804] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:19.297547  4776 raft_consensus.cc:2393] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 2: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:19.304754  5077 raft_consensus.cc:2804] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:19.305361  5077 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.305768  5077 raft_consensus.cc:3060] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.317415  4821 raft_consensus.cc:359] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.321061  4821 raft_consensus.cc:385] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.321453  4821 raft_consensus.cc:740] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.325703  4824 tablet_bootstrap.cc:492] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:19.326282  4824 ts_tablet_manager.cc:1403] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.170s	user 0.016s	sys 0.007s
I20260709 13:15:19.326710  4552 raft_consensus.cc:2468] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:19.328109  4740 leader_election.cc:304] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:19.329008  5051 raft_consensus.cc:2804] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:19.329487  5051 raft_consensus.cc:697] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:19.329276  4824 raft_consensus.cc:359] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.329984  4824 raft_consensus.cc:385] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.330288  4824 raft_consensus.cc:740] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.330394  5051 consensus_queue.cc:237] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.322935  4821 consensus_queue.cc:260] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.330996  4824 consensus_queue.cc:260] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.333226  4821 ts_tablet_manager.cc:1434] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.073s	user 0.002s	sys 0.005s
I20260709 13:15:19.333611  4824 ts_tablet_manager.cc:1434] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:19.333511  5054 raft_consensus.cc:493] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.334389  4821 tablet_bootstrap.cc:492] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:19.334156  5054 raft_consensus.cc:515] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.334759  4824 tablet_bootstrap.cc:492] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:19.297238  4889 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.335193  4889 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.336367  5054 leader_election.cc:290] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:19.347034  4824 tablet_bootstrap.cc:654] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.347450  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:19.348310  4627 raft_consensus.cc:2468] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:19.350610  4740 leader_election.cc:304] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:19.351816  5054 raft_consensus.cc:2804] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:19.352228  5054 raft_consensus.cc:493] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.352583  5054 raft_consensus.cc:3060] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.356355  4821 tablet_bootstrap.cc:654] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.362625  4898 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.369318  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:19.372871  4822 tablet_bootstrap.cc:492] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:19.376643  4822 ts_tablet_manager.cc:1403] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.175s	user 0.019s	sys 0.001s
I20260709 13:15:19.381170  4822 raft_consensus.cc:359] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.382427  4822 raft_consensus.cc:385] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.382928  4822 raft_consensus.cc:740] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
W20260709 13:15:19.387253  4740 leader_election.cc:343] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:19.392009  4823 tablet_bootstrap.cc:492] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:19.392622  4823 ts_tablet_manager.cc:1403] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.262s	user 0.043s	sys 0.048s
I20260709 13:15:19.402429  4821 tablet_bootstrap.cc:492] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:19.403048  4821 ts_tablet_manager.cc:1403] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.003s
I20260709 13:15:19.403211  4823 raft_consensus.cc:359] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.403996  4823 raft_consensus.cc:385] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.404349  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:19.404575  4898 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:19.409148  4824 tablet_bootstrap.cc:492] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:19.409731  4824 ts_tablet_manager.cc:1403] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.010s
I20260709 13:15:19.411059  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:19.411976  4776 raft_consensus.cc:2393] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 2: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:19.412901  4823 raft_consensus.cc:740] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.413722  4823 consensus_queue.cc:260] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.415850  4823 ts_tablet_manager.cc:1434] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.000s
I20260709 13:15:19.416486  4887 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.417006  4823 tablet_bootstrap.cc:492] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:19.417110  4887 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.418280  4824 raft_consensus.cc:359] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.419041  4824 raft_consensus.cc:385] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.419354  4824 raft_consensus.cc:740] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.419963  4887 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:19.421247  5077 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.420260  4824 consensus_queue.cc:260] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.423702  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:19.423980  4824 ts_tablet_manager.cc:1434] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:15:19.421857  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:19.424633  4553 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:19.424847  4693 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:19.425271  4951 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.426730  4824 tablet_bootstrap.cc:492] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:19.426929  4741 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:19.427134  4627 raft_consensus.cc:2468] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:19.429036  5057 raft_consensus.cc:2804] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:19.428813  4666 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:19.430249  5057 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.430723  5057 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.430492  5051 consensus_queue.cc:1048] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:19.435637  4889 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.437513  4823 tablet_bootstrap.cc:654] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.443694  5057 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.375506  4701 raft_consensus.cc:2468] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:19.384097  4822 consensus_queue.cc:260] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.439530  5065 raft_consensus.cc:2804] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:19.446436  5065 raft_consensus.cc:697] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:19.438421  4824 tablet_bootstrap.cc:654] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.447573  5065 consensus_queue.cc:237] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.459045  4822 ts_tablet_manager.cc:1434] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.082s	user 0.010s	sys 0.000s
I20260709 13:15:19.442894  4821 raft_consensus.cc:359] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.460413  4821 raft_consensus.cc:385] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.463531  5077 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:19.463989  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:19.464061  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:19.464912  4693 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.465255  4822 tablet_bootstrap.cc:492] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:19.465078  4776 raft_consensus.cc:3060] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.469614  4951 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:19.470077  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:19.470408  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:19.470786  4772 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.471145  4547 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.461373  4821 raft_consensus.cc:740] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.445379  5054 raft_consensus.cc:515] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.480633  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:19.482026  4822 tablet_bootstrap.cc:654] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.482120  4898 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:19.484925  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:19.490340  4552 raft_consensus.cc:3060] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.489125  4627 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.489307  4898 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.488047  4821 consensus_queue.cc:260] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.505291  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:19.506093  4701 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.507995  5054 leader_election.cc:290] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:19.509660  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:19.510408  4628 raft_consensus.cc:3060] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.511402  4772 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:19.511848  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:19.512575  4551 raft_consensus.cc:3060] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.516608  5057 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:19.497617  5065 raft_consensus.cc:493] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.517987  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:19.518163  5065 raft_consensus.cc:515] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.518838  4549 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:19.525998  4594 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:19.527011  5075 raft_consensus.cc:2804] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:19.527536  5075 raft_consensus.cc:697] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:19.528417  5075 consensus_queue.cc:237] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.530933  4693 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:19.532270  4741 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:19.532496  4547 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
W20260709 13:15:19.508647  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:19.533506  5051 raft_consensus.cc:2804] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:19.534032  5051 raft_consensus.cc:697] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:19.534852  5051 consensus_queue.cc:237] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.504473  4776 raft_consensus.cc:2468] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:19.547178  4821 ts_tablet_manager.cc:1434] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.144s	user 0.004s	sys 0.007s
I20260709 13:15:19.548833  4594 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:19.550089  5076 raft_consensus.cc:2804] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:19.550622  5076 raft_consensus.cc:697] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:19.552472  5076 consensus_queue.cc:237] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.554680  4988 consensus_queue.cc:1048] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:15:19.537894  5057 consensus_queue.cc:1048] T 3b05eadf0e794d249a3ee1e016745baf P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:15:19.563570  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:19.565433  4666 leader_election.cc:343] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:19.566325  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:19.567018  4769 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:19.568233  4668 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:19.569274  4898 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:19.569782  4898 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:19.570418  4821 tablet_bootstrap.cc:492] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:19.576066  4824 tablet_bootstrap.cc:492] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:19.576503  4824 ts_tablet_manager.cc:1403] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.150s	user 0.015s	sys 0.010s
I20260709 13:15:19.579376  4823 tablet_bootstrap.cc:492] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:19.580294  4823 ts_tablet_manager.cc:1403] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.164s	user 0.010s	sys 0.005s
I20260709 13:15:19.578300  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:19.582818  4547 raft_consensus.cc:2393] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:19.583863  5065 leader_election.cc:290] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:19.575284  4898 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:19.587946  4898 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:19.589479  4823 raft_consensus.cc:359] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.590361  4823 raft_consensus.cc:385] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.590471  4889 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:19.590693  4823 raft_consensus.cc:740] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.577419  4701 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:19.594444  4823 consensus_queue.cc:260] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.595260  4824 raft_consensus.cc:359] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.595983  4824 raft_consensus.cc:385] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.596271  4824 raft_consensus.cc:740] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.598235  4823 ts_tablet_manager.cc:1434] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:15:19.599522  4552 raft_consensus.cc:2468] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:19.612257  4823 tablet_bootstrap.cc:492] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:19.575501  4628 raft_consensus.cc:2468] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:19.598975  4447 catalog_manager.cc:5697] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 0 to 1, leader changed from <none> to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 1 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.602258  4549 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:19.601516  5060 consensus_queue.cc:1048] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:19.600555  4741 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:19.588104  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:19.581784  4627 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:19.622396  4624 raft_consensus.cc:3060] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.612685  4824 consensus_queue.cc:260] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:19.632025  4666 leader_election.cc:343] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:19.633023  4666 leader_election.cc:304] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:19.623234  4551 raft_consensus.cc:2468] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:19.636116  5065 raft_consensus.cc:2749] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:19.648561  4447 catalog_manager.cc:5697] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 0 to 1, leader changed from <none> to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 1 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.652406  4740 leader_election.cc:304] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:19.668941  5051 raft_consensus.cc:2804] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:19.630396  5057 raft_consensus.cc:2804] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:19.678599  4822 tablet_bootstrap.cc:492] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:19.679813  4822 ts_tablet_manager.cc:1403] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.215s	user 0.018s	sys 0.001s
W20260709 13:15:19.680387  5088 log.cc:927] Time spent T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:15:19.669504  5051 raft_consensus.cc:697] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:19.683962  4822 raft_consensus.cc:359] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.685065  5057 raft_consensus.cc:697] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:19.686888  5057 consensus_queue.cc:237] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.687659  4824 ts_tablet_manager.cc:1434] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.111s	user 0.007s	sys 0.001s
I20260709 13:15:19.691628  4823 tablet_bootstrap.cc:654] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.696605  4822 raft_consensus.cc:385] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.697635  4822 raft_consensus.cc:740] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.698655  4821 tablet_bootstrap.cc:654] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.685134  5051 consensus_queue.cc:237] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.699240  4822 consensus_queue.cc:260] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.701846  4824 tablet_bootstrap.cc:492] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:19.701937  4988 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.709537  4988 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.710614  4957 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.711153  4957 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.713299  4957 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:19.716964  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:19.712184  4988 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:19.716533  4869 consensus_queue.cc:1048] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:15:19.719702  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:19.715802  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:19.721585  4769 raft_consensus.cc:2410] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:15:19.718434  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:19.727056  4666 leader_election.cc:343] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:19.728941  4668 leader_election.cc:343] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:19.729482  4668 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:19.732373  5065 raft_consensus.cc:2749] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:19.752085  5075 consensus_queue.cc:1048] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:19.757275  4824 tablet_bootstrap.cc:654] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.748600  4449 catalog_manager.cc:5697] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.772426  4822 ts_tablet_manager.cc:1434] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.092s	user 0.007s	sys 0.003s
I20260709 13:15:19.779078  4988 raft_consensus.cc:493] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.779830  4988 raft_consensus.cc:515] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.779651  4449 catalog_manager.cc:5697] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.784035  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:19.784999  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:19.785748  4822 tablet_bootstrap.cc:492] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:19.786720  4772 raft_consensus.cc:2468] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:19.789304  4988 leader_election.cc:290] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:19.793977  4823 tablet_bootstrap.cc:492] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:19.795039  4823 ts_tablet_manager.cc:1403] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.194s	user 0.018s	sys 0.028s
I20260709 13:15:19.785902  4624 raft_consensus.cc:2468] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:19.796821  4824 tablet_bootstrap.cc:492] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:19.797957  4824 ts_tablet_manager.cc:1403] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.101s	user 0.007s	sys 0.006s
I20260709 13:15:19.790045  4449 catalog_manager.cc:5697] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.795295  4668 leader_election.cc:304] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:19.809659  5065 raft_consensus.cc:2804] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:19.810112  5065 raft_consensus.cc:493] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.812381  4898 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.814060  4701 raft_consensus.cc:2410] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:15:19.815068  4898 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:19.810523  5065 raft_consensus.cc:3060] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.831487  5075 consensus_queue.cc:1048] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:19.835441  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:19.836364  4551 raft_consensus.cc:2393] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:19.836675  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:19.841161  4824 raft_consensus.cc:359] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.842288  4824 raft_consensus.cc:385] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.842869  4824 raft_consensus.cc:740] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.844491  4824 consensus_queue.cc:260] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.850145  4821 tablet_bootstrap.cc:492] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:19.843422  4518 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:19.850584  4821 ts_tablet_manager.cc:1403] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.295s	user 0.015s	sys 0.031s
I20260709 13:15:19.852638  4824 ts_tablet_manager.cc:1434] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 13:15:19.852453  4823 raft_consensus.cc:359] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.855126  4823 raft_consensus.cc:385] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.855484  4823 raft_consensus.cc:740] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.854871  4988 consensus_queue.cc:1048] T 556306e5b2024ff493e111028a819954 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:15:19.857561  4823 consensus_queue.cc:260] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.860002  4824 tablet_bootstrap.cc:492] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:19.854923  4957 raft_consensus.cc:2749] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:19.867065  4822 tablet_bootstrap.cc:654] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.853752  4448 catalog_manager.cc:5697] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.879748  4448 catalog_manager.cc:5697] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.882954  4821 raft_consensus.cc:359] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.887050  4821 raft_consensus.cc:385] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.887408  4821 raft_consensus.cc:740] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.892338  4823 ts_tablet_manager.cc:1434] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.097s	user 0.008s	sys 0.009s
I20260709 13:15:19.886650  4448 catalog_manager.cc:5697] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.893898  4821 consensus_queue.cc:260] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:19.904310  4822 tablet_bootstrap.cc:492] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:19.905068  4821 ts_tablet_manager.cc:1434] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.003s
I20260709 13:15:19.905135  4822 ts_tablet_manager.cc:1403] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.120s	user 0.018s	sys 0.002s
I20260709 13:15:19.906458  4821 tablet_bootstrap.cc:492] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:19.894800  4823 tablet_bootstrap.cc:492] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:19.884627  5065 raft_consensus.cc:515] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:19.886498  4772 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:19.902359  4824 tablet_bootstrap.cc:654] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.922783  4668 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:19.924052  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:19.924796  4625 raft_consensus.cc:3060] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.909505  4822 raft_consensus.cc:359] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.925173  4898 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:19.925575  4822 raft_consensus.cc:385] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:19.925930  4898 raft_consensus.cc:697] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:19.925290  4821 tablet_bootstrap.cc:654] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:19.927199  4898 consensus_queue.cc:237] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:19.925977  4822 raft_consensus.cc:740] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:19.942636  4449 catalog_manager.cc:5697] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 1 to 2, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 2 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.915493  4957 raft_consensus.cc:493] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.941710  4822 consensus_queue.cc:260] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.947299  4957 raft_consensus.cc:515] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.952677  4769 raft_consensus.cc:1275] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:19.958982  4887 consensus_queue.cc:1048] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:19.959115  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:19.959504  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:19.960270  5057 consensus_queue.cc:1048] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:19.960457  4695 raft_consensus.cc:2468] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:19.962359  4518 leader_election.cc:304] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:19.965561  5108 raft_consensus.cc:2804] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:19.965965  5108 raft_consensus.cc:493] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.966291  5108 raft_consensus.cc:3060] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.971822  4957 leader_election.cc:290] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:19.974277  5075 consensus_queue.cc:1048] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:19.977453  4624 raft_consensus.cc:2468] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:19.978922  4822 ts_tablet_manager.cc:1434] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.073s	user 0.005s	sys 0.004s
I20260709 13:15:19.980249  4822 tablet_bootstrap.cc:492] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:19.989354  5065 leader_election.cc:290] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:19.993683  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:19.994366  4772 raft_consensus.cc:3060] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:19.994766  5108 raft_consensus.cc:515] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:19.998338  4625 raft_consensus.cc:2468] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:19.999574  4666 leader_election.cc:304] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:20.002470  5065 raft_consensus.cc:2804] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.004530  5065 raft_consensus.cc:697] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:20.009867  5065 consensus_queue.cc:237] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.011555  4772 raft_consensus.cc:2468] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:20.014513  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:20.017331  4695 raft_consensus.cc:3060] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.024646  4450 catalog_manager.cc:5697] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 1 to 2, leader changed from 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196) to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 2 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.030226  4553 raft_consensus.cc:1275] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:20.032404  4822 tablet_bootstrap.cc:654] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.032838  5108 leader_election.cc:290] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:20.033181  5077 consensus_queue.cc:1048] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:20.041503  4769 raft_consensus.cc:1275] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:20.016183  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:20.050019  4625 raft_consensus.cc:3060] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.051908  4988 consensus_queue.cc:1048] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:20.052340  4769 raft_consensus.cc:1275] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:20.053879  5075 consensus_queue.cc:1048] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:20.057561  4624 raft_consensus.cc:1275] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:20.059799  4988 consensus_queue.cc:1048] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:20.060982  4823 tablet_bootstrap.cc:654] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.063683  4695 raft_consensus.cc:2468] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:20.071794  4518 leader_election.cc:304] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:20.072928  5108 raft_consensus.cc:2804] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.073518  5108 raft_consensus.cc:697] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:20.074832  4824 tablet_bootstrap.cc:492] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:20.074455  5108 consensus_queue.cc:237] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:20.075410  4824 ts_tablet_manager.cc:1403] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.216s	user 0.015s	sys 0.010s
I20260709 13:15:20.079478  4625 raft_consensus.cc:2468] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:20.080818  4957 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.081398  4957 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.083102  4821 tablet_bootstrap.cc:492] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:20.083786  4821 ts_tablet_manager.cc:1403] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.178s	user 0.020s	sys 0.008s
I20260709 13:15:20.086608  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:20.083801  4957 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:20.081398  4551 raft_consensus.cc:1275] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:20.090942  5075 consensus_queue.cc:1048] T e2c78af3c856414e866ff06aa637ada1 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:20.091341  4551 raft_consensus.cc:1275] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:20.092381  4824 raft_consensus.cc:359] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:20.092808  4517 leader_election.cc:343] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.093231  4824 raft_consensus.cc:385] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.093663  4824 raft_consensus.cc:740] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.103240  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:20.104087  4769 raft_consensus.cc:2468] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:20.107481  4821 raft_consensus.cc:359] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.109782  4821 raft_consensus.cc:385] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.111665  4821 raft_consensus.cc:740] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.113067  4821 consensus_queue.cc:260] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.115324  4821 ts_tablet_manager.cc:1434] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.031s	user 0.001s	sys 0.007s
I20260709 13:15:20.101397  4449 catalog_manager.cc:5697] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.110831  5051 consensus_queue.cc:1048] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.117206  4824 consensus_queue.cc:260] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.122397  4869 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.131079  4869 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:20.133249  4869 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:20.134410  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:20.135673  4823 tablet_bootstrap.cc:492] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:20.136260  4823 ts_tablet_manager.cc:1403] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.242s	user 0.010s	sys 0.014s
W20260709 13:15:20.136534  4666 leader_election.cc:343] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.137516  4821 tablet_bootstrap.cc:492] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:20.139189  4824 ts_tablet_manager.cc:1434] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.002s
I20260709 13:15:20.138726  4695 raft_consensus.cc:1275] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:20.140992  4519 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:20.145933  4823 raft_consensus.cc:359] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.146644  4823 raft_consensus.cc:385] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.146559  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:20.146914  4822 tablet_bootstrap.cc:492] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:20.147651  4824 tablet_bootstrap.cc:492] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:20.148151  5107 raft_consensus.cc:2804] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.148551  5107 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.149178  5107 raft_consensus.cc:3060] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.147694  4822 ts_tablet_manager.cc:1403] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.168s	user 0.012s	sys 0.012s
W20260709 13:15:20.153759  4666 leader_election.cc:343] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.154626  4824 tablet_bootstrap.cc:654] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.154340  4666 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:20.155388  5065 raft_consensus.cc:2749] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:20.158527  4821 tablet_bootstrap.cc:654] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.160193  4823 raft_consensus.cc:740] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.154088  4822 raft_consensus.cc:359] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.161803  4887 consensus_queue.cc:1048] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:20.163254  4887 consensus_queue.cc:1048] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.160977  4822 raft_consensus.cc:385] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.170501  4822 raft_consensus.cc:740] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.170848  5107 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.174733  4822 consensus_queue.cc:260] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.183732  4822 ts_tablet_manager.cc:1434] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:15:20.177294  4823 consensus_queue.cc:260] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.188938  4450 catalog_manager.cc:5697] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 0 to 1, leader changed from <none> to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 1 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.221982  4447 catalog_manager.cc:5697] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.233964  4822 tablet_bootstrap.cc:492] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:20.235711  4823 ts_tablet_manager.cc:1434] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.099s	user 0.011s	sys 0.000s
I20260709 13:15:20.237380  4823 tablet_bootstrap.cc:492] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:20.240922  4887 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.241680  4887 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.244186  4887 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:20.245512  5107 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:20.247010  4821 tablet_bootstrap.cc:492] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:20.247870  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:20.250870  4824 tablet_bootstrap.cc:492] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:20.252192  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:20.253723  4821 ts_tablet_manager.cc:1403] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.129s	user 0.012s	sys 0.003s
I20260709 13:15:20.254447  4824 ts_tablet_manager.cc:1403] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.107s	user 0.005s	sys 0.012s
I20260709 13:15:20.255975  4769 raft_consensus.cc:3060] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.262209  4693 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:20.261879  4821 raft_consensus.cc:359] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.266130  4821 raft_consensus.cc:385] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.266566  4821 raft_consensus.cc:740] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.254063  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:20.267599  4553 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:20.267463  4821 consensus_queue.cc:260] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.268338  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:20.269901  5051 consensus_queue.cc:1048] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:20.270231  4741 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:20.281335  4821 ts_tablet_manager.cc:1434] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 13:15:20.282620  4821 tablet_bootstrap.cc:492] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
W20260709 13:15:20.280834  4517 leader_election.cc:343] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.285029  5057 raft_consensus.cc:2804] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.285542  5057 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.285926  5057 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.289153  4821 tablet_bootstrap.cc:654] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.290894  5076 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.293462  5076 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.297370  5076 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:20.299823  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:20.283636  4769 raft_consensus.cc:2468] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:20.300908  4772 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:20.303156  4594 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:20.308106  4519 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:20.309105  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:20.309684  5107 raft_consensus.cc:2804] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.310827  5107 raft_consensus.cc:697] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:20.311769  5107 consensus_queue.cc:237] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.317750  5076 raft_consensus.cc:2804] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.318312  5076 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.318697  5076 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.333153  5057 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.336678  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:20.337589  4693 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.341910  4822 tablet_bootstrap.cc:654] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.343310  5076 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.356050  4821 tablet_bootstrap.cc:492] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:20.356668  4821 ts_tablet_manager.cc:1403] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.074s	user 0.022s	sys 0.038s
I20260709 13:15:20.361054  4821 raft_consensus.cc:359] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.361979  4821 raft_consensus.cc:385] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.362313  4821 raft_consensus.cc:740] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.363088  4821 consensus_queue.cc:260] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.371378  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:20.372160  4772 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.375031  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:20.378870  5076 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:20.381909  4772 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:20.383423  4594 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:20.384466  5076 raft_consensus.cc:2804] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.384651  4693 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:20.384840  5057 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:20.385468  4821 ts_tablet_manager.cc:1434] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 13:15:20.388259  4823 tablet_bootstrap.cc:654] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.389294  4821 tablet_bootstrap.cc:492] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:20.387508  4450 catalog_manager.cc:5697] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.391247  4822 tablet_bootstrap.cc:492] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:20.391794  4822 ts_tablet_manager.cc:1403] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.182s	user 0.023s	sys 0.016s
W20260709 13:15:20.392022  4593 leader_election.cc:343] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.397469  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:20.398449  4553 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.400795  5060 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.401530  5060 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.409374  5054 consensus_queue.cc:1048] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.405011  5076 raft_consensus.cc:697] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:20.415247  4741 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:20.416343  4821 tablet_bootstrap.cc:654] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.417042  5057 raft_consensus.cc:2804] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.418179  5057 raft_consensus.cc:697] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:20.420671  5057 consensus_queue.cc:237] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.423416  4552 raft_consensus.cc:1275] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:20.424141  4553 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:20.425979  4869 consensus_queue.cc:1048] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:20.426347  5060 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:20.428633  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:20.422433  4824 raft_consensus.cc:359] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.429924  4824 raft_consensus.cc:385] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.430260  4824 raft_consensus.cc:740] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
W20260709 13:15:20.430373  4668 leader_election.cc:343] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.431118  4824 consensus_queue.cc:260] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.414247  5076 consensus_queue.cc:237] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.436266  5054 consensus_queue.cc:1048] T ae70b5bde8ec47b1931014ebfe03ea2a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.439782  4822 raft_consensus.cc:359] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.440970  4822 raft_consensus.cc:385] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.441354  4822 raft_consensus.cc:740] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.448561  4822 consensus_queue.cc:260] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.454067  4824 ts_tablet_manager.cc:1434] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.199s	user 0.006s	sys 0.004s
I20260709 13:15:20.454073  4822 ts_tablet_manager.cc:1434] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.062s	user 0.011s	sys 0.000s
W20260709 13:15:20.467195  4593 leader_election.cc:343] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.467338  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:20.468804  4666 leader_election.cc:343] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.469380  4666 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:20.470708  4869 raft_consensus.cc:2749] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:20.471890  4824 tablet_bootstrap.cc:492] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:20.472038  4821 tablet_bootstrap.cc:492] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:20.480772  4821 ts_tablet_manager.cc:1403] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.092s	user 0.023s	sys 0.011s
I20260709 13:15:20.484352  4822 tablet_bootstrap.cc:492] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:20.488873  4824 tablet_bootstrap.cc:654] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.508107  4822 tablet_bootstrap.cc:654] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.496296  4448 catalog_manager.cc:5697] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.510772  4772 raft_consensus.cc:1275] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:20.512662  4869 consensus_queue.cc:1048] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:15:20.525427  4823 tablet_bootstrap.cc:492] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:20.526358  4823 ts_tablet_manager.cc:1403] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.289s	user 0.023s	sys 0.022s
I20260709 13:15:20.549270  4821 raft_consensus.cc:359] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.550199  4821 raft_consensus.cc:385] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.550508  4821 raft_consensus.cc:740] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.551285  4821 consensus_queue.cc:260] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.557233  4869 consensus_queue.cc:1048] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:20.563532  4821 ts_tablet_manager.cc:1434] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.000s
I20260709 13:15:20.568843  4823 raft_consensus.cc:359] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.569643  4823 raft_consensus.cc:385] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.569952  4823 raft_consensus.cc:740] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.570604  4821 tablet_bootstrap.cc:492] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:20.572891  4823 consensus_queue.cc:260] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.575284  4988 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.581341  5054 raft_consensus.cc:493] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.575891  4988 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.581998  5054 raft_consensus.cc:515] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.584183  4988 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:20.586174  4823 ts_tablet_manager.cc:1434] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 13:15:20.587522  4823 tablet_bootstrap.cc:492] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:20.592628  4988 consensus_queue.cc:1048] T cd411637f5c84d50b6c9ba5cae086a07 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:20.593789  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:20.596663  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:20.597469  4776 raft_consensus.cc:2393] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:20.604967  5054 leader_election.cc:290] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:20.612757  4668 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:20.614074  4824 tablet_bootstrap.cc:492] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:20.614784  4869 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.617636  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:20.617925  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:20.618435  4552 raft_consensus.cc:2468] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
W20260709 13:15:20.619673  4740 leader_election.cc:343] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.621021  4823 tablet_bootstrap.cc:654] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.621659  4740 leader_election.cc:304] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:20.622984  5054 raft_consensus.cc:2804] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.623382  5054 raft_consensus.cc:493] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.623713  5054 raft_consensus.cc:3060] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.614638  4824 ts_tablet_manager.cc:1403] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.143s	user 0.021s	sys 0.023s
I20260709 13:15:20.637485  5107 consensus_queue.cc:1048] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:20.638486  4824 raft_consensus.cc:359] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.639261  4824 raft_consensus.cc:385] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.639545  4824 raft_consensus.cc:740] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.643111  4821 tablet_bootstrap.cc:654] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.647687  4869 raft_consensus.cc:2749] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:20.647258  4448 catalog_manager.cc:5697] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.669646  4824 consensus_queue.cc:260] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.699278  4824 ts_tablet_manager.cc:1434] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.000s
I20260709 13:15:20.702683  5057 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.703367  5057 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.704003  5075 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.704563  5075 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.713587  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:20.716838  5145 log.cc:927] Time spent T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:15:20.716279  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:20.718315  4822 tablet_bootstrap.cc:492] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:20.718820  4769 raft_consensus.cc:2393] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:20.718988  4822 ts_tablet_manager.cc:1403] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.238s	user 0.018s	sys 0.002s
I20260709 13:15:20.706981  5075 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:20.719944  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:20.720746  4695 raft_consensus.cc:2468] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:20.737457  4822 raft_consensus.cc:359] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:20.741540  4822 raft_consensus.cc:385] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.741798  4824 tablet_bootstrap.cc:492] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:20.742004  4822 raft_consensus.cc:740] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.745553  4822 consensus_queue.cc:260] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:20.742903  5057 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:20.735746  5107 consensus_queue.cc:1048] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:15:20.743891  4741 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:20.755461  4592 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:20.759922  4821 tablet_bootstrap.cc:492] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:20.760671  4821 ts_tablet_manager.cc:1403] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.191s	user 0.016s	sys 0.014s
I20260709 13:15:20.760011  4823 tablet_bootstrap.cc:492] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:20.757961  5054 raft_consensus.cc:515] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.762171  4823 ts_tablet_manager.cc:1403] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.175s	user 0.041s	sys 0.044s
I20260709 13:15:20.763609  4821 raft_consensus.cc:359] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.764411  4821 raft_consensus.cc:385] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.764761  4821 raft_consensus.cc:740] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.765406  4822 ts_tablet_manager.cc:1434] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.000s
I20260709 13:15:20.765468  4821 consensus_queue.cc:260] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.768136  4821 ts_tablet_manager.cc:1434] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:15:20.768366  4822 tablet_bootstrap.cc:492] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
W20260709 13:15:20.770610  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 45312.4us
I20260709 13:15:20.755048  4887 raft_consensus.cc:2804] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:20.771817  4887 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.772258  4887 raft_consensus.cc:3060] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.774129  5077 raft_consensus.cc:3060] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.775817  4821 tablet_bootstrap.cc:492] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:20.775952  4822 tablet_bootstrap.cc:654] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.779320  4823 raft_consensus.cc:359] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.780511  4823 raft_consensus.cc:385] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.780911  4823 raft_consensus.cc:740] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.781908  4823 consensus_queue.cc:260] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.784981  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:20.785677  4553 raft_consensus.cc:2468] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:20.791383  4821 tablet_bootstrap.cc:654] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.795728  5054 leader_election.cc:290] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:20.797389  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:20.798317  5077 raft_consensus.cc:2749] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:20.800423  4553 raft_consensus.cc:3060] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.801153  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:20.813125  4823 ts_tablet_manager.cc:1434] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.001s
I20260709 13:15:20.814507  4823 tablet_bootstrap.cc:492] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:20.825639  4553 raft_consensus.cc:2468] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:20.835119  4824 tablet_bootstrap.cc:654] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.840608  4957 consensus_queue.cc:1048] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:20.845701  4740 leader_election.cc:343] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:20.846692  4740 leader_election.cc:304] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:20.847627  5051 raft_consensus.cc:2804] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.848163  5051 raft_consensus.cc:697] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:20.849171  5051 consensus_queue.cc:237] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.832626  4887 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:20.877545  4887 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:20.878434  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:20.879422  4693 raft_consensus.cc:3060] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.890280  4823 tablet_bootstrap.cc:654] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:20.892967  4822 tablet_bootstrap.cc:492] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:20.893791  4822 ts_tablet_manager.cc:1403] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.126s	user 0.017s	sys 0.047s
I20260709 13:15:20.897478  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:20.898249  4553 raft_consensus.cc:3060] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:20.904112  4553 raft_consensus.cc:2468] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:20.923776  4822 raft_consensus.cc:359] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.927624  4822 raft_consensus.cc:385] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.934001  4821 tablet_bootstrap.cc:492] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:20.934830  4821 ts_tablet_manager.cc:1403] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.166s	user 0.013s	sys 0.003s
I20260709 13:15:20.928073  4822 raft_consensus.cc:740] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.949901  4740 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:20.951248  5057 raft_consensus.cc:2804] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:20.937682  4822 consensus_queue.cc:260] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:20.987272  4821 raft_consensus.cc:359] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.988193  4821 raft_consensus.cc:385] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:20.988528  4821 raft_consensus.cc:740] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:20.995914  4448 catalog_manager.cc:5697] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.996683  4821 consensus_queue.cc:260] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:20.999413  4822 ts_tablet_manager.cc:1434] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.105s	user 0.008s	sys 0.003s
I20260709 13:15:20.995537  5077 consensus_queue.cc:1048] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20260709 13:15:21.004149  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:20.970938  5057 raft_consensus.cc:697] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:21.005561  5057 consensus_queue.cc:237] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.008039  4822 tablet_bootstrap.cc:492] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:21.001155  5075 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.015482  5075 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.018157  5075 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
W20260709 13:15:20.965794  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 84341.6us
I20260709 13:15:21.021252  4821 ts_tablet_manager.cc:1434] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.001s
I20260709 13:15:21.034767  4821 tablet_bootstrap.cc:492] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:21.037093  4887 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.032480  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:21.042187  4696 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:21.043195  5077 consensus_queue.cc:1048] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:21.047003  4822 tablet_bootstrap.cc:654] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.047152  4593 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:21.049211  5163 consensus_queue.cc:1048] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:21.050720  4821 tablet_bootstrap.cc:654] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.052697  4824 tablet_bootstrap.cc:492] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:21.053005  5152 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:21.053272  4824 ts_tablet_manager.cc:1403] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.322s	user 0.022s	sys 0.014s
I20260709 13:15:21.053632  5152 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:21.054296  5152 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.037691  4887 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.057643  4887 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:21.060539  4824 raft_consensus.cc:359] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.061415  4824 raft_consensus.cc:385] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.061703  4824 raft_consensus.cc:740] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.062016  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:21.062386  4824 consensus_queue.cc:260] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
W20260709 13:15:21.064520  4740 leader_election.cc:343] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:21.052556  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:21.053555  5162 consensus_queue.cc:1048] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:21.077105  4824 ts_tablet_manager.cc:1434] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
W20260709 13:15:21.079361  4592 leader_election.cc:343] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:21.075371  4693 raft_consensus.cc:2468] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:21.130590  4824 tablet_bootstrap.cc:492] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:21.138868  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:21.151147  4823 tablet_bootstrap.cc:492] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:21.151711  4823 ts_tablet_manager.cc:1403] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.337s	user 0.013s	sys 0.031s
I20260709 13:15:21.142208  5151 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.163926  4824 tablet_bootstrap.cc:654] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.169713  4822 tablet_bootstrap.cc:492] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:21.170243  4822 ts_tablet_manager.cc:1403] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.163s	user 0.012s	sys 0.007s
I20260709 13:15:21.167016  5151 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.193743  4823 raft_consensus.cc:359] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
W20260709 13:15:21.201050  4740 leader_election.cc:343] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:21.201962  4740 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:21.203548  5151 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:21.205574  5057 raft_consensus.cc:2749] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:21.206334  5057 consensus_queue.cc:1048] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:21.208861  4823 raft_consensus.cc:385] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.209964  4823 raft_consensus.cc:740] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.209789  4822 raft_consensus.cc:359] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.212903  4822 raft_consensus.cc:385] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.213284  4822 raft_consensus.cc:740] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.217092  4988 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.217429  4822 consensus_queue.cc:260] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.217782  4988 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.219400  4821 tablet_bootstrap.cc:492] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:21.219952  4821 ts_tablet_manager.cc:1403] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.185s	user 0.032s	sys 0.024s
I20260709 13:15:21.220003  4988 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:21.222672  4821 raft_consensus.cc:359] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.223387  4821 raft_consensus.cc:385] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.223665  4821 raft_consensus.cc:740] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.224349  4821 consensus_queue.cc:260] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.214761  4823 consensus_queue.cc:260] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.234853  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:21.235926  5169 log.cc:927] Time spent T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.099s	user 0.004s	sys 0.000s
W20260709 13:15:21.239341  4666 leader_election.cc:343] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:21.233047  4822 ts_tablet_manager.cc:1434] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.002s
I20260709 13:15:21.244831  4821 ts_tablet_manager.cc:1434] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:15:21.246163  4821 tablet_bootstrap.cc:492] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:21.246822  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:21.231376  4823 ts_tablet_manager.cc:1434] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 13:15:21.213734  5152 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.262421  4822 tablet_bootstrap.cc:492] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:21.264460  4823 tablet_bootstrap.cc:492] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:21.266356  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:21.267113  4701 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
W20260709 13:15:21.267133  4666 leader_election.cc:343] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:21.268409  4666 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:21.255151  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 50939.3us
I20260709 13:15:21.272673  5075 raft_consensus.cc:493] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.273243  5075 raft_consensus.cc:515] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.275558  5075 leader_election.cc:290] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:21.287824  4593 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:21.290017  5075 raft_consensus.cc:2804] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:21.290457  5075 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:21.290817  5075 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.271790  4988 raft_consensus.cc:2749] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:21.301966  4821 tablet_bootstrap.cc:654] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.307629  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:21.308545  4551 raft_consensus.cc:2393] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:21.305717  5054 raft_consensus.cc:493] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.324651  5054 raft_consensus.cc:515] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.331077  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:21.331769  5054 leader_election.cc:290] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:21.333410  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:21.334218  4770 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:21.329686  4447 catalog_manager.cc:5697] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:21.340647  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:21.344857  4741 leader_election.cc:343] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:21.360685  4822 tablet_bootstrap.cc:654] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.359141  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:21.367867  5164 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.368508  5164 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.370931  5164 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:21.376353  4552 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.378327  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:21.386935  4821 tablet_bootstrap.cc:492] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:21.387818  4821 ts_tablet_manager.cc:1403] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.142s	user 0.014s	sys 0.006s
I20260709 13:15:21.378494  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:21.391520  4624 raft_consensus.cc:2393] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:21.404965  4594 leader_election.cc:304] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:21.402786  5152 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:21.397264  5175 consensus_queue.cc:1048] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:15:21.408422  5076 raft_consensus.cc:3060] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.436628  4821 raft_consensus.cc:359] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.439353  4821 raft_consensus.cc:385] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.441376  4552 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:21.439625  4821 raft_consensus.cc:740] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.446877  4821 consensus_queue.cc:260] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:21.427033  4740 leader_election.cc:343] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:21.448292  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:21.448593  4740 leader_election.cc:304] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:21.449029  4696 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.449592  5051 raft_consensus.cc:2749] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:21.454247  4592 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:21.459300  5177 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:21.459877  5177 raft_consensus.cc:697] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:21.461026  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:21.460657  5177 consensus_queue.cc:237] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.420490  5075 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.475510  4696 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:21.477792  4821 ts_tablet_manager.cc:1434] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.090s	user 0.008s	sys 0.004s
I20260709 13:15:21.479306  4821 tablet_bootstrap.cc:492] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:21.479313  5174 consensus_queue.cc:1048] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:21.488854  4823 tablet_bootstrap.cc:654] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.489866  4821 tablet_bootstrap.cc:654] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.486124  5076 raft_consensus.cc:2749] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:21.493014  4518 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:21.495198  5107 raft_consensus.cc:2749] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:21.515908  4822 tablet_bootstrap.cc:492] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:21.516690  4822 ts_tablet_manager.cc:1403] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.257s	user 0.028s	sys 0.012s
I20260709 13:15:21.528805  5075 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:21.551812  4822 raft_consensus.cc:359] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.571529  4822 raft_consensus.cc:385] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.572022  4822 raft_consensus.cc:740] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.561515  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:21.573261  4769 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.575301  5057 consensus_queue.cc:1048] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:21.581732  4769 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:21.582978  4822 consensus_queue.cc:260] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.585319  4822 ts_tablet_manager.cc:1434] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.068s	user 0.004s	sys 0.003s
I20260709 13:15:21.586404  4822 tablet_bootstrap.cc:492] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:21.586287  4594 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:21.588002  5076 raft_consensus.cc:2804] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:21.588677  5076 raft_consensus.cc:697] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:21.589732  5076 consensus_queue.cc:237] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.565500  4823 tablet_bootstrap.cc:492] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:21.567420  4824 tablet_bootstrap.cc:492] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:21.591173  4823 ts_tablet_manager.cc:1403] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.331s	user 0.018s	sys 0.005s
I20260709 13:15:21.555968  5163 consensus_queue.cc:1048] T 672896bdac10415b85afc981d3c6eac9 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:21.568593  4821 tablet_bootstrap.cc:492] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:21.591989  4821 ts_tablet_manager.cc:1403] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.113s	user 0.017s	sys 0.017s
I20260709 13:15:21.556563  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:21.592710  4695 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.591212  4824 ts_tablet_manager.cc:1403] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.497s	user 0.033s	sys 0.010s
I20260709 13:15:21.594081  4823 raft_consensus.cc:359] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.594811  4823 raft_consensus.cc:385] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.595094  4823 raft_consensus.cc:740] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.595753  4823 consensus_queue.cc:260] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.597853  4823 ts_tablet_manager.cc:1434] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:15:21.597782  4821 raft_consensus.cc:359] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.598634  4821 raft_consensus.cc:385] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.598994  4823 tablet_bootstrap.cc:492] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:21.596935  4824 raft_consensus.cc:359] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.599733  4824 raft_consensus.cc:385] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.600010  4824 raft_consensus.cc:740] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.600900  4824 consensus_queue.cc:260] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.598938  4821 raft_consensus.cc:740] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.602420  4821 consensus_queue.cc:260] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.605003  4822 tablet_bootstrap.cc:654] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.606019  4449 catalog_manager.cc:5697] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:21.613665  4823 tablet_bootstrap.cc:654] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.615612  4824 ts_tablet_manager.cc:1434] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:15:21.616780  4824 tablet_bootstrap.cc:492] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:21.623085  4695 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:21.603977  4821 ts_tablet_manager.cc:1434] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:15:21.653972  4821 tablet_bootstrap.cc:492] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:21.673898  4824 tablet_bootstrap.cc:654] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.697113  4821 tablet_bootstrap.cc:654] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.732466  5175 raft_consensus.cc:493] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.735635  4823 tablet_bootstrap.cc:492] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:21.736199  4823 ts_tablet_manager.cc:1403] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.137s	user 0.017s	sys 0.041s
I20260709 13:15:21.735848  5175 raft_consensus.cc:515] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.738193  5175 leader_election.cc:290] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:21.739174  4823 raft_consensus.cc:359] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.739917  4823 raft_consensus.cc:385] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.740219  4823 raft_consensus.cc:740] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.745978  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0c53efa0ee4dfbba4c3f1922027f50" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:21.746572  4822 tablet_bootstrap.cc:492] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:21.746713  4695 raft_consensus.cc:2468] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:21.747087  4822 ts_tablet_manager.cc:1403] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.161s	user 0.030s	sys 0.050s
I20260709 13:15:21.733121  5054 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.747851  5054 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.749245  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0c53efa0ee4dfbba4c3f1922027f50" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:21.752875  4822 raft_consensus.cc:359] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.753682  4822 raft_consensus.cc:385] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.753964  4822 raft_consensus.cc:740] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.754707  4822 consensus_queue.cc:260] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.757239  4822 ts_tablet_manager.cc:1434] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:15:21.770787  4822 tablet_bootstrap.cc:492] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:21.757781  4741 leader_election.cc:304] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:21.769414  4824 tablet_bootstrap.cc:492] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:21.782583  4824 ts_tablet_manager.cc:1403] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.166s	user 0.012s	sys 0.005s
I20260709 13:15:21.762064  5054 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:21.781888  5175 raft_consensus.cc:2804] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:21.785341  5175 raft_consensus.cc:493] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:21.785751  5175 raft_consensus.cc:3060] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.785444  4824 raft_consensus.cc:359] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.786190  4824 raft_consensus.cc:385] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.786500  4824 raft_consensus.cc:740] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.787281  4824 consensus_queue.cc:260] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.790331  4823 consensus_queue.cc:260] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.793869  4823 ts_tablet_manager.cc:1434] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.057s	user 0.002s	sys 0.005s
W20260709 13:15:21.802341  4740 leader_election.cc:343] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:21.794970  4823 tablet_bootstrap.cc:492] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:21.816926  4824 ts_tablet_manager.cc:1434] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:15:21.840796  4824 tablet_bootstrap.cc:492] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:21.846619  4822 tablet_bootstrap.cc:654] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.847747  4823 tablet_bootstrap.cc:654] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.851364  4824 tablet_bootstrap.cc:654] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.856282  5175 raft_consensus.cc:515] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.851898  4821 tablet_bootstrap.cc:492] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:21.857702  4821 ts_tablet_manager.cc:1403] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.210s	user 0.048s	sys 0.038s
I20260709 13:15:21.861187  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0c53efa0ee4dfbba4c3f1922027f50" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:21.861886  4695 raft_consensus.cc:3060] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.861629  4821 raft_consensus.cc:359] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.862707  4821 raft_consensus.cc:385] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.863073  4821 raft_consensus.cc:740] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.864114  4821 consensus_queue.cc:260] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.868062  5175 leader_election.cc:290] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:21.870047  4822 tablet_bootstrap.cc:492] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:21.870846  4822 ts_tablet_manager.cc:1403] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.100s	user 0.025s	sys 0.002s
I20260709 13:15:21.872539  4821 ts_tablet_manager.cc:1434] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:15:21.873812  4821 tablet_bootstrap.cc:492] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:21.873972  4822 raft_consensus.cc:359] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.874704  4822 raft_consensus.cc:385] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.874979  4822 raft_consensus.cc:740] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.875660  4822 consensus_queue.cc:260] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.877921  4822 ts_tablet_manager.cc:1434] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:21.878947  4822 tablet_bootstrap.cc:492] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:21.879499  4695 raft_consensus.cc:2468] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:21.880868  5162 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.880656  4821 tablet_bootstrap.cc:654] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.881461  5162 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.882021  4741 leader_election.cc:304] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:21.882899  5175 raft_consensus.cc:2804] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:21.883356  4824 tablet_bootstrap.cc:492] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:21.883713  5162 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:21.883965  4824 ts_tablet_manager.cc:1403] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.007s
I20260709 13:15:21.887001  4824 raft_consensus.cc:359] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.887769  4824 raft_consensus.cc:385] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.888057  4824 raft_consensus.cc:740] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.888151  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:21.888988  4695 raft_consensus.cc:2468] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:21.891100  4518 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:21.888906  4824 consensus_queue.cc:260] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.894093  4824 ts_tablet_manager.cc:1434] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:15:21.895156  4824 tablet_bootstrap.cc:492] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:21.883402  5175 raft_consensus.cc:697] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:21.896720  5175 consensus_queue.cc:237] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.900444  5162 raft_consensus.cc:2804] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:21.900949  5162 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:21.901314  5162 raft_consensus.cc:3060] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.906756  4823 tablet_bootstrap.cc:492] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:21.907320  4823 ts_tablet_manager.cc:1403] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.113s	user 0.005s	sys 0.006s
I20260709 13:15:21.907441  4821 tablet_bootstrap.cc:492] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:21.907934  4821 ts_tablet_manager.cc:1403] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.000s
I20260709 13:15:21.910180  4823 raft_consensus.cc:359] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.910952  4823 raft_consensus.cc:385] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.910648  4821 raft_consensus.cc:359] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.911221  4823 raft_consensus.cc:740] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.911338  4821 raft_consensus.cc:385] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.912014  4823 consensus_queue.cc:260] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.911593  5162 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.913918  4823 ts_tablet_manager.cc:1434] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:15:21.915019  4823 tablet_bootstrap.cc:492] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:21.915874  5162 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:21.916055  4822 tablet_bootstrap.cc:654] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.917353  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:21.918011  4695 raft_consensus.cc:3060] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.924831  4824 tablet_bootstrap.cc:654] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.911643  4821 raft_consensus.cc:740] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.929724  4821 consensus_queue.cc:260] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:21.931134  4450 catalog_manager.cc:5697] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:21.932827  4898 raft_consensus.cc:493] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.933387  4898 raft_consensus.cc:515] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.934125  4823 tablet_bootstrap.cc:654] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.935529  4898 leader_election.cc:290] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:21.936865  5075 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.937345  5075 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:21.939491  5075 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:21.940398  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:21.940658  4824 tablet_bootstrap.cc:492] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:21.941391  4824 ts_tablet_manager.cc:1403] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.046s	user 0.005s	sys 0.007s
I20260709 13:15:21.932853  4821 ts_tablet_manager.cc:1434] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.004s
I20260709 13:15:21.945377  4821 tablet_bootstrap.cc:492] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:21.945268  4824 raft_consensus.cc:359] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.946468  4824 raft_consensus.cc:385] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:21.946817  4824 raft_consensus.cc:740] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:21.949208  4824 consensus_queue.cc:260] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:21.949925  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0c53efa0ee4dfbba4c3f1922027f50" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:21.950649  4624 raft_consensus.cc:3060] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.952317  4695 raft_consensus.cc:2468] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:21.955790  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:21.952240  4824 ts_tablet_manager.cc:1434] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:15:21.956454  4774 raft_consensus.cc:2468] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:21.957449  4824 tablet_bootstrap.cc:492] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:21.958408  4518 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:21.960549  4821 tablet_bootstrap.cc:654] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:21.960518  5162 raft_consensus.cc:2804] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:21.965843  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:21.970243  4668 leader_election.cc:304] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:21.974888  4957 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.969337  5162 raft_consensus.cc:697] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:21.978675  5162 consensus_queue.cc:237] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:21.983096  4869 raft_consensus.cc:2804] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:15:21.977061  4740 leader_election.cc:343] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:21.994189  4666 leader_election.cc:343] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:21.985330  4869 raft_consensus.cc:493] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:21.998207  4869 raft_consensus.cc:3060] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:21.975841  4957 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:22.005942  4957 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:22.013698  4823 tablet_bootstrap.cc:492] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:22.014315  4823 ts_tablet_manager.cc:1403] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.100s	user 0.021s	sys 0.037s
I20260709 13:15:22.017289  4823 raft_consensus.cc:359] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.018046  4823 raft_consensus.cc:385] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:22.018347  4823 raft_consensus.cc:740] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:22.019083  4823 consensus_queue.cc:260] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.021294  4823 ts_tablet_manager.cc:1434] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:15:22.022383  4823 tablet_bootstrap.cc:492] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:22.022266  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:22.023093  4769 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:22.029394  4823 tablet_bootstrap.cc:654] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:22.059976  4869 raft_consensus.cc:515] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.062321  4869 leader_election.cc:290] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:22.217087  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:22.246335  4822 tablet_bootstrap.cc:492] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:22.247838  4822 ts_tablet_manager.cc:1403] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.369s	user 0.035s	sys 0.035s
I20260709 13:15:22.253922  4624 raft_consensus.cc:2468] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:22.244992  4594 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:22.256116  5151 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:22.256780  5151 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.257258  5151 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:22.242769  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:22.258466  4769 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:22.270354  4824 tablet_bootstrap.cc:654] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:22.244637  5162 raft_consensus.cc:493] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.276593  4821 tablet_bootstrap.cc:492] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:22.277204  4821 ts_tablet_manager.cc:1403] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.332s	user 0.023s	sys 0.012s
I20260709 13:15:22.280848  5076 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.345428  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:22.349279  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:22.352077  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:22.357872  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:22.358644  4551 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
W20260709 13:15:22.363001  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 325756us
I20260709 13:15:22.364785  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:22.414289  5209 raft_consensus.cc:493] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.415107  5209 raft_consensus.cc:515] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:22.418345  5209 leader_election.cc:290] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:22.439735  4823 tablet_bootstrap.cc:492] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:22.440593  4823 ts_tablet_manager.cc:1403] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.418s	user 0.071s	sys 0.071s
I20260709 13:15:22.493647  4823 raft_consensus.cc:359] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:22.494679  4823 raft_consensus.cc:385] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:22.495069  4823 raft_consensus.cc:740] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:22.503741  4823 consensus_queue.cc:260] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:22.508924  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 131498us
I20260709 13:15:22.512980  5212 consensus_queue.cc:1048] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:22.520021  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:22.520704  5217 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.521373  5217 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.531067  4823 ts_tablet_manager.cc:1434] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.090s	user 0.009s	sys 0.000s
I20260709 13:15:22.538204  5215 raft_consensus.cc:493] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.538986  5215 raft_consensus.cc:515] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:22.532207  5217 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:22.541257  5107 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.551505  5107 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:22.553347  4823 tablet_bootstrap.cc:492] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:22.558676  5107 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:22.560633  5215 leader_election.cc:290] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:22.563937  4823 tablet_bootstrap.cc:654] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:22.565259  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:22.568274  4517 leader_election.cc:343] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:22.581636  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:22.582563  4696 raft_consensus.cc:2468] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:22.592604  4518 leader_election.cc:304] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:22.594012  5107 raft_consensus.cc:2804] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:22.594574  5107 raft_consensus.cc:493] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.594996  5107 raft_consensus.cc:3060] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:22.597582  5151 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:22.599985  4627 raft_consensus.cc:2393] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:22.601784  5151 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:22.606477  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:22.607223  4551 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:22.607676  5209 consensus_queue.cc:1048] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:22.628217  4824 tablet_bootstrap.cc:492] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:22.628769  4824 ts_tablet_manager.cc:1403] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.672s	user 0.005s	sys 0.021s
I20260709 13:15:22.629354  4823 tablet_bootstrap.cc:492] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:22.629910  4823 ts_tablet_manager.cc:1403] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.018s
I20260709 13:15:22.630842  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:22.637404  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:22.643167  5107 raft_consensus.cc:515] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:22.645210  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:22.645311  4666 leader_election.cc:343] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:22.646338  4549 raft_consensus.cc:2393] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:22.638892  5162 raft_consensus.cc:515] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.648181  4547 raft_consensus.cc:3060] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:22.649537  5162 leader_election.cc:290] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:22.651889  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:22.653139  4701 raft_consensus.cc:2393] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:22.654783  4823 raft_consensus.cc:359] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.655535  4824 raft_consensus.cc:359] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:22.656486  4824 raft_consensus.cc:385] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:22.655582  4823 raft_consensus.cc:385] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:22.657877  4823 raft_consensus.cc:740] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:22.658785  4823 consensus_queue.cc:260] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.661823  4823 ts_tablet_manager.cc:1434] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.002s
I20260709 13:15:22.656781  4824 raft_consensus.cc:740] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:22.664083  4824 consensus_queue.cc:260] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:22.663015  4547 raft_consensus.cc:2468] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:22.669440  4519 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:22.676126  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:22.676977  4770 raft_consensus.cc:2468] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:22.678074  4824 ts_tablet_manager.cc:1434] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 13:15:22.678681  4823 tablet_bootstrap.cc:492] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:22.684711  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:22.685509  4770 raft_consensus.cc:3060] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:22.689415  4823 tablet_bootstrap.cc:654] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:22.694948  4824 tablet_bootstrap.cc:492] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:22.681668  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:22.695822  4772 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:22.703841  4740 leader_election.cc:343] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:22.704623  4740 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:22.705618  5223 raft_consensus.cc:2749] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:22.705870  5075 consensus_queue.cc:1048] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:22.281418  5076 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:22.712668  4628 raft_consensus.cc:2468] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:22.724610  4824 tablet_bootstrap.cc:654] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:22.724460  4770 raft_consensus.cc:2468] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:22.735136  5076 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:22.739499  4824 tablet_bootstrap.cc:492] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:22.741583  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:22.742522  4824 ts_tablet_manager.cc:1403] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.048s	user 0.015s	sys 0.013s
I20260709 13:15:22.745146  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:22.745955  4773 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:22.746610  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:22.747468  4776 raft_consensus.cc:2393] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:22.740532  4822 raft_consensus.cc:359] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:22.758725  4822 raft_consensus.cc:385] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:22.759102  4772 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:22.759310  4822 raft_consensus.cc:740] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:22.757895  4824 raft_consensus.cc:359] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:22.760070  4824 raft_consensus.cc:385] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:22.760365  4824 raft_consensus.cc:740] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:22.760667  4822 consensus_queue.cc:260] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:22.761847  4625 raft_consensus.cc:2468] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:22.764909  4822 ts_tablet_manager.cc:1434] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.517s	user 0.010s	sys 0.007s
W20260709 13:15:22.765535  4822 ts_tablet_manager.cc:1467] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82: Tablet startup took 517ms
I20260709 13:15:22.764521  4824 consensus_queue.cc:260] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:22.767069  4824 ts_tablet_manager.cc:1434] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.002s
I20260709 13:15:22.768239  4824 tablet_bootstrap.cc:492] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:22.767879  5051 consensus_queue.cc:1048] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:22.773610  4519 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:22.765926  4822 ts_tablet_manager.cc:1469] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82: Trace:
0709 13:15:13.548207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.548503 (+   296us) service_pool.cc:224] Handling call
0709 13:15:13.550467 (+  1964us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:15:13.550561 (+    94us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:15:13.598377 (+ 47816us) tablet_metadata.cc:665] Metadata flushed
0709 13:15:13.696809 (+ 98432us) tablet_replica.cc:206] Creating consensus instance
0709 13:15:13.697397 (+   588us) inbound_call.cc:177] Queueing success response
0709 13:15:21.878529 (+8181132us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:15:22.248233 (+369704us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:15:22.738482 (+490249us) tablet_replica.cc:261] Starting instrumentation
0709 13:15:22.738976 (+   494us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":1462,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":112,"tablet-open.queue_time_us":8181268}
I20260709 13:15:22.776902  4823 tablet_bootstrap.cc:492] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:22.777455  4823 ts_tablet_manager.cc:1403] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.115s	user 0.020s	sys 0.018s
I20260709 13:15:22.280275  4821 raft_consensus.cc:359] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.778415  4821 raft_consensus.cc:385] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:22.778779  4821 raft_consensus.cc:740] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:22.779229  4519 leader_election.cc:304] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:22.779758  4821 consensus_queue.cc:260] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.788681  4821 ts_tablet_manager.cc:1434] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.511s	user 0.012s	sys 0.007s
W20260709 13:15:22.789043  4821 ts_tablet_manager.cc:1467] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512: Tablet startup took 511ms
I20260709 13:15:22.780352  5210 raft_consensus.cc:2749] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:15:22.789359  4821 ts_tablet_manager.cc:1469] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512: Trace:
0709 13:15:13.899122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.899385 (+   263us) service_pool.cc:224] Handling call
0709 13:15:13.901737 (+  2352us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:15:13.901848 (+   111us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:15:13.944294 (+ 42446us) tablet_metadata.cc:665] Metadata flushed
0709 13:15:14.066794 (+122500us) tablet_replica.cc:206] Creating consensus instance
0709 13:15:14.112176 (+ 45382us) inbound_call.cc:177] Queueing success response
0709 13:15:21.944997 (+7832821us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:15:22.277487 (+332490us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:15:22.278000 (+   513us) tablet_replica.cc:261] Starting instrumentation
0709 13:15:22.278685 (+   685us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":250,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"tablet-open.queue_time_us":7866229}
I20260709 13:15:22.792692  4821 tablet_bootstrap.cc:492] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:22.780498  4823 raft_consensus.cc:359] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:22.796936  4823 raft_consensus.cc:385] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:22.797322  4823 raft_consensus.cc:740] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:22.797217  5075 consensus_queue.cc:1048] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:22.798257  4823 consensus_queue.cc:260] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:22.777455  5162 raft_consensus.cc:2804] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:22.799557  5162 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.799996  5162 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:22.801043  4823 ts_tablet_manager.cc:1434] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:15:22.802451  4823 tablet_bootstrap.cc:492] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:22.803308  4624 raft_consensus.cc:3060] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:22.764556  4551 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:22.781796  4824 tablet_bootstrap.cc:654] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:22.783820  4668 leader_election.cc:304] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:22.805024  5209 raft_consensus.cc:2764] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:15:22.785730  4666 leader_election.cc:304] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:22.809876  4594 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:22.810830  5076 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:22.811355  5076 raft_consensus.cc:697] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:22.811993  4988 raft_consensus.cc:2804] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:22.812182  5076 consensus_queue.cc:237] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:22.812705  4988 raft_consensus.cc:697] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:22.813650  4988 consensus_queue.cc:237] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.814302  5065 raft_consensus.cc:3060] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:22.815941  5051 raft_consensus.cc:493] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.818821  4821 tablet_bootstrap.cc:654] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:22.809902  4668 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:22.820971  4869 raft_consensus.cc:2804] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:22.821413  4869 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.816458  5051 raft_consensus.cc:515] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:22.821780  4869 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:22.825021  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:22.824580  4823 tablet_bootstrap.cc:654] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:22.855698  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 49422.4us
I20260709 13:15:22.857613  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:22.858189  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beeb8c8bf7343c297daf017c542c67b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:22.859455  5107 leader_election.cc:290] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:22.880100  4822 tablet_bootstrap.cc:492] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
W20260709 13:15:22.881031  4740 leader_election.cc:343] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:22.897955  5175 raft_consensus.cc:493] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.911386  5065 raft_consensus.cc:2749] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:22.907661  5051 leader_election.cc:290] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:22.891386  5212 consensus_queue.cc:1048] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:15:22.901863  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:22.934059  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:22.934787  4625 raft_consensus.cc:3060] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:22.914187  5175 raft_consensus.cc:515] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.939203  5175 leader_election.cc:290] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:22.940163  5209 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.940785  5209 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:22.956609  5209 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:22.958477  4824 tablet_bootstrap.cc:492] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:22.958264  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:22.961922  5163 consensus_queue.cc:1048] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:22.963850  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beeb8c8bf7343c297daf017c542c67b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:22.964525  4701 raft_consensus.cc:2468] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:22.966138  4593 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:22.969051  5212 raft_consensus.cc:2749] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:22.969802  4824 ts_tablet_manager.cc:1403] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.202s	user 0.023s	sys 0.006s
I20260709 13:15:22.970227  5162 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:22.975486  4822 tablet_bootstrap.cc:654] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:22.977104  4693 raft_consensus.cc:2468] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:22.979216  5162 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:22.982648  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:22.983114  5107 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.002440  4741 leader_election.cc:304] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:22.983250  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:22.984630  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:22.996395  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:22.999750  4824 raft_consensus.cc:359] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:23.009199  4824 raft_consensus.cc:385] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:23.009680  4824 raft_consensus.cc:740] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:23.005496  4821 tablet_bootstrap.cc:492] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:23.010694  4821 ts_tablet_manager.cc:1403] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.218s	user 0.020s	sys 0.016s
I20260709 13:15:23.011675  4824 consensus_queue.cc:260] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:23.015247  4694 raft_consensus.cc:2468] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:23.002344  4869 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:23.016240  4772 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:23.016307  4625 raft_consensus.cc:2468] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:23.017459  4770 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:23.018188  4517 leader_election.cc:304] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:23.019170  5162 raft_consensus.cc:2804] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:23.019703  5162 raft_consensus.cc:697] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:23.020328  4695 raft_consensus.cc:2393] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:23.019335  5107 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:23.022569  4519 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:23.022913  5107 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:23.020576  5162 consensus_queue.cc:237] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:23.024243  4957 raft_consensus.cc:2804] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:23.024722  4957 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:23.025013  5223 raft_consensus.cc:2804] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:23.025168  4957 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:23.025496  5223 raft_consensus.cc:493] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:23.025555  4772 raft_consensus.cc:2468] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:23.026000  5223 raft_consensus.cc:3060] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:23.037914  4823 tablet_bootstrap.cc:492] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
W20260709 13:15:23.038125  4740 leader_election.cc:343] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:23.038540  4823 ts_tablet_manager.cc:1403] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.236s	user 0.008s	sys 0.015s
I20260709 13:15:23.014885  4824 ts_tablet_manager.cc:1434] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 13:15:23.030330  4821 raft_consensus.cc:359] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:23.041493  4821 raft_consensus.cc:385] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:23.017369  4869 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:23.041935  4821 raft_consensus.cc:740] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:23.046357  4821 consensus_queue.cc:260] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:23.058818  4824 tablet_bootstrap.cc:492] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:23.059542  4821 ts_tablet_manager.cc:1434] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.003s
I20260709 13:15:23.061681  4741 leader_election.cc:304] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:23.048777  4988 raft_consensus.cc:493] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.055030  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:23.064527  5054 raft_consensus.cc:2804] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:23.078902  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:23.074680  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:23.082048  5054 raft_consensus.cc:493] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:23.101006  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:23.062446  4519 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:23.074357  4988 raft_consensus.cc:515] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:23.128181  5164 raft_consensus.cc:2804] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:15:23.127964  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 70997.8us
I20260709 13:15:23.123410  5054 raft_consensus.cc:3060] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:23.115505  4770 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:23.078447  4957 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:23.150733  4823 raft_consensus.cc:359] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:23.151885  4823 raft_consensus.cc:385] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:23.152185  4823 raft_consensus.cc:740] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:23.155774  4988 leader_election.cc:290] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:23.157568  4547 raft_consensus.cc:2393] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:23.152689  4823 consensus_queue.cc:260] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:23.147040  5164 raft_consensus.cc:697] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:23.162040  4824 tablet_bootstrap.cc:654] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:23.161736  5164 consensus_queue.cc:237] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:23.164405  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:23.166682  4518 leader_election.cc:343] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:23.167936  4666 leader_election.cc:343] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:23.172717  4823 ts_tablet_manager.cc:1434] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.134s	user 0.010s	sys 0.000s
I20260709 13:15:23.155776  4957 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:23.178129  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:23.180325  4770 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:23.197155  4822 tablet_bootstrap.cc:492] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:23.200137  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
W20260709 13:15:23.206584  4666 leader_election.cc:343] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:23.206789  4694 raft_consensus.cc:2393] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:23.207077  4666 leader_election.cc:304] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:23.207820  4988 raft_consensus.cc:2749] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:23.208814  4822 ts_tablet_manager.cc:1403] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.422s	user 0.030s	sys 0.023s
I20260709 13:15:23.208587  4668 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:23.211115  4988 raft_consensus.cc:2804] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:23.211910  4988 raft_consensus.cc:697] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:23.212822  4988 consensus_queue.cc:237] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:23.213570  4822 raft_consensus.cc:359] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:23.221019  4822 raft_consensus.cc:385] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:23.221416  4822 raft_consensus.cc:740] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:23.222903  4822 consensus_queue.cc:260] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:23.228595  4822 ts_tablet_manager.cc:1434] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:15:23.235412  5223 raft_consensus.cc:515] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:23.244689  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:23.245623  4770 raft_consensus.cc:2393] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:23.246220  4822 tablet_bootstrap.cc:492] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:23.261245  5151 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.261861  5151 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:23.264266  5151 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:23.249363  5054 raft_consensus.cc:515] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:23.267439  5223 leader_election.cc:290] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:23.271819  4823 tablet_bootstrap.cc:492] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:23.269411  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beeb8c8bf7343c297daf017c542c67b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
W20260709 13:15:23.275695  4740 leader_election.cc:343] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:23.273403  4821 tablet_bootstrap.cc:492] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:23.271843  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
W20260709 13:15:23.276453  4517 leader_election.cc:343] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:23.282191  4547 raft_consensus.cc:3060] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:23.288379  4823 tablet_bootstrap.cc:654] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:23.282464  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
W20260709 13:15:23.317754  4649 debug-util.cc:398] Leaking SignalData structure 0x7b08002d5f00 after lost signal to thread 26962
I20260709 13:15:23.409974  4822 tablet_bootstrap.cc:654] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:23.573782  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 332994us
I20260709 13:15:23.267678  5054 leader_election.cc:290] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:23.633512  4623 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:23.726454  4824 tablet_bootstrap.cc:492] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:23.727313  4824 ts_tablet_manager.cc:1403] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.671s	user 0.023s	sys 0.018s
I20260709 13:15:23.731979  4821 tablet_bootstrap.cc:654] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:23.678764  5175 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:23.749660  4649 debug-util.cc:398] Leaking SignalData structure 0x7b08002baf00 after lost signal to thread 4485
I20260709 13:15:23.753451  5175 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:24.247498  4821 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.023s	sys 0.039s
W20260709 13:15:24.248023  4821 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.023s	sys 0.039s
I20260709 13:15:24.250912  4824 raft_consensus.cc:359] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.251864  4824 raft_consensus.cc:385] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.252221  4824 raft_consensus.cc:740] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.253034  4824 consensus_queue.cc:260] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.244413  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:24.265688  5251 log.cc:927] Time spent T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.443s	user 0.001s	sys 0.000s
I20260709 13:15:24.268122  5175 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:24.276678  4821 tablet_bootstrap.cc:492] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:24.277319  4821 ts_tablet_manager.cc:1403] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 1.004s	user 0.032s	sys 0.045s
I20260709 13:15:24.277474  4824 ts_tablet_manager.cc:1434] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.550s	user 0.009s	sys 0.000s
W20260709 13:15:24.279999  4824 ts_tablet_manager.cc:1467] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a: Tablet startup took 552ms
I20260709 13:15:24.279613  4822 tablet_bootstrap.cc:492] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
W20260709 13:15:24.280375  4824 ts_tablet_manager.cc:1469] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a: Trace:
0709 13:15:13.987252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.006696 (+ 19444us) service_pool.cc:224] Handling call
0709 13:15:14.008358 (+  1662us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:15:14.008442 (+    84us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:15:14.043261 (+ 34819us) tablet_metadata.cc:665] Metadata flushed
0709 13:15:14.057997 (+ 14736us) tablet_replica.cc:206] Creating consensus instance
0709 13:15:14.058564 (+   567us) inbound_call.cc:177] Queueing success response
0709 13:15:23.051870 (+8993306us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:15:23.727681 (+675811us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:15:24.228409 (+500728us) tablet_replica.cc:261] Starting instrumentation
0709 13:15:24.230395 (+  1986us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":8209,"raft.queue_time_us":8617,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":129,"tablet-open.queue_time_us":8993404}
I20260709 13:15:24.283107  4822 ts_tablet_manager.cc:1403] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 1.039s	user 0.017s	sys 0.007s
I20260709 13:15:24.284232  4824 tablet_bootstrap.cc:492] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:24.286626  4823 tablet_bootstrap.cc:492] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:24.287635  4823 ts_tablet_manager.cc:1403] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 1.016s	user 0.008s	sys 0.011s
I20260709 13:15:24.290606  4823 raft_consensus.cc:359] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.291199  4823 raft_consensus.cc:385] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.291443  4823 raft_consensus.cc:740] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.291602  4824 tablet_bootstrap.cc:654] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:24.292060  4823 consensus_queue.cc:260] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.297962  4824 tablet_bootstrap.cc:492] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:24.298657  4824 ts_tablet_manager.cc:1403] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:15:24.301946  4824 raft_consensus.cc:359] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.302421  4823 ts_tablet_manager.cc:1434] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:15:24.302731  4824 raft_consensus.cc:385] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.303166  4824 raft_consensus.cc:740] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.304035  4823 tablet_bootstrap.cc:492] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:24.303936  4824 consensus_queue.cc:260] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.306157  4824 ts_tablet_manager.cc:1434] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:15:24.307395  4824 tablet_bootstrap.cc:492] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:24.312968  4823 tablet_bootstrap.cc:654] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:24.314172  4824 tablet_bootstrap.cc:654] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:24.320850  4623 consensus_meta.cc:287] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82: Time spent flushing consensus metadata: real 0.607s	user 0.005s	sys 0.015s
I20260709 13:15:24.321540  4623 raft_consensus.cc:2468] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:24.322461  4623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48806 (request call id 62) took 1332 ms. Trace:
W20260709 13:15:24.342180  4665 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.010s	sys 0.026s
W20260709 13:15:24.342685  4665 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.010s	sys 0.026s
I20260709 13:15:24.343955  4824 tablet_bootstrap.cc:492] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:24.345765  5257 raft_consensus.cc:493] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:24.346060  5256 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.346460  5257 raft_consensus.cc:515] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.346871  5256 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.349725  5256 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:24.349822  5257 leader_election.cc:290] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:24.350587  4824 ts_tablet_manager.cc:1403] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.005s
I20260709 13:15:24.322979  4623 rpcz_store.cc:276] 0709 13:15:22.989897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.998794 (+  8897us) service_pool.cc:224] Handling call
0709 13:15:23.282259 (+283465us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b5800003f98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:15:24.322335 (+1040076us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13069184}
I20260709 13:15:24.352445  5257 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.353102  5257 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.354861  4824 raft_consensus.cc:359] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.355412  5257 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.355675  4824 raft_consensus.cc:385] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.356056  4824 raft_consensus.cc:740] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.356894  4824 consensus_queue.cc:260] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.360844  4823 tablet_bootstrap.cc:492] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:24.361461  4823 ts_tablet_manager.cc:1403] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.058s	user 0.029s	sys 0.018s
I20260709 13:15:24.364766  4823 raft_consensus.cc:359] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.365723  4823 raft_consensus.cc:385] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.365721  4824 ts_tablet_manager.cc:1434] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:15:24.366074  4823 raft_consensus.cc:740] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.367125  4823 consensus_queue.cc:260] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.369638  4823 ts_tablet_manager.cc:1434] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:15:24.367586  4824 tablet_bootstrap.cc:492] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:24.370775  4823 tablet_bootstrap.cc:492] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:24.377820  4823 tablet_bootstrap.cc:654] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:23.282380  4517 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:24.383708  4824 tablet_bootstrap.cc:654] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:24.385130  5264 raft_consensus.cc:2749] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:15:24.385389  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 1.10886e+06us
I20260709 13:15:24.395231  5270 consensus_queue.cc:1048] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:24.399333  4824 tablet_bootstrap.cc:492] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:24.400010  4824 ts_tablet_manager.cc:1403] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.005s
I20260709 13:15:24.416883  4823 tablet_bootstrap.cc:492] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:24.417361  4823 ts_tablet_manager.cc:1403] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.007s
W20260709 13:15:24.417836  4649 debug-util.cc:398] Leaking SignalData structure 0x7b080011d200 after lost signal to thread 4577
W20260709 13:15:24.419445  4649 debug-util.cc:398] Leaking SignalData structure 0x7b080022ca60 after lost signal to thread 4652
W20260709 13:15:24.421226  4649 debug-util.cc:398] Leaking SignalData structure 0x7b08000c8180 after lost signal to thread 4726
W20260709 13:15:24.422761  4649 debug-util.cc:398] Leaking SignalData structure 0x7b08002a7b00 after lost signal to thread 4800
W20260709 13:15:24.423617  4649 debug-util.cc:398] Leaking SignalData structure 0x7b0800060580 after lost signal to thread 5140
W20260709 13:15:24.424188  4649 debug-util.cc:398] Leaking SignalData structure 0x7b0800321820 after lost signal to thread 5174
W20260709 13:15:24.424650  4649 debug-util.cc:398] Leaking SignalData structure 0x7b08000b1e60 after lost signal to thread 5193
I20260709 13:15:24.432953  5257 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.433710  5257 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.433917  4824 raft_consensus.cc:359] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.434132  4823 raft_consensus.cc:359] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.434760  4824 raft_consensus.cc:385] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.434983  4823 raft_consensus.cc:385] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.435083  4824 raft_consensus.cc:740] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.435490  4823 raft_consensus.cc:740] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.435914  4824 consensus_queue.cc:260] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.436192  4823 consensus_queue.cc:260] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.436172  5257 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:24.437758  4824 ts_tablet_manager.cc:1434] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.003s
I20260709 13:15:24.433001  5271 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.438819  4824 tablet_bootstrap.cc:492] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:24.438539  5271 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.440006  4823 ts_tablet_manager.cc:1434] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
W20260709 13:15:24.424975  4649 debug-util.cc:398] Leaking SignalData structure 0x7b08002fb720 after lost signal to thread 5196
I20260709 13:15:24.442644  4823 tablet_bootstrap.cc:492] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
W20260709 13:15:24.446342  4666 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.030s	sys 0.064s
W20260709 13:15:24.446774  4666 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.030s	sys 0.064s
I20260709 13:15:24.449328  5271 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:24.452963  5271 raft_consensus.cc:493] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.453593  5271 raft_consensus.cc:515] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:24.454880  4668 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.020s	sys 0.027s
W20260709 13:15:24.455325  4668 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.020s	sys 0.027s
I20260709 13:15:24.455598  4824 tablet_bootstrap.cc:654] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:24.456538  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:24.456673  4823 tablet_bootstrap.cc:654] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:24.456815  5271 leader_election.cc:290] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:24.461586  5271 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:24.462211  5271 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.464814  5271 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.467052  4823 tablet_bootstrap.cc:492] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:24.467880  4823 ts_tablet_manager.cc:1403] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.026s	user 0.004s	sys 0.012s
I20260709 13:15:24.472283  4824 tablet_bootstrap.cc:492] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:24.472911  4824 ts_tablet_manager.cc:1403] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.000s
I20260709 13:15:24.477895  4824 raft_consensus.cc:359] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.478828  4824 raft_consensus.cc:385] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.479243  4824 raft_consensus.cc:740] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.479990  4824 consensus_queue.cc:260] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.482174  4824 ts_tablet_manager.cc:1434] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:15:24.484261  4824 tablet_bootstrap.cc:492] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:24.485517  4823 raft_consensus.cc:359] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.488890  5065 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:24.491470  4823 raft_consensus.cc:385] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.491945  4823 raft_consensus.cc:740] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.491627  5065 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.492872  4823 consensus_queue.cc:260] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.494526  5065 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:24.495507  4823 ts_tablet_manager.cc:1434] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:15:24.499615  4823 tablet_bootstrap.cc:492] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:24.506934  4823 tablet_bootstrap.cc:654] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:24.498346  4824 tablet_bootstrap.cc:654] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:24.514320  5270 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:24.515012  5270 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.516897  5271 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.517535  5271 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.520792  5270 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.512058  4448 catalog_manager.cc:5697] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 1 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:24.523684  4448 catalog_manager.cc:5697] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } }
I20260709 13:15:24.529085  5065 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.536253  5065 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.538791  5065 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
W20260709 13:15:24.538944  4547 consensus_meta.cc:287] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc: Time spent flushing consensus metadata: real 1.256s	user 0.013s	sys 0.004s
I20260709 13:15:24.539391  4547 raft_consensus.cc:2468] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:24.532537  5271 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:24.540002  4547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51884 (request call id 91) took 1268 ms. Trace:
I20260709 13:15:24.540453  4547 rpcz_store.cc:276] 0709 13:15:23.271335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.271522 (+   187us) service_pool.cc:224] Handling call
0709 13:15:24.539908 (+1268386us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:24.557229  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:24.603775  4823 tablet_bootstrap.cc:492] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:24.607460  4823 ts_tablet_manager.cc:1403] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.108s	user 0.019s	sys 0.024s
I20260709 13:15:24.611346  4823 raft_consensus.cc:359] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.606730  4824 tablet_bootstrap.cc:492] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:24.612996  4824 ts_tablet_manager.cc:1403] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.129s	user 0.028s	sys 0.019s
I20260709 13:15:24.616595  4824 raft_consensus.cc:359] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.617542  4824 raft_consensus.cc:385] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.617877  4824 raft_consensus.cc:740] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.618688  4824 consensus_queue.cc:260] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.621016  4823 raft_consensus.cc:385] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.621500  4823 raft_consensus.cc:740] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.623450  4824 ts_tablet_manager.cc:1434] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:15:24.626495  4824 tablet_bootstrap.cc:492] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:24.624281  4823 consensus_queue.cc:260] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.636574  5256 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.641816  4824 tablet_bootstrap.cc:654] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:24.637501  5256 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.664047  5271 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.671208  5279 raft_consensus.cc:493] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:24.671941  5279 raft_consensus.cc:515] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.674717  5279 leader_election.cc:290] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:24.676398  4823 ts_tablet_manager.cc:1434] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.068s	user 0.018s	sys 0.017s
I20260709 13:15:24.680186  5249 raft_consensus.cc:493] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:24.682577  4821 raft_consensus.cc:359] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.683876  4821 raft_consensus.cc:385] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:24.684216  4821 raft_consensus.cc:740] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:24.670285  5271 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.680625  4823 tablet_bootstrap.cc:492] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:24.689586  4821 consensus_queue.cc:260] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.695399  5175 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.696076  5175 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.696628  5271 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:24.699807  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:24.701503  4770 raft_consensus.cc:2468] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:24.703881  5258 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.704584  5258 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.708703  5258 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.711650  5279 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:24.712330  5279 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.714238  5270 raft_consensus.cc:493] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:24.715005  5279 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.714918  5270 raft_consensus.cc:515] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.715983  5271 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.716545  5271 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.717939  5256 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:24.718575  5256 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.718943  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:24.719100  5256 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.693825  5248 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:24.720777  5271 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:24.721324  5248 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.722018  5275 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.689311  5257 raft_consensus.cc:493] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.726256  5281 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:24.726400  5271 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:24.730973  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.733592  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.733592  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.683588  5249 raft_consensus.cc:515] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.741210  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:24.741649  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:24.742422  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:24.745347  4694 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:24.751417  5270 leader_election.cc:290] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.752354  4701 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:24.753593  5270 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:24.754354  5270 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.736711  5279 raft_consensus.cc:493] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.755664  5279 raft_consensus.cc:515] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.756263  4776 raft_consensus.cc:2393] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:24.758076  5279 leader_election.cc:290] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.758646  5279 raft_consensus.cc:493] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:24.759158  5270 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.759964  5270 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:24.760588  4548 raft_consensus.cc:2468] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:24.739656  5257 raft_consensus.cc:515] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.760553  5270 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.763166  5257 leader_election.cc:290] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:24.743476  5275 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.759189  5279 raft_consensus.cc:515] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.770537  5270 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.771265  5275 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:24.776648  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:24.777431  4553 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:24.744378  5271 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.781455  5271 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:24.782439  5256 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:24.782891  5258 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.783211  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.783448  5258 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.764825  5281 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.784037  4694 raft_consensus.cc:2468] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:24.785115  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.788347  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:24.790089  4546 raft_consensus.cc:2468] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:24.788542  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.791193  5281 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:24.791399  4693 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:24.792668  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:24.793694  4548 raft_consensus.cc:2393] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:24.796285  5057 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:24.796998  5057 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.798409  5258 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.800501  4666 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:24.801471  4666 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:24.802261  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.803211  5256 raft_consensus.cc:2804] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:24.803745  5256 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.803668  5286 consensus_queue.cc:1048] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:15:24.804096  4519 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.196:36999 timed out after 1.521s (SENT) [suppressed 1 similar messages]
I20260709 13:15:24.805151  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.806041  4518 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:24.778241  5257 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:24.806941  5286 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.804602  4821 ts_tablet_manager.cc:1434] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.527s	user 0.006s	sys 0.004s
W20260709 13:15:24.809690  4821 ts_tablet_manager.cc:1467] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512: Tablet startup took 532ms
I20260709 13:15:24.811429  5260 raft_consensus.cc:493] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
W20260709 13:15:24.810148  4821 ts_tablet_manager.cc:1469] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512: Trace:
0709 13:15:13.934093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.936823 (+  2730us) service_pool.cc:224] Handling call
0709 13:15:13.939052 (+  2229us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:15:13.939195 (+   143us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:15:14.031132 (+ 91937us) tablet_metadata.cc:665] Metadata flushed
0709 13:15:14.145568 (+114436us) tablet_replica.cc:206] Creating consensus instance
0709 13:15:14.146066 (+   498us) inbound_call.cc:177] Queueing success response
0709 13:15:23.272902 (+9126836us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:15:24.277633 (+1004731us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:15:24.278212 (+   579us) tablet_replica.cc:261] Starting instrumentation
0709 13:15:24.278738 (+   526us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":514694,"raft.queue_time_us":118319,"tablet-open.queue_time_us":9126922}
I20260709 13:15:24.807950  4518 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:24.808478  5286 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.812116  5260 raft_consensus.cc:515] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.804381  5256 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:24.821230  5255 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.816769  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:24.824630  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:24.808853  5258 raft_consensus.cc:2804] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.822513  4518 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:24.832191  5271 raft_consensus.cc:2804] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.832896  5271 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.833444  5271 raft_consensus.cc:3060] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.834995  5259 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.825814  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.835682  5259 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.837010  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.825644  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:24.823352  5286 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.838613  4518 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
W20260709 13:15:24.843578  4667 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.119s	user 0.014s	sys 0.023s
W20260709 13:15:24.845775  4667 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.128s	user 0.015s	sys 0.024s
I20260709 13:15:24.826068  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:24.810637  5257 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.829424  5281 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.830654  5258 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.830577  4770 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:24.846632  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:24.847299  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 45701us
I20260709 13:15:24.851107  4694 raft_consensus.cc:2468] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:24.853204  4776 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:24.861754  5281 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:24.866703  5258 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.894407  5281 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.873697  4775 raft_consensus.cc:2468] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:24.875190  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:24.841289  5255 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.896679  4548 raft_consensus.cc:2468] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:24.908907  5280 raft_consensus.cc:493] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.909632  5280 raft_consensus.cc:515] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:24.917328  5280 leader_election.cc:290] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.861306  4700 raft_consensus.cc:2468] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:24.878625  4519 leader_election.cc:304] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:24.938175  5275 raft_consensus.cc:2804] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.938728  5275 raft_consensus.cc:493] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:24.939257  5275 raft_consensus.cc:3060] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:24.946025  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 67520.9us
I20260709 13:15:24.952278  4519 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:24.887444  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:24.888893  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:24.890573  5257 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:24.876534  5283 raft_consensus.cc:2804] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:24.952442  5270 consensus_queue.cc:1048] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:24.965099  5283 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:24.965780  5283 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:24.966352  5270 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.967141  5270 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.958572  4666 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:24.972680  5257 raft_consensus.cc:493] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.977118  5270 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:24.981896  5257 raft_consensus.cc:515] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:24.986873  5175 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:24.990126  4519 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:24.998957  4519 leader_election.cc:304] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:24.992465  5257 leader_election.cc:290] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.002998  5257 raft_consensus.cc:2804] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:25.003567  5257 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.004033  5257 raft_consensus.cc:3060] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:24.994014  5286 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:24.997366  5163 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:24.997944  4824 tablet_bootstrap.cc:492] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:25.011813  4824 ts_tablet_manager.cc:1403] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.386s	user 0.012s	sys 0.019s
I20260709 13:15:25.015082  5163 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.015859  5271 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.018306  5163 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.018996  5163 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:25.019551  5163 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:24.885602  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.020577  4771 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:25.000298  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.023407  4824 raft_consensus.cc:359] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.024246  4824 raft_consensus.cc:385] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:25.024595  4824 raft_consensus.cc:740] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:25.024930  5280 raft_consensus.cc:2804] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:25.025384  5280 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:25.025830  5280 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:25.027262  5163 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.027932  5163 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:25.032713  5057 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.035147  5271 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:24.684568  5065 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:25.017998  5256 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.036528  5065 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.039283  5065 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:24.990881  5270 raft_consensus.cc:2804] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.039669  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:25.039961  5270 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:25.040428  4551 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:25.041649  5249 leader_election.cc:290] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:25.044518  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.044895  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.045487  4548 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 3.
I20260709 13:15:25.045823  4549 raft_consensus.cc:2393] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:25.040530  5270 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.047446  4666 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:25.049335  5065 consensus_queue.cc:1048] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:25.052323  5065 raft_consensus.cc:2804] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:25.054046  5065 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:25.053083  4824 consensus_queue.cc:260] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.054610  5065 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:25.052455  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.056067  4774 raft_consensus.cc:2468] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:25.057868  5286 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.004411  5283 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.055685  4824 ts_tablet_manager.cc:1434] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.043s	user 0.003s	sys 0.003s
I20260709 13:15:25.028491  5163 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.064164  5270 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.146209  5270 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.069509  4824 tablet_bootstrap.cc:492] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:25.075166  5264 raft_consensus.cc:493] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
W20260709 13:15:25.124136  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 125310us
I20260709 13:15:25.161849  5264 raft_consensus.cc:515] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.164181  5264 leader_election.cc:290] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.069980  5283 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.165210  4519 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:25.169476  5283 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:25.169876  5283 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:25.170300  5283 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:25.170083  5257 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.171797  5293 raft_consensus.cc:493] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.172633  5257 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.173688  4519 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:25.138520  5163 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.166337  4518 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:25.167337  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.168267  4824 tablet_bootstrap.cc:654] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:25.174835  5291 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.175160  4823 tablet_bootstrap.cc:700] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc: Time spent opening tablet: real 0.112s	user 0.002s	sys 0.001s
I20260709 13:15:25.175951  4823 tablet_bootstrap.cc:654] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:25.175614  5291 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.177965  5257 raft_consensus.cc:493] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.178651  5257 raft_consensus.cc:515] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.179334  4551 raft_consensus.cc:2468] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:25.181203  5257 leader_election.cc:290] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.184872  5256 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:25.187477  4824 tablet_bootstrap.cc:492] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:25.188371  4824 ts_tablet_manager.cc:1403] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.119s	user 0.009s	sys 0.007s
I20260709 13:15:25.188972  5271 raft_consensus.cc:2804] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.189457  5271 raft_consensus.cc:493] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.189801  5271 raft_consensus.cc:3060] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.191365  4824 raft_consensus.cc:359] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.192107  4824 raft_consensus.cc:385] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:25.192394  4824 raft_consensus.cc:740] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:25.024430  5258 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.195247  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.195952  4694 raft_consensus.cc:3060] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:25.196177  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.197456  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beeb8c8bf7343c297daf017c542c67b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.198007  4693 raft_consensus.cc:3060] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:25.198462  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.172627  5293 raft_consensus.cc:515] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.200049  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.201066  5293 leader_election.cc:290] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.205330  5291 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:25.205602  4694 raft_consensus.cc:2468] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:25.196063  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beeb8c8bf7343c297daf017c542c67b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.209638  5270 raft_consensus.cc:2804] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.210253  5270 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:25.210749  5270 raft_consensus.cc:3060] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.211190  4740 leader_election.cc:304] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:24.931782  5281 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.161306  5275 raft_consensus.cc:515] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.214959  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.224674  4696 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.225135  5270 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.226395  5258 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.227487  5270 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.215648  5260 leader_election.cc:290] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.216022  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86791c9adcc64babb9560e224fd9abd9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.232074  4699 raft_consensus.cc:2468] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:25.215137  5291 raft_consensus.cc:2804] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:25.216368  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.233461  5291 raft_consensus.cc:697] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:25.233592  4697 raft_consensus.cc:3060] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.234417  5291 consensus_queue.cc:237] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.216981  5264 raft_consensus.cc:2804] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.237442  5264 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:25.237901  5264 raft_consensus.cc:3060] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.225304  5065 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.218111  5259 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.239770  5271 raft_consensus.cc:515] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.241041  5065 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:25.222491  5275 leader_election.cc:290] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.213136  4740 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:25.219691  4824 consensus_queue.cc:260] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:25.217330  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 51084.6us
I20260709 13:15:25.245129  4824 ts_tablet_manager.cc:1434] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 13:15:25.242306  4549 raft_consensus.cc:1240] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:25.240955  4698 raft_consensus.cc:1240] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:25.212698  4693 raft_consensus.cc:2468] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:25.243880  5256 raft_consensus.cc:493] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.222855  5257 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:25.230720  5280 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.249487  4824 tablet_bootstrap.cc:492] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a: Bootstrap starting.
I20260709 13:15:25.249648  5295 raft_consensus.cc:2804] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:25.255961  5295 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.250396  5065 raft_consensus.cc:493] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.256431  5295 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:25.256984  5065 raft_consensus.cc:515] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.259940  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.250327  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.249048  5256 raft_consensus.cc:515] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.250059  5257 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.230741  5283 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.267006  5283 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.219617  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.271546  5065 leader_election.cc:290] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.248561  5281 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.279155  4823 tablet_bootstrap.cc:492] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:25.277529  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beeb8c8bf7343c297daf017c542c67b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.275653  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.283792  4823 ts_tablet_manager.cc:1403] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.603s	user 0.020s	sys 0.002s
I20260709 13:15:25.283965  4775 raft_consensus.cc:2393] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:25.281682  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.274981  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.276754  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.272549  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.285409  4771 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:25.285713  4700 raft_consensus.cc:2468] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:25.277298  4741 leader_election.cc:304] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:25.240955  4696 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:25.288013  4824 tablet_bootstrap.cc:654] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:25.252285  5280 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.276391  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.276808  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.285411  4769 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:25.303318  4768 raft_consensus.cc:2468] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:25.290449  5260 raft_consensus.cc:2804] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:25.276286  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.291361  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.292925  4741 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:25.286697  4697 raft_consensus.cc:2468] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:25.275038  5295 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.297896  5264 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.299126  5065 consensus_queue.cc:1048] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:25.307104  5175 raft_consensus.cc:2804] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.310353  5175 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:25.310822  5175 raft_consensus.cc:3060] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.312217  5291 raft_consensus.cc:493] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.300264  5258 consensus_queue.cc:1048] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:25.301776  5163 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.312719  5291 raft_consensus.cc:515] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.313063  5163 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.314805  5295 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.314859  5291 leader_election.cc:290] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.305233  5260 raft_consensus.cc:697] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:25.315265  5163 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.316008  5260 consensus_queue.cc:237] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.301182  4668 leader_election.cc:304] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:25.320976  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0c73f991a42e4817d8916fa0a7c30" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.321316  5258 raft_consensus.cc:2749] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.250327  5271 leader_election.cc:290] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.321827  4697 raft_consensus.cc:2468] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:25.305006  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.310858  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.323648  5264 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.302268  4823 raft_consensus.cc:359] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.324767  4823 raft_consensus.cc:385] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:25.325117  4823 raft_consensus.cc:740] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:25.325804  4823 consensus_queue.cc:260] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.327853  5175 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.328354  4823 ts_tablet_manager.cc:1434] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.044s	user 0.002s	sys 0.002s
I20260709 13:15:25.330101  4823 tablet_bootstrap.cc:492] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:25.330673  5175 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.331368  4824 tablet_bootstrap.cc:492] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a: No bootstrap required, opened a new log
I20260709 13:15:25.332083  4824 ts_tablet_manager.cc:1403] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.002s
W20260709 13:15:25.220012  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 46416.5us
I20260709 13:15:25.380651  4518 leader_election.cc:304] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
W20260709 13:15:25.394670  4668 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 72710.5us
W20260709 13:15:25.402473  4668 leader_election.cc:343] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:25.405989  4741 leader_election.cc:304] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:25.398437  5281 raft_consensus.cc:493] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.409539  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.409705  5281 raft_consensus.cc:515] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.411537  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.412443  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86791c9adcc64babb9560e224fd9abd9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.413071  4695 raft_consensus.cc:2393] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 2: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:25.408056  5175 raft_consensus.cc:2804] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:25.414741  5175 raft_consensus.cc:493] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.417541  5175 raft_consensus.cc:3060] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:25.411656  4768 raft_consensus.cc:2393] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:25.414103  5280 raft_consensus.cc:2804] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.421919  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.423156  4702 raft_consensus.cc:3060] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:25.424654  4668 leader_election.cc:343] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:25.421945  5280 raft_consensus.cc:493] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:25.421989  5258 raft_consensus.cc:493] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.427778  4668 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:25.427677  5280 raft_consensus.cc:3060] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.428925  5258 raft_consensus.cc:515] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.438395  5255 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.441874  5264 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.426312  5281 leader_election.cc:290] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.458164  4988 raft_consensus.cc:2804] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.458804  4988 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:25.459311  4988 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.461688  5279 leader_election.cc:290] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.454468  5248 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.453241  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.465518  4668 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:25.467298  4668 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:25.468855  4898 raft_consensus.cc:2804] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.469388  4898 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:25.469817  4898 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.470204  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.471298  4549 raft_consensus.cc:2393] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:25.471933  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.473618  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.474187  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.474264  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.474619  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.475557  4774 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.475601  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.479691  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.481191  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.480796  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.482300  4767 raft_consensus.cc:3060] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.452535  5258 leader_election.cc:290] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.482725  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.483714  4549 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:25.483349  4822 raft_consensus.cc:359] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.484387  4822 raft_consensus.cc:385] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:25.484724  4822 raft_consensus.cc:740] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:25.485704  4822 consensus_queue.cc:260] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.494014  5256 leader_election.cc:290] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:25.480927  4552 raft_consensus.cc:2393] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 4: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:25.499162  4823 tablet_bootstrap.cc:654] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:25.485960  4518 leader_election.cc:304] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:25.500463  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.505158  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.505425  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:25.506078  4776 raft_consensus.cc:3060] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.506368  4546 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:25.506567  5283 raft_consensus.cc:2804] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:25.507097  5283 raft_consensus.cc:493] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.507483  5283 raft_consensus.cc:3060] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:25.506079  4518 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:25.514165  5279 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.514815  5279 raft_consensus.cc:697] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:25.516788  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.520339  4821 tablet_bootstrap.cc:492] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:25.500226  4668 leader_election.cc:304] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:25.525986  4668 leader_election.cc:343] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:25.527014  5216 raft_consensus.cc:493] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:25.529392  5285 raft_consensus.cc:493] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:25.530241  5285 raft_consensus.cc:515] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.539914  5285 leader_election.cc:290] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.539794  4740 leader_election.cc:304] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:25.541224  5299 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:25.542281  5299 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.545360  5075 consensus_queue.cc:1048] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:25.553256  5151 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:25.553678  5312 raft_consensus.cc:493] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:25.554392  5151 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.555094  5314 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:25.555994  5314 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.557030  5300 raft_consensus.cc:493] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:25.558221  5054 raft_consensus.cc:2749] T d356c2066af241e4a89fad36578a7f89 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.558722  5314 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:25.559458  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.560048  5272 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:25.560752  5272 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.516185  5279 consensus_queue.cc:237] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.563273  5272 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.563707  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.554557  4822 ts_tablet_manager.cc:1434] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 1.271s	user 0.010s	sys 0.011s
I20260709 13:15:25.528028  5216 raft_consensus.cc:515] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.565541  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.567163  5216 leader_election.cc:290] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
W20260709 13:15:25.564711  4822 ts_tablet_manager.cc:1467] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82: Tablet startup took 1281ms
I20260709 13:15:25.569696  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.570634  4624 raft_consensus.cc:2468] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:25.571806  5151 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.572204  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.554551  5312 raft_consensus.cc:515] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.581725  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.581868  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.582718  4775 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:25.583601  5285 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:25.560403  5177 raft_consensus.cc:493] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:25.506560  5175 raft_consensus.cc:515] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.585289  5177 raft_consensus.cc:515] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.455237  5264 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.589380  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.586107  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.590049  5314 raft_consensus.cc:493] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.590703  5314 raft_consensus.cc:515] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.591601  5177 leader_election.cc:290] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:25.584511  5285 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:25.568044  4822 ts_tablet_manager.cc:1469] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82: Trace:
0709 13:15:13.731098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.731249 (+   151us) service_pool.cc:224] Handling call
0709 13:15:13.733381 (+  2132us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:15:13.733483 (+   102us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:15:13.841007 (+107524us) tablet_metadata.cc:665] Metadata flushed
0709 13:15:13.861346 (+ 20339us) tablet_replica.cc:206] Creating consensus instance
0709 13:15:13.900715 (+ 39369us) inbound_call.cc:177] Queueing success response
0709 13:15:23.241173 (+9340458us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:15:24.283457 (+1042284us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:15:24.285466 (+  2009us) tablet_replica.cc:261] Starting instrumentation
0709 13:15:24.286072 (+   606us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":1201152,"raft.queue_time_us":77122,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":179,"tablet-open.queue_time_us":9341732}
I20260709 13:15:25.557343  4740 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:25.592859  5256 raft_consensus.cc:2749] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:25.593508  4822 tablet_bootstrap.cc:492] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:25.594630  5054 raft_consensus.cc:2804] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.586974  5212 raft_consensus.cc:493] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:25.587218  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.583760  5312 leader_election.cc:290] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.557920  5300 raft_consensus.cc:515] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.594959  4666 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:25.573539  5299 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.598654  5300 leader_election.cc:290] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.595129  5054 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.599517  5054 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.600539  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.602717  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.589524  5175 leader_election.cc:290] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.603823  4546 raft_consensus.cc:2468] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:25.604115  5065 raft_consensus.cc:2804] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:25.604662  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.605135  4824 raft_consensus.cc:359] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.613713  4824 raft_consensus.cc:385] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:25.614063  4824 raft_consensus.cc:740] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Initialized, Role: FOLLOWER
I20260709 13:15:25.615165  4824 consensus_queue.cc:260] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.607667  4988 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.612368  5065 raft_consensus.cc:697] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:25.618395  4898 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:25.626744  4519 leader_election.cc:343] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:25.599565  5212 raft_consensus.cc:515] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.584407  5283 raft_consensus.cc:515] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.641369  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.642176  4519 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:25.627017  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.630007  5285 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.639303  5314 leader_election.cc:290] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.651666  5258 raft_consensus.cc:493] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.664778  5271 raft_consensus.cc:2749] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:25.660068  4698 raft_consensus.cc:2393] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 2: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
W20260709 13:15:25.666772  4519 leader_election.cc:343] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:25.667438  4898 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.654230  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.665663  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0c73f991a42e4817d8916fa0a7c30" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.676826  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0059f848a34f47b5a3fe29efe38914b3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.677382  4700 raft_consensus.cc:3060] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:25.678701  4449 catalog_manager.cc:5697] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:25.681250  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.676905  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.682045  4625 raft_consensus.cc:2468] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:25.686475  4774 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:25.673872  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.676085  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.686911  5264 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.670113  5258 raft_consensus.cc:515] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.661604  4775 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:25.673420  4824 ts_tablet_manager.cc:1434] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a: Time spent starting tablet: real 0.340s	user 0.005s	sys 0.006s
I20260709 13:15:25.693759  4519 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:25.695017  4988 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.689190  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:25.701522  4797 debug-util.cc:398] Leaking SignalData structure 0x7b080023b120 after lost signal to thread 26962
I20260709 13:15:25.701860  4624 raft_consensus.cc:2468] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:25.702420  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
W20260709 13:15:25.703030  4797 debug-util.cc:398] Leaking SignalData structure 0x7b08001b3000 after lost signal to thread 4485
I20260709 13:15:25.701612  4769 raft_consensus.cc:1240] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.703943  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:25.704988  4797 debug-util.cc:398] Leaking SignalData structure 0x7b08001265a0 after lost signal to thread 4577
I20260709 13:15:25.704918  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beeb8c8bf7343c297daf017c542c67b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.705751  4622 raft_consensus.cc:2468] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:25.706087  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.704442  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:25.706712  4797 debug-util.cc:398] Leaking SignalData structure 0x7b080034afc0 after lost signal to thread 4652
W20260709 13:15:25.707072  4797 debug-util.cc:398] Leaking SignalData structure 0x7b08003f7140 after lost signal to thread 4653
I20260709 13:15:25.689167  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.690264  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.709167  4448 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42678 (request call id 49) took 2635 ms. Trace:
I20260709 13:15:25.716265  4767 raft_consensus.cc:2468] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:25.715109  5175 consensus_queue.cc:1059] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:15:25.719394  4449 catalog_manager.cc:5697] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.718469  4821 tablet_bootstrap.cc:654] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:25.709609  4448 rpcz_store.cc:276] 0709 13:15:23.064067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.064230 (+   163us) service_pool.cc:224] Handling call
0709 13:15:25.696631 (+2632401us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:24.680593 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:24.681367 (+   774us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:24.681425 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:15:24.681448 (+    23us) tablet.cc:662] Decoding operations
0709 13:15:24.681932 (+   484us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:24.682029 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:24.682062 (+    33us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:15:24.682434 (+   372us) tablet.cc:701] Row locks acquired
0709 13:15:24.682455 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:15:24.682623 (+   168us) write_op.cc:270] Start()
0709 13:15:24.682760 (+   137us) write_op.cc:276] Timestamp: P: 1783602924682575 usec, L: 0
0709 13:15:24.682790 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:15:24.704200 (+ 21410us) log.cc:844] Serialized 928 byte log entry
0709 13:15:25.065849 (+361649us) op_driver.cc:464] REPLICATION: finished
0709 13:15:25.193628 (+127779us) write_op.cc:301] APPLY: starting
0709 13:15:25.193717 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:25.193837 (+   120us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:25.193962 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:25.194341 (+   379us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:25.194499 (+   158us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:15:25.194545 (+    46us) write_op.cc:312] APPLY: finished
0709 13:15:25.194729 (+   184us) log.cc:844] Serialized 28 byte log entry
0709 13:15:25.194903 (+   174us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:25.194989 (+    86us) write_op.cc:454] Released schema lock
0709 13:15:25.195030 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:25.228662 (+ 33632us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:25.228692 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":127533,"apply.run_cpu_time_us":1800,"apply.run_wall_time_us":35019,"num_ops":2,"prepare.queue_time_us":136422,"prepare.run_cpu_time_us":66376,"prepare.run_wall_time_us":149634,"raft.queue_time_us":231251,"raft.run_cpu_time_us":7032,"raft.run_wall_time_us":143364,"replication_time_us":382898,"thread_start_us":721397,"threads_started":4,"wal-append.queue_time_us":127547}]]}
I20260709 13:15:25.724082  4771 raft_consensus.cc:1240] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.726179  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.726470  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.727345  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:25.729030  5258 leader_election.cc:290] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.731694  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:25.734189  4552 raft_consensus.cc:1240] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.734541  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:25.735157  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.736070  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:25.736027  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0c73f991a42e4817d8916fa0a7c30" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.737532  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.739418  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.740058  4823 tablet_bootstrap.cc:492] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:25.741365  4740 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
W20260709 13:15:25.743762  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 70491.7us
I20260709 13:15:25.744011  5264 raft_consensus.cc:2749] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:15:25.759868  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 117293us
I20260709 13:15:25.696074  5280 raft_consensus.cc:515] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.745316  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.746170  5283 leader_election.cc:290] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:25.710453  5212 leader_election.cc:290] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:25.747936  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:25.749397  4797 debug-util.cc:398] Leaking SignalData structure 0x7b08003ef460 after lost signal to thread 4726
I20260709 13:15:25.775657  5280 leader_election.cc:290] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
W20260709 13:15:25.777220  4797 debug-util.cc:398] Leaking SignalData structure 0x7b08000bc8e0 after lost signal to thread 4800
I20260709 13:15:25.777393  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.778422  4627 raft_consensus.cc:2468] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:25.778999  4519 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:25.749475  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.780655  5264 raft_consensus.cc:2804] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:25.781284  5264 raft_consensus.cc:697] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:25.749820  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.750803  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.728664  5054 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.782539  5264 consensus_queue.cc:237] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.783494  4694 raft_consensus.cc:2468] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:25.785113  5054 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:25.785305  4769 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:25.753598  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.784855  4449 catalog_manager.cc:5697] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:25.785943  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 44745us
I20260709 13:15:25.786788  4770 raft_consensus.cc:1240] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:25.688668  4702 raft_consensus.cc:2468] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:25.718549  4776 raft_consensus.cc:2468] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:25.759495  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.801441  4768 raft_consensus.cc:1240] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:25.802724  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.803140  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0c73f991a42e4817d8916fa0a7c30" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:25.803581  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.803938  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.802783  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0c53efa0ee4dfbba4c3f1922027f50" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.803535  4621 raft_consensus.cc:2468] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:25.805305  4623 raft_consensus.cc:3060] T 6beeb8c8bf7343c297daf017c542c67b P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:25.708127  4988 consensus_queue.cc:1059] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.057s
I20260709 13:15:25.773627  4547 raft_consensus.cc:2468] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:25.766295  4546 raft_consensus.cc:3060] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:25.768482  4626 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:25.774577  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:25.773666  4551 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.789196  5259 raft_consensus.cc:2804] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:25.793457  4545 raft_consensus.cc:1240] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:25.793594  4553 raft_consensus.cc:1240] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.794026  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.794962  4823 ts_tablet_manager.cc:1403] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.465s	user 0.013s	sys 0.020s
I20260709 13:15:25.795500  4822 tablet_bootstrap.cc:654] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:25.795776  4549 raft_consensus.cc:1240] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.795838  4548 raft_consensus.cc:2468] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 0.
I20260709 13:15:25.796250  5318 consensus_queue.cc:1059] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:25.827342  5318 raft_consensus.cc:3055] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.827804  5318 raft_consensus.cc:740] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:25.829542  4823 raft_consensus.cc:359] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.830530  4823 raft_consensus.cc:385] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:25.830842  4823 raft_consensus.cc:740] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:25.834187  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.836192  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.837513  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:25.769322  4668 leader_election.cc:343] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:25.796249  4700 raft_consensus.cc:2468] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:25.816401  5293 raft_consensus.cc:2764] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:15:25.817540  4544 raft_consensus.cc:1240] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.818114  4549 raft_consensus.cc:1240] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.820286  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.820874  5259 raft_consensus.cc:697] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:25.821053  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:25.822680  4771 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.858119  4545 raft_consensus.cc:1240] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.823107  4548 raft_consensus.cc:1240] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.858857  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.859669  4552 raft_consensus.cc:1240] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:25.857738  5255 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:25.868055  5255 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:25.861869  4767 raft_consensus.cc:1240] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.789960  5065 consensus_queue.cc:237] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.862984  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0059f848a34f47b5a3fe29efe38914b3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.870083  4519 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:25.867211  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86791c9adcc64babb9560e224fd9abd9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.867134  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.868484  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.831688  4823 consensus_queue.cc:260] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.853654  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:25.849853  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.837617  5257 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:25.865948  4776 raft_consensus.cc:1240] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:25.896155  4898 raft_consensus.cc:3055] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.874706  4518 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:25.896654  4898 raft_consensus.cc:740] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
W20260709 13:15:25.850689  4668 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 83775us
I20260709 13:15:25.880043  4519 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:25.854537  5285 consensus_queue.cc:1059] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:15:25.877513  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.890754  5281 raft_consensus.cc:2804] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:25.880946  4741 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:25.890435  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:25.906136  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.906840  5075 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:25.908138  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 69673.8us
I20260709 13:15:25.922819  4898 consensus_queue.cc:260] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.923170  5075 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:25.924336  4898 raft_consensus.cc:3060] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.924966  5281 raft_consensus.cc:697] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:25.925916  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.926350  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.927282  4625 raft_consensus.cc:2468] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:25.929183  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:25.928845  5281 consensus_queue.cc:237] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.932554  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.933228  4693 raft_consensus.cc:1240] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.933337  4697 raft_consensus.cc:1240] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.935070  4694 raft_consensus.cc:1240] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.935945  4702 raft_consensus.cc:1240] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.935073  4698 raft_consensus.cc:1240] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.937764  5314 raft_consensus.cc:3055] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.938297  5314 raft_consensus.cc:740] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:25.939679  5314 consensus_queue.cc:260] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.940820  5318 consensus_queue.cc:260] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.941762  5314 raft_consensus.cc:3060] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.910877  5255 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.898386  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:25.944695  4666 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:25.945007  4668 leader_election.cc:343] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:25.945971  5322 raft_consensus.cc:2749] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.946622  5318 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.946377  5258 consensus_queue.cc:1059] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:25.947795  4988 raft_consensus.cc:3055] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:25.948201  4988 raft_consensus.cc:740] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:25.908500  4546 raft_consensus.cc:2468] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:25.949246  4988 consensus_queue.cc:260] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.950605  4988 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.916249  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.904352  5280 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:25.960407  5280 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.960881  5280 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.961344  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.961673  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.963037  5254 raft_consensus.cc:3055] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:25.916360  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:25.963587  5254 raft_consensus.cc:740] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:25.963847  5075 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:25.964819  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0c53efa0ee4dfbba4c3f1922027f50" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.873231  5259 consensus_queue.cc:237] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:25.965027  5254 consensus_queue.cc:260] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:25.966225  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.919486  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:25.968752  4552 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:25.969867  4666 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:25.970821  5257 raft_consensus.cc:2749] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:25.971417  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.962905  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.917331  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.921141  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:25.973589  4666 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:25.986379  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 115415us
I20260709 13:15:25.975242  4592 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:25.977092  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.978127  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.979249  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:25.984885  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:25.967293  5254 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:25.992249  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 142209us
I20260709 13:15:25.992705  4519 leader_election.cc:304] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:25.995779  5283 raft_consensus.cc:2804] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:26.000484  4551 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:26.002895  4700 raft_consensus.cc:1240] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:25.994239  5054 raft_consensus.cc:2804] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:26.005661  5054 raft_consensus.cc:697] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:26.006786  4822 tablet_bootstrap.cc:492] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:26.006762  5054 consensus_queue.cc:237] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:26.008090  4822 ts_tablet_manager.cc:1403] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.415s	user 0.015s	sys 0.019s
W20260709 13:15:26.012295  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 111064us
I20260709 13:15:26.014691  4823 ts_tablet_manager.cc:1434] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.196s	user 0.008s	sys 0.004s
I20260709 13:15:25.987331  4771 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:26.018193  4519 leader_election.cc:304] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:26.024138  5270 raft_consensus.cc:2804] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:26.026932  4821 tablet_bootstrap.cc:492] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:26.027714  4821 ts_tablet_manager.cc:1403] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.508s	user 0.008s	sys 0.011s
I20260709 13:15:26.014282  5283 raft_consensus.cc:697] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:26.033469  5283 consensus_queue.cc:237] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.035960  4547 raft_consensus.cc:1240] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.036922  4898 raft_consensus.cc:2804] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:26.038635  4898 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:26.039070  4898 raft_consensus.cc:3060] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.021263  5075 raft_consensus.cc:2749] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:26.002933  4552 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:26.048305  4553 raft_consensus.cc:1240] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
W20260709 13:15:26.050364  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 63031.2us
W20260709 13:15:26.051244  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 76284.5us
I20260709 13:15:26.052666  4592 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.052999  4518 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.059540  4822 raft_consensus.cc:359] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.061849  4822 raft_consensus.cc:385] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:26.062162  4822 raft_consensus.cc:740] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:26.057767  4741 leader_election.cc:304] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:25.997373  5280 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:26.078074  4898 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.066587  4822 consensus_queue.cc:260] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.083678  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:26.085614  4518 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.086773  4823 tablet_bootstrap.cc:492] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:26.092442  4898 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:26.048892  5259 consensus_queue.cc:1059] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:15:26.064414  5075 raft_consensus.cc:2749] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.073856  5212 consensus_queue.cc:1048] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
W20260709 13:15:26.100278  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 126818us
I20260709 13:15:26.073107  5275 raft_consensus.cc:2749] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.079350  5255 raft_consensus.cc:2804] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:26.080614  5271 raft_consensus.cc:2804] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:26.108991  5275 raft_consensus.cc:2749] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.061326  5248 consensus_queue.cc:1048] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20260709 13:15:26.081082  5175 consensus_queue.cc:1059] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:15:26.073617  5317 consensus_queue.cc:1059] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:15:26.126019  4518 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.112753  5314 consensus_queue.cc:1059] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:15:26.119362  5270 raft_consensus.cc:697] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:26.128268  4821 raft_consensus.cc:359] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:26.134366  4821 raft_consensus.cc:385] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:26.134714  4821 raft_consensus.cc:740] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:26.136637  4821 consensus_queue.cc:260] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:26.119951  5285 consensus_queue.cc:1059] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
W20260709 13:15:26.143412  5323 log.cc:927] Time spent T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:15:26.144018  5249 consensus_queue.cc:1048] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:15:26.152050  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.153016  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.153972  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.154452  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.154755  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.153151  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.154055  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.167542  4822 ts_tablet_manager.cc:1434] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.159s	user 0.007s	sys 0.004s
I20260709 13:15:26.163800  4593 leader_election.cc:304] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.175745  5315 consensus_queue.cc:1059] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:15:26.176769  4627 raft_consensus.cc:2468] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:26.177199  5212 raft_consensus.cc:3055] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:15:26.188493  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 98615.1us
I20260709 13:15:26.192955  5216 consensus_queue.cc:1059] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:15:26.193914  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 64271.6us
W20260709 13:15:26.191742  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 110840us
W20260709 13:15:26.190918  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 69235.6us
I20260709 13:15:26.194114  5279 raft_consensus.cc:2749] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.199373  4822 tablet_bootstrap.cc:492] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
W20260709 13:15:26.199746  5328 log.cc:927] Time spent T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:15:26.200033  4821 ts_tablet_manager.cc:1434] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.172s	user 0.012s	sys 0.001s
I20260709 13:15:26.201190  4821 tablet_bootstrap.cc:492] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:26.202332  5270 consensus_queue.cc:237] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:26.207669  4518 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.187489  5212 raft_consensus.cc:740] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:26.191421  5285 raft_consensus.cc:2749] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.215520  4740 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.210841  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.226028  4518 leader_election.cc:304] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:26.196624  5317 consensus_queue.cc:1059] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:15:26.215042  5280 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:26.227264  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.212321  5175 consensus_queue.cc:1059] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:15:26.230275  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.211334  5255 raft_consensus.cc:697] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
W20260709 13:15:26.230728  4666 leader_election.cc:343] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.245265  5075 raft_consensus.cc:3055] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.245779  4666 leader_election.cc:304] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:26.233157  5338 raft_consensus.cc:2749] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:26.234321  5212 consensus_queue.cc:260] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:26.238950  5076 raft_consensus.cc:3055] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:26.252344  4898 raft_consensus.cc:2804] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:26.252861  4898 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:26.253261  4898 raft_consensus.cc:3060] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.239032  5315 consensus_queue.cc:1059] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:26.240273  5314 raft_consensus.cc:3055] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.251752  5076 raft_consensus.cc:740] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:26.231051  5279 raft_consensus.cc:2804] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:26.257484  5314 raft_consensus.cc:740] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:26.257753  5279 raft_consensus.cc:493] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.258246  5279 raft_consensus.cc:3060] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:26.258662  5314 consensus_queue.cc:260] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.260109  5314 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:26.230734  5331 log.cc:927] Time spent T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:15:26.262107  4822 tablet_bootstrap.cc:654] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:26.266541  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 179169us
W20260709 13:15:26.252481  5329 log.cc:927] Time spent T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:15:26.245819  5075 raft_consensus.cc:740] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:26.222262  5216 consensus_queue.cc:1059] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:15:26.256287  4593 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.242658  5317 raft_consensus.cc:3055] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:15:26.221657  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 111486us
I20260709 13:15:26.228292  5271 raft_consensus.cc:697] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:26.257501  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:26.283021  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 194109us
W20260709 13:15:26.288128  4519 leader_election.cc:343] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.280822  5212 raft_consensus.cc:3060] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.278259  5317 raft_consensus.cc:740] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
W20260709 13:15:26.301884  4740 leader_election.cc:343] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.274189  5248 consensus_queue.cc:1059] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:15:26.329573  4697 raft_consensus.cc:1240] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.345655  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86791c9adcc64babb9560e224fd9abd9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:26.329672  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:26.349123  5271 consensus_queue.cc:237] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:26.350482  5076 consensus_queue.cc:260] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:26.352160  5076 raft_consensus.cc:3060] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:26.353648  5337 raft_consensus.cc:2804] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:26.354249  5335 raft_consensus.cc:3055] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.354683  5335 raft_consensus.cc:740] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:26.349143  4694 raft_consensus.cc:3060] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:26.355698  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:26.317492  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
W20260709 13:15:26.353298  4667 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.358182  5249 raft_consensus.cc:3055] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.355947  5339 raft_consensus.cc:3055] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.380127  5339 raft_consensus.cc:740] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:26.381289  5339 consensus_queue.cc:260] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:26.302385  5285 consensus_queue.cc:1059] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:15:26.358795  5249 raft_consensus.cc:740] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
W20260709 13:15:26.382309  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 79635.8us
I20260709 13:15:26.395753  5275 raft_consensus.cc:493] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.396492  5275 raft_consensus.cc:515] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:26.409634  4898 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:26.423811  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:26.423621  5255 consensus_queue.cc:237] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:26.424451  4627 raft_consensus.cc:2376] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1. Re-sending same reply.
W20260709 13:15:26.425256  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 123576us
W20260709 13:15:26.426055  4666 leader_election.cc:343] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:26.426970  4666 leader_election.cc:343] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.427352  4666 leader_election.cc:304] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.428580  4898 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
W20260709 13:15:26.431608  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 175379us
I20260709 13:15:26.438141  4593 leader_election.cc:304] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.441430  4593 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.441839  4740 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.442936  4593 leader_election.cc:304] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.443415  4740 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.444234  5343 raft_consensus.cc:2804] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:26.444677  5343 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:26.445051  5343 raft_consensus.cc:3060] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:26.445766  4620 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:26.446059  5332 consensus_queue.cc:1059] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:26.446688  5216 raft_consensus.cc:2749] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.447225  5332 raft_consensus.cc:3055] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.447374  5216 raft_consensus.cc:2749] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:26.447647  5332 raft_consensus.cc:740] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:26.448091  5216 raft_consensus.cc:2749] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.452461  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.453858  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:26.454921  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:26.455350  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.456782  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:26.344094  4594 leader_election.cc:304] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.457237  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:26.457736  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:26.458112  4625 raft_consensus.cc:2468] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
W20260709 13:15:26.458226  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 127273us
I20260709 13:15:26.459218  4594 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.458024  5216 raft_consensus.cc:2749] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.406545  5317 consensus_queue.cc:260] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:26.460139  5216 raft_consensus.cc:2749] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.405207  5337 raft_consensus.cc:697] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:26.412649  5338 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:26.408212  5175 consensus_queue.cc:1048] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.068s
I20260709 13:15:26.438331  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:26.442989  4821 tablet_bootstrap.cc:654] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:26.469954  5339 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.468937  4698 raft_consensus.cc:1240] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.501650  4823 tablet_bootstrap.cc:654] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:26.488052  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 61130.9us
I20260709 13:15:26.460717  5317 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.466688  5347 raft_consensus.cc:3055] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.472834  5285 raft_consensus.cc:2749] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:26.477026  5065 raft_consensus.cc:2749] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:26.445788  5336 raft_consensus.cc:2749] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.454958  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:26.520237  4518 leader_election.cc:304] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.528193  4666 leader_election.cc:304] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:26.527168  4740 leader_election.cc:304] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.518079  4695 raft_consensus.cc:1275] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:26.460618  4594 leader_election.cc:304] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.565387  4898 raft_consensus.cc:2764] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:15:26.479197  5279 raft_consensus.cc:515] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.465204  5338 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:26.471030  5075 consensus_queue.cc:260] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.567445  5352 raft_consensus.cc:2749] T cd411637f5c84d50b6c9ba5cae086a07 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.570178  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.571458  5280 raft_consensus.cc:2749] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.539649  5347 raft_consensus.cc:740] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:26.462273  5343 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:26.477800  5337 consensus_queue.cc:237] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:26.599749  5338 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:26.599787  5343 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
W20260709 13:15:26.591104  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 148221us
W20260709 13:15:26.593041  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 71432.9us
W20260709 13:15:26.601547  4666 leader_election.cc:343] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:26.603003  4666 leader_election.cc:343] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.603912  4623 raft_consensus.cc:2468] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:26.605901  4593 leader_election.cc:304] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.608848  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:26.464129  5335 consensus_queue.cc:260] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:26.611934  5076 raft_consensus.cc:2749] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.616973  4666 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.619732  5065 raft_consensus.cc:2749] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:26.621951  5065 consensus_queue.cc:1059] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:26.622884  4517 leader_election.cc:304] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.511659  5332 consensus_queue.cc:260] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.583030  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.591143  5348 consensus_queue.cc:1048] T 3fb194bf63a4451f8c4d28502acdad60 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260709 13:15:26.627071  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 112165us
W20260709 13:15:26.590662  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 127951us
I20260709 13:15:26.587350  5355 raft_consensus.cc:2749] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.631529  5280 consensus_queue.cc:1048] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
W20260709 13:15:26.636201  4518 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.644423  5338 consensus_queue.cc:1048] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:15:26.633296  4594 leader_election.cc:304] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.654489  5275 leader_election.cc:290] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:26.654763  4594 leader_election.cc:304] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.656324  5340 raft_consensus.cc:2749] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.657088  5340 raft_consensus.cc:2749] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.661309  5335 raft_consensus.cc:3060] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.663059  5347 consensus_queue.cc:260] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.669874  4628 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:26.671855  5332 raft_consensus.cc:3060] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.641851  5348 raft_consensus.cc:2749] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.646306  4694 raft_consensus.cc:2468] T 86791c9adcc64babb9560e224fd9abd9 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:26.675559  4740 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:26.693105  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 43820.1us
I20260709 13:15:26.694567  4740 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:26.696264  4740 leader_election.cc:343] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.636795  5279 leader_election.cc:290] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
W20260709 13:15:26.677230  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 49983us
I20260709 13:15:26.698791  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
W20260709 13:15:26.699378  4666 leader_election.cc:343] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.700089  4666 leader_election.cc:304] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.689543  5065 raft_consensus.cc:3055] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.703727  4620 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:26.704556  4517 leader_election.cc:304] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.702989  5366 raft_consensus.cc:2749] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:26.707798  4517 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.708933  5348 raft_consensus.cc:2749] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.709385  5271 raft_consensus.cc:2749] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:26.710579  4591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:26.710672  4517 leader_election.cc:343] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.702255  5175 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.711485  4517 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.711966  4771 raft_consensus.cc:1240] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:26.712536  5271 raft_consensus.cc:2749] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:26.718021  4517 leader_election.cc:304] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.719134  5271 raft_consensus.cc:2749] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.761986  4898 raft_consensus.cc:2749] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:26.761612  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:26.747732  4518 leader_election.cc:304] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:26.766752  4627 raft_consensus.cc:3060] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:26.753947  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 57736.3us
I20260709 13:15:26.748005  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:26.793377  4701 raft_consensus.cc:3060] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:26.764886  4591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.747486  4551 raft_consensus.cc:1240] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:26.800478  4740 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.745965  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:26.755230  5271 raft_consensus.cc:493] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.743532  5065 raft_consensus.cc:740] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:26.748672  4700 raft_consensus.cc:1275] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:26.796236  5279 raft_consensus.cc:2804] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:26.816212  4622 raft_consensus.cc:2468] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:26.745460  5249 consensus_queue.cc:260] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:26.748502  5285 consensus_queue.cc:1059] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:15:26.827406  4701 raft_consensus.cc:2468] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:26.839596  4518 leader_election.cc:304] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:26.832572  4553 raft_consensus.cc:2468] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
W20260709 13:15:26.851958  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 51610.9us
I20260709 13:15:26.855540  4701 raft_consensus.cc:1275] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:15:26.855957  4701 pending_rounds.cc:85] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a: Aborting all ops after (but not including) 0
I20260709 13:15:26.858899  4547 raft_consensus.cc:1240] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:26.862471  5175 raft_consensus.cc:2749] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.816746  5271 raft_consensus.cc:515] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:26.863986  4821 tablet_bootstrap.cc:492] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:26.856976  5279 raft_consensus.cc:697] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:26.865721  5271 leader_election.cc:290] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:26.866021  5279 consensus_queue.cc:237] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:26.866879  5065 consensus_queue.cc:260] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:26.868136  5065 raft_consensus.cc:3060] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.837765  5337 raft_consensus.cc:2749] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:26.865098  4624 raft_consensus.cc:1275] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:26.870339  4822 tablet_bootstrap.cc:492] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:26.870186  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.870867  4822 ts_tablet_manager.cc:1403] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.672s	user 0.014s	sys 0.054s
I20260709 13:15:26.857690  5348 raft_consensus.cc:2804] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:26.873251  4517 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.873502  5348 raft_consensus.cc:697] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:26.874182  4822 raft_consensus.cc:359] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:26.874647  5338 consensus_queue.cc:1048] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:26.875033  4822 raft_consensus.cc:385] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:26.875097  5280 raft_consensus.cc:2749] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:26.875497  4822 raft_consensus.cc:740] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:26.876457  4821 ts_tablet_manager.cc:1403] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.675s	user 0.029s	sys 0.014s
I20260709 13:15:26.876924  4822 consensus_queue.cc:260] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:26.878960  4822 ts_tablet_manager.cc:1434] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:15:26.879498  4821 raft_consensus.cc:359] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:26.880128  4822 tablet_bootstrap.cc:492] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:26.871711  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.856187  4701 pending_rounds.cc:107] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:15:26.880260  4821 raft_consensus.cc:385] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:26.881475  4701 raft_consensus.cc:2889] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:26.881623  4821 raft_consensus.cc:740] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:26.884968  4821 consensus_queue.cc:260] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:26.886451  5349 raft_consensus.cc:3055] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.886973  5349 raft_consensus.cc:740] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:26.887564  4821 ts_tablet_manager.cc:1434] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:15:26.887811  5349 consensus_queue.cc:260] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:26.888937  4740 leader_election.cc:343] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.889882  5349 raft_consensus.cc:3060] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.890246  4740 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:26.891584  4740 leader_election.cc:343] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.892951  5337 raft_consensus.cc:2804] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:26.893455  5337 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:26.894749  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:26.893862  5337 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.893842  4627 raft_consensus.cc:2468] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:26.893894  5348 consensus_queue.cc:237] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.899804  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:26.902657  4822 tablet_bootstrap.cc:654] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:26.903198  4821 tablet_bootstrap.cc:492] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:26.905314  4823 tablet_bootstrap.cc:492] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:26.906703  4823 ts_tablet_manager.cc:1403] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.824s	user 0.031s	sys 0.021s
I20260709 13:15:26.892588  4740 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.895514  4770 raft_consensus.cc:2468] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:26.913596  4519 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:26.916499  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 217158us
W20260709 13:15:26.919381  4665 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.918021  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:26.924964  5373 raft_consensus.cc:2749] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.925397  4770 raft_consensus.cc:1275] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:26.926484  4774 raft_consensus.cc:1275] T fb2de72a119a4bb2b88f82a3713d8f7c P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:26.927671  4769 raft_consensus.cc:1275] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:26.928773  4823 raft_consensus.cc:359] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:26.929401  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:26.929761  4823 raft_consensus.cc:385] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:26.930162  4823 raft_consensus.cc:740] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:26.918223  4519 leader_election.cc:304] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:26.930920  5212 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.931428  5373 raft_consensus.cc:2804] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:26.931855  5373 raft_consensus.cc:493] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.931543  5212 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:26.932230  5373 raft_consensus.cc:3060] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:26.914404  5349 raft_consensus.cc:2804] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:26.932996  5349 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:26.933387  5349 raft_consensus.cc:3060] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.941274  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:26.942317  4553 raft_consensus.cc:2393] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:26.946988  5212 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
W20260709 13:15:26.949056  4739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.955106  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:26.962344  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 69842us
I20260709 13:15:26.963212  4740 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.964006  5335 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:26.964429  5335 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:26.964792  5335 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:26.965888  4517 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:26.968794  5346 raft_consensus.cc:3055] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.969264  5346 raft_consensus.cc:740] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:26.969686  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:26.970152  5054 consensus_queue.cc:1059] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:26.970100  5346 consensus_queue.cc:260] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:26.976851  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:26.977111  5281 consensus_queue.cc:1048] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:26.977825  5271 consensus_queue.cc:1048] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:26.976838  5054 raft_consensus.cc:3055] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:26.981238  4666 leader_election.cc:304] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:26.985615  4593 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:26.987205  5318 raft_consensus.cc:2749] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:26.991719  5336 consensus_queue.cc:1048] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:26.979305  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:27.010299  4740 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:27.010489  4519 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:27.017253  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 51442.1us
I20260709 13:15:27.019812  4592 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:27.022047  4823 consensus_queue.cc:260] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.026362  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:27.004347  5275 raft_consensus.cc:2764] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:15:26.982370  5054 raft_consensus.cc:740] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
W20260709 13:15:27.037722  4517 leader_election.cc:343] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:26.977875  5332 consensus_queue.cc:1048] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:15:26.975060  5349 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:27.062122  4668 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:27.063324  4898 raft_consensus.cc:2804] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:27.036053  5075 raft_consensus.cc:3060] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.034535  5335 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:27.012789  5281 consensus_queue.cc:1048] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:15:27.065810  4823 ts_tablet_manager.cc:1434] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.159s	user 0.010s	sys 0.000s
I20260709 13:15:27.043902  5346 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.037026  5318 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:27.054075  5338 raft_consensus.cc:2749] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:27.036341  5254 raft_consensus.cc:2749] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:27.114214  4898 raft_consensus.cc:697] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:27.054064  5065 raft_consensus.cc:2804] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:27.026623  5212 consensus_queue.cc:1059] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
W20260709 13:15:27.073629  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 92482.8us
I20260709 13:15:27.116240  4449 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42670 (request call id 54) took 2340 ms. Trace:
I20260709 13:15:27.016217  5264 consensus_queue.cc:1048] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:15:27.197759  4898 consensus_queue.cc:237] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:27.077845  4518 leader_election.cc:304] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:27.117205  4623 raft_consensus.cc:1275] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:27.071563  4622 raft_consensus.cc:1275] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:27.211071  4821 tablet_bootstrap.cc:700] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512: Time spent opening tablet: real 0.100s	user 0.001s	sys 0.000s
W20260709 13:15:27.209966  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 132218us
I20260709 13:15:27.210457  4772 raft_consensus.cc:1240] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:27.215430  4628 raft_consensus.cc:1275] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:27.217885  5249 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.219995  5373 raft_consensus.cc:515] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:27.066553  5175 consensus_queue.cc:1048] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.139s
I20260709 13:15:27.113945  5355 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:27.234190  4821 tablet_bootstrap.cc:654] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:27.235452  4593 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.070420  5280 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:27.166776  5065 raft_consensus.cc:493] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:27.263453  5377 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:27.264040  5378 log.cc:927] Time spent T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:15:27.237361  4776 raft_consensus.cc:1275] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:15:27.235675  5338 raft_consensus.cc:2749] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:27.264778  4594 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.239409  5065 raft_consensus.cc:3060] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:27.238435  5366 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:27.202095  4449 rpcz_store.cc:276] 0709 13:15:24.733042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.733675 (+   633us) service_pool.cc:224] Handling call
0709 13:15:25.009640 (+275965us) spinlock_profiling.cc:243] Waited 93.2 ms on lock 0x7b5400a190d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e1090 00007f59ce67ff3a 00007f59d44361cc 00007f59d453437e 00007f59d29f52a4 00007f59d29f5232 00007f59d29f5161 00007f59d29f50dc 00007f59d29f4392 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:27.072873 (+2063233us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:26.105848 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:26.149674 (+ 43826us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:26.149725 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:15:26.149747 (+    22us) tablet.cc:662] Decoding operations
0709 13:15:26.150365 (+   618us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:26.150457 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:26.150489 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:15:26.150914 (+   425us) tablet.cc:701] Row locks acquired
0709 13:15:26.150954 (+    40us) write_op.cc:260] PREPARE: finished
0709 13:15:26.151131 (+   177us) write_op.cc:270] Start()
0709 13:15:26.151300 (+   169us) write_op.cc:276] Timestamp: P: 1783602926151060 usec, L: 0
0709 13:15:26.151338 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:15:26.152046 (+   708us) log.cc:844] Serialized 1316 byte log entry
0709 13:15:26.302241 (+150195us) op_driver.cc:464] REPLICATION: finished
0709 13:15:26.575224 (+272983us) write_op.cc:301] APPLY: starting
0709 13:15:26.577597 (+  2373us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:26.594417 (+ 16820us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:26.605871 (+ 11454us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:26.606546 (+   675us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:26.606835 (+   289us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:15:26.606883 (+    48us) write_op.cc:312] APPLY: finished
0709 13:15:26.607211 (+   328us) log.cc:844] Serialized 34 byte log entry
0709 13:15:26.607414 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:26.607517 (+   103us) write_op.cc:454] Released schema lock
0709 13:15:26.607690 (+   173us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:26.608194 (+   504us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:26.608225 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":150436,"reported_tablets":11,"spinlock_wait_cycles":214401280,"child_traces":[["op",{"apply.queue_time_us":212665,"apply.run_cpu_time_us":3118,"apply.run_wall_time_us":46355,"num_ops":3,"prepare.queue_time_us":202866,"prepare.run_cpu_time_us":3901,"prepare.run_wall_time_us":64064,"replication_time_us":131193,"thread_start_us":88161,"threads_started":2}]]}
I20260709 13:15:27.254659  5280 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.314414  4553 raft_consensus.cc:1275] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260709 13:15:27.333570  5379 log.cc:927] Time spent T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:15:27.333774  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:27.248761  5355 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.334179  5350 raft_consensus.cc:493] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.264143  5377 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.334594  5334 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:27.352067  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:27.353827  5373 leader_election.cc:290] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:27.355809  5352 raft_consensus.cc:493] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.357576  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:27.357160  5350 raft_consensus.cc:515] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.306627  5337 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:27.359334  4591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.360262  5344 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.310838  5366 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.360970  5344 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.362650  5366 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:27.363678  5344 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:27.364002  5391 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:27.364542  5391 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:27.365639  5377 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:27.366319  5334 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.347221  4774 raft_consensus.cc:1240] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:27.370656  5280 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:27.366704  5391 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:27.327406  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:27.378301  4822 tablet_bootstrap.cc:492] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:27.373000  4768 raft_consensus.cc:1240] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:27.381129  4822 ts_tablet_manager.cc:1403] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.501s	user 0.012s	sys 0.008s
I20260709 13:15:27.381850  5337 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:27.360201  5350 leader_election.cc:290] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:27.356438  5352 raft_consensus.cc:515] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.392135  4628 raft_consensus.cc:3060] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:27.383383  5334 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:27.400516  4694 raft_consensus.cc:1275] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:15:27.399648  5335 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
W20260709 13:15:27.416430  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 115323us
I20260709 13:15:27.363287  5366 consensus_queue.cc:1048] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:27.413767  4694 pending_rounds.cc:85] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a: Aborting all ops after (but not including) 0
I20260709 13:15:27.433051  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:27.439224  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 237190us
I20260709 13:15:27.426084  4694 pending_rounds.cc:107] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:15:27.422830  5280 raft_consensus.cc:493] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.441140  4694 raft_consensus.cc:2889] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:27.441442  5280 raft_consensus.cc:515] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
W20260709 13:15:27.442934  4665 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.443768  5280 leader_election.cc:290] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:27.428973  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:27.437043  4822 raft_consensus.cc:359] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:27.446159  4822 raft_consensus.cc:385] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:27.446484  4822 raft_consensus.cc:740] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:27.446781  4553 raft_consensus.cc:1240] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:27.455801  4628 raft_consensus.cc:2468] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
W20260709 13:15:27.457026  5386 log.cc:927] Time spent T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.138s	user 0.001s	sys 0.001s
I20260709 13:15:27.457195  4821 tablet_bootstrap.cc:492] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:27.457317  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:27.458912  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:27.460274  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:27.461483  4740 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:27.463099  5335 raft_consensus.cc:2804] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:27.463613  5335 raft_consensus.cc:697] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:27.464489  5335 consensus_queue.cc:237] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:27.457688  4821 ts_tablet_manager.cc:1403] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.560s	user 0.016s	sys 0.005s
W20260709 13:15:27.467180  4517 leader_election.cc:343] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:27.469144  4666 leader_election.cc:304] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:27.448652  5318 raft_consensus.cc:2749] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:27.471851  4822 consensus_queue.cc:260] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:27.439277  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:27.474944  4740 consensus_peers.cc:597] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 -> Peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Couldn't send request to peer fb69c26178814cacb4f3abd0750d4b82. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:27.475692  4517 leader_election.cc:343] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:27.477532  5396 consensus_queue.cc:1048] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:15:27.467597  5315 consensus_queue.cc:1059] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:15:27.469610  5400 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:27.478812  5400 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.479022  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:27.476526  5391 raft_consensus.cc:2764] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:15:27.480829  4698 raft_consensus.cc:2468] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:27.483839  5400 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:27.485045  5355 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:27.485601  4822 ts_tablet_manager.cc:1434] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.101s	user 0.007s	sys 0.000s
I20260709 13:15:27.495045  4518 leader_election.cc:304] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:27.486810  5352 leader_election.cc:290] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:27.486810  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:27.489228  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:27.503201  4553 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:27.504714  4592 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:27.435863  5394 consensus_queue.cc:1048] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:15:27.498147  5373 raft_consensus.cc:2804] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:27.499671  5283 consensus_queue.cc:1048] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:27.464691  5065 raft_consensus.cc:515] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.501729  4700 raft_consensus.cc:1275] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:27.510850  5317 raft_consensus.cc:2804] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:27.512181  5373 raft_consensus.cc:493] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:27.521165  4666 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:27.525693  4821 raft_consensus.cc:359] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.526424  4821 raft_consensus.cc:385] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:27.526685  4821 raft_consensus.cc:740] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:27.528126  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:27.528981  4773 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:27.530165  5377 raft_consensus.cc:2804] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:27.531015  5377 raft_consensus.cc:697] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:27.531782  5076 raft_consensus.cc:3055] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:27.532339  5377 consensus_queue.cc:237] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.541491  5054 consensus_queue.cc:260] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:27.543155  5054 raft_consensus.cc:3060] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.518306  5317 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:27.544026  5317 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.518394  5373 raft_consensus.cc:3060] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:27.522571  5400 consensus_queue.cc:1059] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:15:27.551489  5400 raft_consensus.cc:3055] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:27.551918  5400 raft_consensus.cc:740] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:27.552187  5270 consensus_queue.cc:1048] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:27.553194  4821 consensus_queue.cc:260] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:27.543133  4666 leader_election.cc:343] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:27.551481  5283 consensus_queue.cc:1048] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:15:27.558090  4822 tablet_bootstrap.cc:492] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
W20260709 13:15:27.563550  5395 log.cc:927] Time spent T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.212s	user 0.004s	sys 0.000s
I20260709 13:15:27.566720  5065 leader_election.cc:290] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:27.532305  5076 raft_consensus.cc:740] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:27.568516  5076 consensus_queue.cc:260] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:27.570206  5076 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.572256  4666 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:27.575724  5393 log.cc:927] Time spent T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:15:27.587329  4821 ts_tablet_manager.cc:1434] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.120s	user 0.005s	sys 0.002s
I20260709 13:15:27.596244  4666 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:27.579385  5065 raft_consensus.cc:2804] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:27.585861  5366 consensus_queue.cc:1048] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
W20260709 13:15:27.599995  5380 log.cc:927] Time spent T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
W20260709 13:15:27.600437  5385 log.cc:927] Time spent T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:15:27.606505  5372 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:27.606326  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:27.607753  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:27.628119  4774 raft_consensus.cc:1240] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:27.607836  5403 raft_consensus.cc:2804] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:27.629281  5403 raft_consensus.cc:697] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:27.609218  5347 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.583199  5400 consensus_queue.cc:260] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.617224  5249 consensus_queue.cc:1048] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:27.628227  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
W20260709 13:15:27.620182  4666 leader_election.cc:343] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:27.621168  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:27.580331  5355 raft_consensus.cc:493] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:27.626796  4546 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:27.617978  4821 tablet_bootstrap.cc:492] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:27.634809  4666 leader_election.cc:304] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:27.635686  4553 raft_consensus.cc:2393] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:27.636976  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:27.629738  5373 raft_consensus.cc:515] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.650769  4666 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:27.622967  5065 raft_consensus.cc:697] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:27.674042  5271 consensus_queue.cc:1059] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:27.662312  5400 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:27.675346  5271 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.675923  5271 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.637524  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:27.676769  5270 raft_consensus.cc:3055] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:27.677232  5270 raft_consensus.cc:740] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:27.677284  4770 raft_consensus.cc:2468] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:27.678194  5270 consensus_queue.cc:260] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:27.679800  5270 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.670182  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:27.670445  5373 leader_election.cc:290] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:27.666783  5283 consensus_queue.cc:1048] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:15:27.671571  5275 consensus_queue.cc:1059] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:27.673599  5065 consensus_queue.cc:237] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.680966  5396 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.710392  5396 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:27.661824  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:27.697867  4822 tablet_bootstrap.cc:654] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:27.713104  4773 raft_consensus.cc:3060] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:27.630231  5403 consensus_queue.cc:237] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.706245  5317 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.665212  4767 raft_consensus.cc:3060] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:27.647739  5355 raft_consensus.cc:515] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:27.684705  5377 raft_consensus.cc:2749] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:27.708896  4623 raft_consensus.cc:1240] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:27.709126  5281 raft_consensus.cc:3055] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:27.710803  5271 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:27.708290  5275 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.722609  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:27.731647  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:27.677284  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
W20260709 13:15:27.733556  4517 leader_election.cc:343] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:27.698482  4774 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:27.700618  4621 raft_consensus.cc:2468] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:27.697125  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:27.736771  4696 raft_consensus.cc:2393] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 4: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:27.734674  4622 raft_consensus.cc:1240] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:27.759585  4823 tablet_bootstrap.cc:492] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:27.734841  5281 raft_consensus.cc:740] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
W20260709 13:15:27.774735  4593 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.755426  5400 raft_consensus.cc:2804] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:27.776224  4517 leader_election.cc:343] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:27.761569  4695 raft_consensus.cc:1240] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:27.766908  4547 raft_consensus.cc:3060] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:27.775648  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:27.762075  5396 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:27.762653  5335 consensus_queue.cc:1048] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:15:27.781769  5349 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:27.783784  4821 tablet_bootstrap.cc:654] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:27.776293  5212 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:27.649216  5372 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.759958  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:27.786217  4519 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
W20260709 13:15:27.796111  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 145460us
I20260709 13:15:27.736889  5275 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:27.775554  5317 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:27.785416  5400 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:27.809994  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:27.810691  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:27.810832  4666 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:27.816089  5355 leader_election.cc:290] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:27.820345  4519 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:27.804350  4773 raft_consensus.cc:2468] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:27.831943  5417 raft_consensus.cc:2749] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:15:27.833842  4666 leader_election.cc:343] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:27.834461  4666 leader_election.cc:304] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:27.835582  5417 raft_consensus.cc:2749] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:27.826151  5394 raft_consensus.cc:2804] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:27.802150  4698 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:27.804415  5212 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.844532  5372 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:27.825554  5400 raft_consensus.cc:3060] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.846580  4822 tablet_bootstrap.cc:492] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:27.847180  4822 ts_tablet_manager.cc:1403] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.293s	user 0.015s	sys 0.025s
I20260709 13:15:27.848037  5344 raft_consensus.cc:2804] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:27.848559  5344 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:27.849054  5344 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:27.829164  5349 raft_consensus.cc:493] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.849987  5349 raft_consensus.cc:515] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.850668  4822 raft_consensus.cc:359] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:27.852058  4822 raft_consensus.cc:385] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:27.852517  4822 raft_consensus.cc:740] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:27.852644  4519 leader_election.cc:304] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:27.853394  4822 consensus_queue.cc:260] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:27.854806  5212 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:27.855775  4822 ts_tablet_manager.cc:1434] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:15:27.856858  5373 raft_consensus.cc:2804] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:27.857074  4822 tablet_bootstrap.cc:492] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:27.857501  5373 raft_consensus.cc:697] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:27.858402  5349 leader_election.cc:290] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:27.859668  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:27.819814  4697 raft_consensus.cc:2468] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
W20260709 13:15:27.883929  4666 leader_election.cc:343] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:27.841130  5394 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:27.856634  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:27.858667  5373 consensus_queue.cc:237] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:27.901674  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:27.902878  4553 raft_consensus.cc:2393] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:27.897750  4773 raft_consensus.cc:1240] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:27.894402  5394 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:27.877941  5397 consensus_queue.cc:1048] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:15:27.909637  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:27.912384  4822 tablet_bootstrap.cc:654] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:27.881752  5283 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.917559  4666 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:27.931104  4624 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:27.932107  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:27.933982  4547 raft_consensus.cc:2468] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:27.934991  5344 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:27.978440  4666 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.941500  5377 raft_consensus.cc:2804] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:27.949515  5348 raft_consensus.cc:493] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.998342  5348 raft_consensus.cc:515] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:27.942602  5283 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:28.004447  5349 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.005396  5349 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:27.923578  5419 consensus_queue.cc:1048] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.056s
I20260709 13:15:27.995482  5271 consensus_queue.cc:1059] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:27.956383  4773 raft_consensus.cc:2468] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:27.976346  5397 consensus_queue.cc:1048] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:15:27.987953  5377 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.018013  5076 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.050712  4823 tablet_bootstrap.cc:654] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:28.014057  5347 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.060391  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 104894us
I20260709 13:15:28.027158  5377 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.065697  4821 tablet_bootstrap.cc:492] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:28.068549  4821 ts_tablet_manager.cc:1403] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.453s	user 0.010s	sys 0.016s
I20260709 13:15:28.023833  5394 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:28.093828  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 152919us
I20260709 13:15:28.064585  5275 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:28.063310  5076 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:28.066555  5347 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:28.106349  4666 leader_election.cc:304] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:28.107219  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:28.124943  4822 tablet_bootstrap.cc:492] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:28.125521  4822 ts_tablet_manager.cc:1403] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.269s	user 0.019s	sys 0.016s
I20260709 13:15:28.117244  4700 raft_consensus.cc:3060] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:28.117039  5283 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:28.118659  5065 raft_consensus.cc:2804] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:15:28.139658  5424 log.cc:927] Time spent T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:15:28.140892  5404 log.cc:927] Time spent T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.499s	user 0.000s	sys 0.004s
I20260709 13:15:28.141312  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.142091  4773 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:28.146245  5348 leader_election.cc:290] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
W20260709 13:15:28.147491  5421 log.cc:927] Time spent T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:15:28.155423  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.155862  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.158749  5076 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:28.160053  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.168247  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:28.169018  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:28.160143  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.173346  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:28.163305  4821 raft_consensus.cc:359] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:28.176863  4821 raft_consensus.cc:385] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:28.177201  4821 raft_consensus.cc:740] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:28.178994  4553 raft_consensus.cc:1240] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:28.182428  4700 raft_consensus.cc:2468] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:28.166330  5349 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:28.173243  5433 consensus_queue.cc:1059] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:28.186241  5403 raft_consensus.cc:493] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:28.186736  5403 raft_consensus.cc:515] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:28.189427  5394 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:28.193796  5403 leader_election.cc:290] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:28.194432  5281 consensus_queue.cc:260] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:28.196239  5281 raft_consensus.cc:3060] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.166164  4668 leader_election.cc:304] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:28.199087  4668 consensus_peers.cc:597] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a -> Peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Couldn't send request to peer 098b8827550f475e8c8fa8a20de8f512. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:28.178176  4821 consensus_queue.cc:260] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:28.220589  4668 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:28.224251  4741 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:28.235672  4741 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:28.227643  4668 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:28.197726  5400 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:28.238557  4701 raft_consensus.cc:1240] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:28.252487  4668 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:28.226191  5335 raft_consensus.cc:493] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.259187  4821 ts_tablet_manager.cc:1434] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.190s	user 0.007s	sys 0.001s
I20260709 13:15:28.233122  5419 raft_consensus.cc:2749] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.232972  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.281857  4627 raft_consensus.cc:1275] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:28.267598  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:28.282938  4553 raft_consensus.cc:2393] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:28.231289  5432 raft_consensus.cc:2749] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.237147  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:28.264074  4740 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:28.254804  5054 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:28.267164  4822 raft_consensus.cc:359] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
W20260709 13:15:28.268911  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 90092.5us
I20260709 13:15:28.287927  4822 raft_consensus.cc:385] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:28.288328  4822 raft_consensus.cc:740] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:28.288575  4518 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:28.289933  4619 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:28.290036  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:28.291064  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:28.291961  4698 raft_consensus.cc:2393] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 4: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:28.292784  5394 raft_consensus.cc:2749] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.293066  4624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48806 (request call id 101) took 1541 ms. Trace:
I20260709 13:15:28.293525  4624 rpcz_store.cc:276] 0709 13:15:26.751255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.871394 (+120139us) service_pool.cc:224] Handling call
0709 13:15:28.292889 (+1421495us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54006e23d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:28.293004 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2414088960}
I20260709 13:15:28.294582  4625 raft_consensus.cc:1275] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:28.295506  4775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57064 (request call id 86) took 1082 ms. Trace:
I20260709 13:15:28.295344  4822 consensus_queue.cc:260] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:28.265589  5403 raft_consensus.cc:2804] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:28.297304  5403 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:28.297784  5403 raft_consensus.cc:3060] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.298632  5271 raft_consensus.cc:3055] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:28.299088  5271 raft_consensus.cc:740] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:28.299893  5271 consensus_queue.cc:260] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:28.267279  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:28.301232  5433 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.301735  5433 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:28.297801  4822 ts_tablet_manager.cc:1434] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.172s	user 0.005s	sys 0.003s
I20260709 13:15:28.227815  5417 consensus_queue.cc:1059] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
W20260709 13:15:28.304876  4668 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 52439.5us
W20260709 13:15:28.305656  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:28.257025  5337 raft_consensus.cc:2804] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:28.309960  4518 leader_election.cc:304] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:28.310969  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 47039.5us
I20260709 13:15:28.312206  4740 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:28.312448  4821 tablet_bootstrap.cc:492] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:28.313512  5349 raft_consensus.cc:2749] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.313735  5065 raft_consensus.cc:697] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:28.263048  5275 raft_consensus.cc:493] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:28.277516  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.266917  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:28.264534  4622 raft_consensus.cc:3060] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:28.277204  5391 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:28.265460  5076 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.265794  4621 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48806 (request call id 83) took 1983 ms (client timeout 1759 ms). Trace:
W20260709 13:15:28.319659  4519 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.278713  5335 raft_consensus.cc:515] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:28.268020  5436 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.335868  4447 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42686 (request call id 41) took 3145 ms. Trace:
I20260709 13:15:28.353384  4447 rpcz_store.cc:276] 0709 13:15:25.163102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.163269 (+   167us) service_pool.cc:224] Handling call
0709 13:15:25.387872 (+224603us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b5400a190d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e1090 00007f59ce67ff3a 00007f59d44361cc 00007f59d453437e 00007f59d29f52a4 00007f59d29f5232 00007f59d29f5161 00007f59d29f50dc 00007f59d29f4392 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:28.308441 (+2920569us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":504784,"reported_tablets":13,"spinlock_wait_cycles":100828032}
I20260709 13:15:28.336671  4822 tablet_bootstrap.cc:492] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:28.336086  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:28.306205  4695 raft_consensus.cc:2468] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:28.266734  4620 raft_consensus.cc:1275] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:28.266264  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86791c9adcc64babb9560e224fd9abd9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.340866  5279 raft_consensus.cc:2804] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:28.295903  4775 rpcz_store.cc:276] 0709 13:15:27.212983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.221761 (+  8778us) service_pool.cc:224] Handling call
0709 13:15:27.297581 (+ 75820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:27.394787 (+ 97206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:27.397692 (+  2905us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:27.398973 (+  1281us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:27.402806 (+  3833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.295235 (+892429us) spinlock_profiling.cc:243] Waited 75.9 ms on lock 0x7b5400510254. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:28.295290 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.295425 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174495104}
I20260709 13:15:28.296482  4625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48806 (request call id 95) took 1754 ms. Trace:
I20260709 13:15:28.287814  5054 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:28.315934  5249 raft_consensus.cc:2749] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.257244  5350 consensus_queue.cc:1048] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260709 13:15:28.335618  5337 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:28.421561  5373 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.344326  4771 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.474726  5373 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:28.475658  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 83777.9us
I20260709 13:15:28.356694  4553 raft_consensus.cc:2468] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:28.476894  5432 consensus_queue.cc:1059] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:28.350250  5434 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.303309  5377 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:28.366842  5335 leader_election.cc:290] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:28.491267  4898 consensus_queue.cc:1048] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:28.357762  5436 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:28.369335  5433 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:28.387022  5054 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.344941  5391 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:28.350847  5076 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:28.379081  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:28.520159  4627 raft_consensus.cc:2468] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:28.421979  5279 raft_consensus.cc:697] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:28.422556  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.422287  5403 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:28.422946  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:28.346473  4621 rpcz_store.cc:269] 0709 13:15:26.282819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.284617 (+  1798us) service_pool.cc:224] Handling call
0709 13:15:28.265546 (+1980929us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400484c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:28.265705 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2789214592}
I20260709 13:15:28.339396  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86791c9adcc64babb9560e224fd9abd9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:28.423913  4620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48806 (request call id 106) took 1665 ms. Trace:
I20260709 13:15:28.421141  4701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53238 (request call id 102) took 1190 ms. Trace:
I20260709 13:15:28.427325  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:28.427563  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.420761  4451 rpcz_store.cc:275] Call kudu.master.MasterService.IsCreateTableDone from 127.0.0.1:42688 (request call id 38) took 1277 ms. Trace:
I20260709 13:15:28.518980  5444 raft_consensus.cc:493] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.419976  5419 raft_consensus.cc:3055] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:28.555927  5419 raft_consensus.cc:740] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:28.417493  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0059f848a34f47b5a3fe29efe38914b3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.558290  4698 raft_consensus.cc:2468] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:28.558395  5419 consensus_queue.cc:260] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:28.559742  5419 raft_consensus.cc:3060] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.560050  5377 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:28.416298  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:28.418785  5347 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:28.416244  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.418318  5400 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:28.383208  4625 rpcz_store.cc:276] 0709 13:15:26.542129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.559956 (+ 17827us) service_pool.cc:224] Handling call
0709 13:15:26.964459 (+404503us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x7b5800003f98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:28.294061 (+1329602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.295139 (+  1078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.296200 (+  1061us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54006e23d0. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:28.296375 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3070862848}
I20260709 13:15:28.386062  5394 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.435364  4450 catalog_manager.cc:5697] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 0 to 1, leader changed from <none> to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 1 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:28.416236  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:28.418762  5344 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:28.572863  5394 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:28.421883  5337 raft_consensus.cc:3060] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.575276  5394 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:28.389333  5212 raft_consensus.cc:493] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:28.577018  5212 raft_consensus.cc:515] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:28.579648  5212 leader_election.cc:290] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:28.449787  4741 leader_election.cc:304] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
W20260709 13:15:28.581187  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 131485us
I20260709 13:15:28.582717  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.582860  4741 leader_election.cc:304] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:28.583554  4698 raft_consensus.cc:2468] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:28.450876  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:28.569634  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.586032  5436 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:28.586362  4552 raft_consensus.cc:2393] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:28.452764  4740 leader_election.cc:304] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:28.587649  5339 raft_consensus.cc:2804] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:28.588045  5339 raft_consensus.cc:493] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:28.588189  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 159464us
I20260709 13:15:28.588532  5259 raft_consensus.cc:2749] T c59401c42fe8430aaed4b49ef7df0372 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:28.589951  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.555951  5444 raft_consensus.cc:515] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:28.595041  4693 raft_consensus.cc:1240] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3. Ops: [2.1-2.1]
I20260709 13:15:28.597556  4450 catalog_manager.cc:5697] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 0 to 1, leader changed from <none> to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 1 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:28.453347  4699 raft_consensus.cc:1240] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:28.430917  5270 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.385524  5065 consensus_queue.cc:237] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:28.457055  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.609658  4700 raft_consensus.cc:2468] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:28.341511  5275 raft_consensus.cc:515] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:28.396695  4519 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:28.352234  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:28.493729  4821 tablet_bootstrap.cc:654] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:28.350065  5271 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.477511  5373 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:28.516716  4553 raft_consensus.cc:1275] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:28.421732  4823 tablet_bootstrap.cc:492] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:28.648474  4823 ts_tablet_manager.cc:1403] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.929s	user 0.034s	sys 0.054s
I20260709 13:15:28.522753  4517 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:28.538546  4822 tablet_bootstrap.cc:654] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:28.521158  5391 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.430809  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:28.544112  4701 rpcz_store.cc:276] 0709 13:15:27.197106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.217399 (+ 20293us) service_pool.cc:224] Handling call
0709 13:15:27.217831 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.251084 (+1033253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.387609 (+136525us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54006c89d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:28.387853 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2346925440}
I20260709 13:15:28.551789  4769 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:28.549124  4451 rpcz_store.cc:276] 0709 13:15:27.110027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.221816 (+111789us) service_pool.cc:224] Handling call
0709 13:15:27.262901 (+ 41085us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0709 13:15:28.314953 (+1052052us) catalog_manager.cc:2317] Verify if the table creation is in progress for test-workload [id=a4b690bde17245419f655f8ee4ad32e2]
0709 13:15:28.387073 (+ 72120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1097299}
I20260709 13:15:28.553645  5076 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:28.539795  4544 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:28.554997  5403 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:28.496016  5434 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:28.567018  5279 consensus_queue.cc:237] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:28.543972  4620 rpcz_store.cc:276] 0709 13:15:26.758949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.962538 (+203589us) service_pool.cc:224] Handling call
0709 13:15:28.266348 (+1303810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.362625 (+ 96277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.423688 (+ 61063us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400484c50. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce6768ad 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:28.423838 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2997593344}
I20260709 13:15:28.568671  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:28.549458  5397 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.584349  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:28.600834  4594 leader_election.cc:343] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:28.601197  5444 leader_election.cc:290] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:28.602097  4518 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:28.584918  5335 raft_consensus.cc:2804] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:28.608457  5339 raft_consensus.cc:3060] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:28.608606  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:28.610343  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:28.611186  5443 raft_consensus.cc:3055] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:28.609354  5270 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:28.668627  5394 raft_consensus.cc:2804] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:28.649298  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 126649us
W20260709 13:15:28.648514  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 252016us
I20260709 13:15:28.655191  4740 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
W20260709 13:15:28.710691  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 55609.4us
I20260709 13:15:28.714071  4592 leader_election.cc:304] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:28.664474  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.656411  5435 raft_consensus.cc:2804] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:28.678964  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 76977.9us
W20260709 13:15:28.711809  4517 leader_election.cc:343] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:28.660506  4771 raft_consensus.cc:2468] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:28.625177  4593 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:28.683233  5373 raft_consensus.cc:2804] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:28.681514  4550 raft_consensus.cc:2393] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 3: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:28.666059  5350 consensus_queue.cc:1048] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:15:28.763618  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0059f848a34f47b5a3fe29efe38914b3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:28.764163  5270 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:28.715060  4701 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:28.678179  5352 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.668392  5349 consensus_queue.cc:1059] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
W20260709 13:15:28.727906  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 102822us
I20260709 13:15:28.731139  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:28.731158  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:28.732115  5433 consensus_queue.cc:1048] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:28.734457  5436 consensus_queue.cc:1048] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:28.735002  4545 raft_consensus.cc:2393] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:28.735533  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:28.736538  5373 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.740020  5076 raft_consensus.cc:2749] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.741585  4696 raft_consensus.cc:1275] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:28.743695  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.746373  4823 raft_consensus.cc:359] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:28.707721  5443 raft_consensus.cc:740] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:28.748859  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.752871  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.721544  5435 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:28.681604  5335 raft_consensus.cc:493] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:28.681778  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 81029.1us
I20260709 13:15:28.825928  4519 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
W20260709 13:15:28.827654  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 116326us
I20260709 13:15:28.671952  5397 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:28.831362  4823 raft_consensus.cc:385] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:28.850041  4823 raft_consensus.cc:740] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:28.831800  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.717554  5175 raft_consensus.cc:2804] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:28.857528  5175 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.858038  5175 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.832166  4544 raft_consensus.cc:2468] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 0.
I20260709 13:15:28.861047  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.861475  4740 leader_election.cc:304] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:28.864159  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:28.721434  5280 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.865464  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:28.865628  5280 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:28.867260  4623 raft_consensus.cc:1275] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:28.869189  5275 leader_election.cc:290] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:28.869737  5334 raft_consensus.cc:2749] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:28.833915  5455 raft_consensus.cc:493] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.872352  5455 raft_consensus.cc:515] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:28.847008  4774 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:28.875380  5455 leader_election.cc:290] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:28.875900  4519 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:28.828377  5373 raft_consensus.cc:3060] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.877930  4740 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
W20260709 13:15:28.881349  4666 leader_election.cc:343] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:28.788662  5432 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:28.766072  5347 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.855152  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.855515  4594 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:28.855872  4823 consensus_queue.cc:260] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:28.765849  5212 raft_consensus.cc:493] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:28.830801  5444 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:28.764624  5054 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:28.896708  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 143786us
I20260709 13:15:28.760461  5450 raft_consensus.cc:2749] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:28.852381  5335 raft_consensus.cc:3060] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.842437  5076 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.926098  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 46319.3us
I20260709 13:15:28.846087  5435 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.828466  5352 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:28.902295  5279 raft_consensus.cc:2804] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:28.976197  4773 raft_consensus.cc:1240] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:28.979328  5397 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:28.979986  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.980967  5352 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:28.981348  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:28.982236  5463 raft_consensus.cc:2804] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:28.982736  5463 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.982908  5350 raft_consensus.cc:3055] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:28.983085  5463 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:28.985126  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:28.985872  5280 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:28.921556  5275 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:28.923138  5366 raft_consensus.cc:2804] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:28.989120  4668 leader_election.cc:343] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:29.003144  4668 leader_election.cc:304] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:28.991659  4592 leader_election.cc:304] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:28.861155  4619 raft_consensus.cc:3060] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:28.930351  5455 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:29.004709  4668 leader_election.cc:343] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:28.986539  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.008577  4592 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:28.985916  4698 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:29.041257  4823 ts_tablet_manager.cc:1434] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.392s	user 0.011s	sys 0.001s
I20260709 13:15:28.983435  5350 raft_consensus.cc:740] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:28.931449  4776 raft_consensus.cc:2468] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:29.054046  4822 tablet_bootstrap.cc:492] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:29.054785  4822 ts_tablet_manager.cc:1403] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.718s	user 0.013s	sys 0.013s
I20260709 13:15:28.906594  5372 consensus_queue.cc:1048] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.089s
I20260709 13:15:28.929414  5212 raft_consensus.cc:515] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:29.003032  5366 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.922299  5347 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:28.994902  5275 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:29.000959  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 145530us
I20260709 13:15:28.922377  5457 consensus_queue.cc:1059] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.171s
I20260709 13:15:28.931565  5444 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:28.987193  5076 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:29.006664  5279 raft_consensus.cc:697] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:29.056497  5455 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.095912  5279 consensus_queue.cc:237] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.097656  5455 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:28.922170  5432 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.098418  4518 leader_election.cc:304] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:29.099498  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 382224us
I20260709 13:15:29.100178  5432 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:29.090715  4668 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:29.003979  5337 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.080998  5366 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.103292  4518 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:29.052708  5434 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.012159  5450 raft_consensus.cc:2749] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:29.105532  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 97051.9us
I20260709 13:15:29.011902  5456 raft_consensus.cc:2749] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:29.086009  5275 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.107633  5455 raft_consensus.cc:2804] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:29.170543  5455 raft_consensus.cc:697] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:29.130230  5434 raft_consensus.cc:2804] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.173123  5434 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.173518  5434 raft_consensus.cc:3060] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.134755  5450 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:29.176828  5450 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.178611  4593 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:29.179085  5450 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:29.179836  5450 raft_consensus.cc:2749] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:29.135243  4518 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
W20260709 13:15:29.180959  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 45804.1us
I20260709 13:15:29.181202  5344 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:29.181661  5344 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.181388  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:29.182133  5344 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:29.135852  5280 consensus_queue.cc:1059] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:29.184515  5280 raft_consensus.cc:3055] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:29.184892  5280 raft_consensus.cc:740] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:29.185730  5280 consensus_queue.cc:260] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:29.142549  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:29.144377  4623 raft_consensus.cc:1275] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:29.187135  5280 raft_consensus.cc:3060] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.146036  5054 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:29.153201  5394 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.154579  5451 raft_consensus.cc:2804] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:29.156002  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.157687  5339 raft_consensus.cc:515] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.161327  4823 tablet_bootstrap.cc:492] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc: Bootstrap starting.
I20260709 13:15:29.209337  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:29.164268  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.113396  5350 consensus_queue.cc:260] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.120625  5433 raft_consensus.cc:2749] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:29.168586  5396 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:29.189406  4821 tablet_bootstrap.cc:492] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:29.195940  4666 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:29.257987  4821 ts_tablet_manager.cc:1403] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.946s	user 0.024s	sys 0.025s
W20260709 13:15:29.258327  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 62514.3us
W20260709 13:15:29.259835  4666 consensus_peers.cc:597] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a -> Peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Couldn't send request to peer fb69c26178814cacb4f3abd0750d4b82. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:29.259907  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.151852  5175 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.260787  4627 raft_consensus.cc:2410] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:15:29.262015  4773 raft_consensus.cc:1240] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:29.262467  5175 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.145126  5337 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:29.109679  4621 raft_consensus.cc:3060] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:29.263324  5444 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:29.171838  5455 consensus_queue.cc:237] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:29.219730  4519 leader_election.cc:343] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:29.266024  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 46326.9us
I20260709 13:15:29.263882  4821 raft_consensus.cc:359] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:28.935348  5443 consensus_queue.cc:260] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.156293  5435 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:29.209532  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:29.202492  5349 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:29.272200  5457 consensus_queue.cc:1059] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:15:29.272883  5391 consensus_meta.cc:287] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a: Time spent flushing consensus metadata: real 0.604s	user 0.008s	sys 0.004s
I20260709 13:15:29.266353  5449 raft_consensus.cc:493] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:29.273211  5457 raft_consensus.cc:3055] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:29.273824  5457 raft_consensus.cc:740] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:29.273497  5449 raft_consensus.cc:515] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.274314  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.274479  4622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48802 (request call id 129) took 1083 ms. Trace:
I20260709 13:15:29.274998  4624 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:29.274796  5457 consensus_queue.cc:260] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.213106  5394 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.276136  5457 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.273440  5391 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.277419  4702 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:29.212517  5451 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.278241  5451 raft_consensus.cc:3060] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:29.263345  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.280012  5391 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:29.243971  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.241119  5396 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.281168  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.243361  5076 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.246800  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.282639  5396 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.283631  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.214107  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.284514  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:29.284932  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.285997  4693 raft_consensus.cc:1240] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3. Ops: [2.1-2.1]
I20260709 13:15:29.286429  4699 raft_consensus.cc:2393] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:29.239897  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.250902  5373 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.255401  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.290854  4700 raft_consensus.cc:2468] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:29.202469  5065 consensus_queue.cc:1048] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:15:29.253269  5212 leader_election.cc:290] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.272632  5349 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:29.291528  5463 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.246377  4619 raft_consensus.cc:1275] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:29.280424  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.267575  4821 raft_consensus.cc:385] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:29.293803  4821 raft_consensus.cc:740] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:29.276136  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.295171  4821 consensus_queue.cc:260] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.248013  4546 raft_consensus.cc:2468] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:29.293856  5349 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:29.297366  5347 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.265651  5350 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.274950  4622 rpcz_store.cc:276] 0709 13:15:28.190821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.226053 (+ 35232us) service_pool.cc:224] Handling call
0709 13:15:28.264349 (+ 38296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:28.653305 (+388956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:28.656327 (+  3022us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:28.707346 (+ 51019us) log.cc:844] Serialized 26 byte log entry
0709 13:15:28.893764 (+186418us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:29.048443 (+154679us) log.cc:844] Serialized 14 byte log entry
0709 13:15:29.264831 (+216388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.268539 (+  3708us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:29.274243 (+  5704us) raft_consensus.cc:1685] finished
0709 13:15:29.274262 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.274417 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":216,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":280087,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":167,"thread_start_us":455743,"threads_started":3,"wal-append.queue_time_us":219409}
I20260709 13:15:29.305465  5450 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:29.306011  5339 leader_election.cc:290] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.306164  5450 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:29.307309  4592 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:29.275913  5344 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:29.313673  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 43654.4us
I20260709 13:15:29.311480  4517 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:29.293200  5373 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:29.265744  5419 raft_consensus.cc:2749] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:29.324656  4553 raft_consensus.cc:1240] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:29.293723  4550 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:29.345604  4822 raft_consensus.cc:359] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:29.358359  4822 raft_consensus.cc:385] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:29.358757  4822 raft_consensus.cc:740] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
W20260709 13:15:29.347198  4740 leader_election.cc:343] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:29.347465  4741 leader_election.cc:304] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:29.348515  4544 raft_consensus.cc:2368] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 3. Current term is 4.
I20260709 13:15:29.316068  4767 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:29.322167  5454 consensus_queue.cc:1059] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 13:15:29.349817  4546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51884 (request call id 127) took 1083 ms. Trace:
I20260709 13:15:29.350227  5463 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:29.339658  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:29.354422  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.355261  5394 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.355886  4823 tablet_bootstrap.cc:654] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:29.356119  4695 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:29.356881  4821 ts_tablet_manager.cc:1434] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.099s	user 0.005s	sys 0.002s
I20260709 13:15:29.317762  5472 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:29.407109  4668 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:29.337438  5076 raft_consensus.cc:2749] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:29.313472  5403 consensus_queue.cc:1048] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:15:29.371060  4822 consensus_queue.cc:260] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:29.321080  5391 consensus_queue.cc:1048] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:15:29.361699  5373 raft_consensus.cc:2804] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:15:29.408905  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 61508.2us
I20260709 13:15:29.356859  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.350283  4694 raft_consensus.cc:2468] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:29.380330  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.380753  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:29.422502  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 44785.2us
I20260709 13:15:29.383841  5473 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.421416  4821 tablet_bootstrap.cc:492] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:29.429019  4741 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:29.384588  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:29.435420  4592 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:29.405411  5450 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:29.416194  5419 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:29.445171  4822 ts_tablet_manager.cc:1434] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.390s	user 0.008s	sys 0.003s
I20260709 13:15:29.446110  4741 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:29.415192  5264 raft_consensus.cc:493] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.410821  4773 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:29.425901  5460 raft_consensus.cc:2804] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.402509  5335 raft_consensus.cc:515] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:29.459995  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 192692us
I20260709 13:15:29.436846  4699 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.382680  4546 rpcz_store.cc:276] 0709 13:15:28.266774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.266999 (+   225us) service_pool.cc:224] Handling call
0709 13:15:29.349597 (+1082598us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b540006e0d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:29.349732 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1824932736}
I20260709 13:15:29.453503  5347 raft_consensus.cc:2804] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.488538  4668 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:29.458132  4549 raft_consensus.cc:2393] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
W20260709 13:15:29.498744  4665 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.466514  5460 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.465931  5419 raft_consensus.cc:697] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:29.465055  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.462908  5337 raft_consensus.cc:2804] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:29.464494  5466 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:29.476141  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:29.415798  5472 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:29.505811  4665 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.519006  4775 raft_consensus.cc:2368] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:29.493397  5396 raft_consensus.cc:493] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.493883  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.506027  5065 raft_consensus.cc:2804] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:29.505364  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0059f848a34f47b5a3fe29efe38914b3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.494407  5347 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.508210  5460 raft_consensus.cc:3060] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.461349  5264 raft_consensus.cc:515] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:29.542424  4694 raft_consensus.cc:3060] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:29.542692  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:29.539326  5347 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.524442  5366 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.553210  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.555267  5366 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:29.535852  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0059f848a34f47b5a3fe29efe38914b3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:29.561064  5483 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:29.561847  5483 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:29.531116  5434 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.564244  5483 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.564666  4545 raft_consensus.cc:2368] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3.
I20260709 13:15:29.566020  5434 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:29.446988  5473 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:29.526530  5344 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:29.566720  5419 consensus_queue.cc:237] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.568638  5473 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:29.570043  5472 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:29.534993  5396 raft_consensus.cc:515] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.534247  5065 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:29.573864  5335 leader_election.cc:290] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:29.519447  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:29.575143  5396 leader_election.cc:290] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.575362  4768 raft_consensus.cc:3060] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.576929  4628 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:29.519289  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.578038  4547 raft_consensus.cc:2393] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:29.525595  5337 raft_consensus.cc:697] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:29.525272  5466 raft_consensus.cc:697] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:29.581045  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:29.566285  4776 raft_consensus.cc:2368] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:29.581218  5337 consensus_queue.cc:237] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.581940  5466 consensus_queue.cc:237] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.573868  5065 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.586354  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
W20260709 13:15:29.585851  4740 leader_election.cc:343] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:29.581838  4772 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.599224  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:29.600212  4771 raft_consensus.cc:2393] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:29.600768  4620 raft_consensus.cc:1240] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:29.600943  4767 raft_consensus.cc:2410] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:15:29.602027  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:29.602722  4767 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:29.535140  4550 raft_consensus.cc:2468] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:29.605060  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.543439  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:29.618257  4552 raft_consensus.cc:2368] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:29.619251  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.620007  4552 raft_consensus.cc:2368] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:29.620167  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.622958  5275 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:29.623529  4769 consensus_meta.cc:287] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512: Time spent flushing consensus metadata: real 0.790s	user 0.005s	sys 0.005s
I20260709 13:15:29.623461  4544 raft_consensus.cc:1275] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:29.619776  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.550813  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:29.626623  4519 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:29.538128  4696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53252 (request call id 169) took 1336 ms. Trace:
I20260709 13:15:29.605741  4550 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 4.
I20260709 13:15:29.633049  4518 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:29.642165  4518 leader_election.cc:304] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:29.645414  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:29.646329  4553 raft_consensus.cc:3060] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.642338  5065 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.630541  4769 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:29.637542  4696 rpcz_store.cc:276] 0709 13:15:28.201905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.219382 (+ 17477us) service_pool.cc:224] Handling call
0709 13:15:28.429092 (+209710us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b580029a398. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:28.429288 (+   196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.827230 (+397942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.517333 (+690103us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400174154. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:29.532321 (+ 14988us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721219072}
I20260709 13:15:29.654299  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:29.656404  5279 raft_consensus.cc:2749] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:29.657259  5275 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:29.632609  4697 raft_consensus.cc:3060] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.664119  4519 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:29.640897  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.657528  5264 leader_election.cc:290] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:29.679597  5065 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:29.657553  5478 raft_consensus.cc:2749] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:29.658583  5483 raft_consensus.cc:2749] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:29.659327  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:29.677044  4547 raft_consensus.cc:2368] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:29.665154  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.675060  5275 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:29.706285  4740 leader_election.cc:343] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:29.697012  5270 raft_consensus.cc:2804] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:29.723201  4553 raft_consensus.cc:2468] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:29.722083  4771 raft_consensus.cc:2393] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:29.716903  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:29.715406  5275 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:29.726056  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:29.738818  4821 tablet_bootstrap.cc:654] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:29.724215  5270 raft_consensus.cc:697] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:29.750252  4823 tablet_bootstrap.cc:492] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc: No bootstrap required, opened a new log
I20260709 13:15:29.750931  4823 ts_tablet_manager.cc:1403] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc: Time spent bootstrapping tablet: real 0.590s	user 0.017s	sys 0.027s
I20260709 13:15:29.744238  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.756115  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.756803  4693 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.731498  5451 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:29.749342  4550 raft_consensus.cc:2393] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 3: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:29.764323  5432 raft_consensus.cc:3055] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:29.764838  5432 raft_consensus.cc:740] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:29.765767  5432 consensus_queue.cc:260] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:29.766736  5451 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:29.767764  4626 raft_consensus.cc:2393] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 1: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:29.770560  4517 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:29.772109  5478 raft_consensus.cc:3060] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:29.766817  5432 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.776494  4822 tablet_bootstrap.cc:492] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:29.778138  4775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57064 (request call id 95) took 1188 ms. Trace:
I20260709 13:15:29.784966  4775 rpcz_store.cc:276] 0709 13:15:28.589449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.589623 (+   174us) service_pool.cc:224] Handling call
0709 13:15:29.777899 (+1188276us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b540025dad4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:29.778036 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076683136}
I20260709 13:15:29.779704  5394 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:29.782178  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.787400  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.785384  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.788551  5275 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:29.795356  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.788887  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.779423  4699 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:29.796119  4551 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:29.799011  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.801597  4701 raft_consensus.cc:3060] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.804425  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:29.806054  4668 leader_election.cc:400] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999) with higher term. Message: Invalid argument: T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:29.806429  4668 leader_election.cc:403] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:29.815816  4701 raft_consensus.cc:2468] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:29.817850  4518 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:29.800480  5451 raft_consensus.cc:493] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.792673  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.827481  5270 consensus_queue.cc:237] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.829753  4823 raft_consensus.cc:359] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.836447  4823 raft_consensus.cc:385] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:29.836783  4823 raft_consensus.cc:740] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Initialized, Role: FOLLOWER
I20260709 13:15:29.837409  4823 consensus_queue.cc:260] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.838367  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.839088  4699 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:29.844380  5397 raft_consensus.cc:493] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:29.845055  5397 raft_consensus.cc:515] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:29.847259  5397 leader_election.cc:290] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:29.832520  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.830436  5451 raft_consensus.cc:515] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:29.849993  4550 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:29.839031  4823 ts_tablet_manager.cc:1434] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc: Time spent starting tablet: real 0.088s	user 0.005s	sys 0.000s
I20260709 13:15:29.851423  5451 leader_election.cc:290] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.827613  5394 raft_consensus.cc:2804] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:29.852258  5394 raft_consensus.cc:697] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:29.853197  5394 consensus_queue.cc:237] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.854997  4699 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:29.856652  4518 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:29.858180  5397 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:29.858920  5397 raft_consensus.cc:697] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:29.859838  5397 consensus_queue.cc:237] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.869166  5478 raft_consensus.cc:2749] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:29.883138  4822 tablet_bootstrap.cc:654] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:29.888664  4551 raft_consensus.cc:2468] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:29.848228  4772 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:29.895628  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.905397  4621 raft_consensus.cc:2393] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 2: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:29.895931  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3079151438004a9a975fdbe1b5b54378" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.896682  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.892303  5403 raft_consensus.cc:493] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.907294  4626 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:29.907471  5403 raft_consensus.cc:515] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:29.897013  4592 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:29.897179  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.909669  5451 raft_consensus.cc:2749] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:29.909870  5403 leader_election.cc:290] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:29.911535  4592 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:29.912519  5451 raft_consensus.cc:2804] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:29.912978  5451 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:29.913394  5451 raft_consensus.cc:3060] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.914812  5347 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:29.915508  4694 raft_consensus.cc:2468] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:29.878535  4768 raft_consensus.cc:2468] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
W20260709 13:15:29.910068  5457 consensus_meta.cc:287] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a: Time spent flushing consensus metadata: real 0.633s	user 0.013s	sys 0.000s
I20260709 13:15:29.919375  4695 raft_consensus.cc:2410] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:29.923578  4519 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:29.924880  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.925191  5270 raft_consensus.cc:2804] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:29.925781  5270 raft_consensus.cc:697] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:29.926471  5435 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:29.926820  5270 consensus_queue.cc:237] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:29.925895  4550 raft_consensus.cc:2393] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:29.927018  5460 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:29.899163  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:29.932672  4693 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:29.932711  4592 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:29.955624  4592 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:29.941013  4700 raft_consensus.cc:2468] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:29.937140  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:29.959550  4518 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
W20260709 13:15:29.963218  4517 consensus_peers.cc:597] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc -> Peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Couldn't send request to peer fb69c26178814cacb4f3abd0750d4b82. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:29.954197  5337 raft_consensus.cc:493] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.965909  4517 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:29.956529  5456 raft_consensus.cc:493] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:29.962569  5472 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:15:29.969184  4517 leader_election.cc:343] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:29.959157  5212 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.972980  4517 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:29.976471  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:29.972570  5212 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.977756  5212 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:29.983417  4697 raft_consensus.cc:2468] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:29.984534  5419 raft_consensus.cc:2749] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999) with higher term. Message: Invalid argument: T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:29.971801  5486 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:29.985518  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:29.969095  5337 raft_consensus.cc:515] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:29.990079  5337 leader_election.cc:290] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:29.991870  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:29.992122  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:29.998828  4593 leader_election.cc:304] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:29.971877  4769 raft_consensus.cc:2368] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3.
I20260709 13:15:29.967478  5433 raft_consensus.cc:2804] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:15:30.002225  4517 consensus_peers.cc:597] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc -> Peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Couldn't send request to peer fb69c26178814cacb4f3abd0750d4b82. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:30.003865  4741 leader_election.cc:304] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:30.004864  4821 tablet_bootstrap.cc:492] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:30.004848  5339 consensus_queue.cc:1048] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:30.005203  4741 leader_election.cc:304] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:30.006162  4822 tablet_bootstrap.cc:492] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:30.006480  5337 raft_consensus.cc:2804] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:30.006736  4822 ts_tablet_manager.cc:1403] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.230s	user 0.015s	sys 0.001s
I20260709 13:15:30.006798  4821 ts_tablet_manager.cc:1403] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.595s	user 0.025s	sys 0.031s
I20260709 13:15:30.007079  5337 raft_consensus.cc:697] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:30.008076  5337 consensus_queue.cc:237] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.006059  5488 raft_consensus.cc:2804] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.011127  4821 raft_consensus.cc:359] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:29.972364  5472 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:29.977226  4694 raft_consensus.cc:1240] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:30.018932  4821 raft_consensus.cc:385] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.035367  4821 raft_consensus.cc:740] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.036377  4821 consensus_queue.cc:260] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.039927  4821 ts_tablet_manager.cc:1434] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:15:29.973878  5456 raft_consensus.cc:515] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.044078  5456 leader_election.cc:290] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:30.011540  5485 raft_consensus.cc:2749] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.046037  4694 raft_consensus.cc:1240] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:30.047734  5443 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.048429  5339 consensus_queue.cc:1059] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:30.049393  4702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53246 (request call id 150) took 1471 ms. Trace:
I20260709 13:15:30.049629  5396 raft_consensus.cc:3055] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:30.050086  5396 raft_consensus.cc:740] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:30.049965  4702 rpcz_store.cc:276] 0709 13:15:28.578140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.584081 (+  5941us) service_pool.cc:224] Handling call
0709 13:15:30.049085 (+1465004us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54006c75d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:30.049273 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1392369536}
I20260709 13:15:30.051222  5396 consensus_queue.cc:260] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:30.052914  4665 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.052567  5396 raft_consensus.cc:3060] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.002660  4593 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
W20260709 13:15:30.058171  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 55587.1us
I20260709 13:15:30.059128  4593 leader_election.cc:304] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:29.995186  5460 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:30.023090  5488 raft_consensus.cc:697] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:30.032377  5433 raft_consensus.cc:697] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:29.996610  5397 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:29.995975  5347 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:29.992457  5451 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.069981  4593 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:29.995808  5212 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.074306  4822 raft_consensus.cc:359] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:30.109108  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:30.110623  4822 raft_consensus.cc:385] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.057478  5472 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.068527  5397 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.111249  4822 raft_consensus.cc:740] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.111086  4627 raft_consensus.cc:3060] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.066967  5485 raft_consensus.cc:2804] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:30.115435  4821 tablet_bootstrap.cc:492] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
W20260709 13:15:30.114995  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 45143.1us
I20260709 13:15:30.118276  4822 consensus_queue.cc:260] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:30.110205  5339 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.076860  5473 raft_consensus.cc:3060] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.094812  5486 raft_consensus.cc:2749] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.068387  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:30.114459  5397 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.119290  5347 raft_consensus.cc:2749] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.131955  4627 raft_consensus.cc:2468] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:30.118784  5488 consensus_queue.cc:237] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:30.137638  4697 raft_consensus.cc:1240] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:30.138911  5347 consensus_queue.cc:1059] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:30.139564  5054 consensus_queue.cc:1059] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:30.142964  4547 raft_consensus.cc:1240] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:30.152835  5433 consensus_queue.cc:237] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.153276  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:30.154417  4619 raft_consensus.cc:2376] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3. Re-sending same reply.
I20260709 13:15:30.157806  4548 raft_consensus.cc:1240] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:30.125533  5339 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.161963  4741 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:30.163221  5339 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:30.163880  5339 raft_consensus.cc:2749] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.165194  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3079151438004a9a975fdbe1b5b54378" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:30.165597  4770 raft_consensus.cc:1275] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:15:30.167349  4740 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:30.168927  4666 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.169523  4666 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.171161  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:30.172117  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:30.172910  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:30.173691  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:30.174093  4776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57064 (request call id 104) took 1071 ms. Trace:
I20260709 13:15:30.174659  4740 leader_election.cc:400] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 3. Current term is 4.
W20260709 13:15:30.195808  4665 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.201406  5396 raft_consensus.cc:2804] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:30.224922  4770 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.179941  5460 consensus_queue.cc:1059] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.027s
W20260709 13:15:30.182428  4519 leader_election.cc:343] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:30.127646  5337 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.228643  5397 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.238396  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:30.240528  5397 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:30.222314  5451 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:30.241768  5473 raft_consensus.cc:2749] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:30.223877  5472 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.229529  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:30.229662  5339 raft_consensus.cc:3055] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:30.246380  5339 raft_consensus.cc:740] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:30.246974  4771 raft_consensus.cc:2410] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:30.247282  5339 consensus_queue.cc:260] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.248081  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:30.224382  4693 raft_consensus.cc:1275] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:15:30.248699  5339 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.232249  5449 leader_election.cc:290] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:30.261451  5449 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:30.262024  5449 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:30.264349  5449 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:30.233242  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:30.265424  4821 tablet_bootstrap.cc:654] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.265583  4547 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.265785  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:30.266832  4550 raft_consensus.cc:2468] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:30.234293  5434 raft_consensus.cc:493] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.268510  5434 raft_consensus.cc:515] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.234643  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:30.234724  4519 leader_election.cc:304] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:30.274750  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 92379.1us
I20260709 13:15:30.276165  5486 raft_consensus.cc:2749] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:30.276690  5478 consensus_queue.cc:1048] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:30.278900  4519 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:30.279446  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:30.280148  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0c73f991a42e4817d8916fa0a7c30" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:30.235262  5396 raft_consensus.cc:697] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:30.281824  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:30.282068  5396 consensus_queue.cc:237] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.282784  4775 raft_consensus.cc:2393] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:30.284102  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:30.211975  4776 rpcz_store.cc:276] 0709 13:15:29.102873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.105650 (+  2777us) service_pool.cc:224] Handling call
0709 13:15:29.181149 (+ 75499us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b58009fb798. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:15:30.173823 (+992674us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54003d8e54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:30.174022 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987266560}
I20260709 13:15:30.224363  4619 raft_consensus.cc:2468] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:30.280440  4769 raft_consensus.cc:2393] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:30.287487  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:30.230479  5212 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:30.214483  4740 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:30.249145  4693 pending_rounds.cc:85] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a: Aborting all ops after (but not including) 0
I20260709 13:15:30.290087  4693 pending_rounds.cc:107] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:15:30.290364  4693 raft_consensus.cc:2889] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:30.291049  5457 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.211086  5511 raft_consensus.cc:3055] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:30.179175  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:30.291553  5457 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.129807  4699 raft_consensus.cc:3060] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.280311  5486 raft_consensus.cc:2749] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.248714  4628 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.284894  4773 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.236207  5337 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.291682  5511 raft_consensus.cc:740] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:30.295167  5511 consensus_queue.cc:260] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:30.295704  5337 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:30.301771  4772 raft_consensus.cc:3060] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.301919  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:30.290196  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 75828us
I20260709 13:15:30.305462  5490 raft_consensus.cc:2749] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.306173  4740 leader_election.cc:400] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3.
I20260709 13:15:30.308954  4699 raft_consensus.cc:2468] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:30.305190  4696 raft_consensus.cc:2468] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 0.
I20260709 13:15:30.313891  4518 leader_election.cc:304] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:30.314882  5275 raft_consensus.cc:2804] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.315508  5275 raft_consensus.cc:493] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.315868  5275 raft_consensus.cc:3060] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:30.316220  4546 raft_consensus.cc:2468] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:30.318547  4547 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:30.320940  4628 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:30.293157  4592 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:30.324179  4592 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:30.205673  4666 leader_election.cc:400] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:30.326603  4592 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:30.215525  4822 ts_tablet_manager.cc:1434] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.208s	user 0.007s	sys 0.018s
I20260709 13:15:30.330195  4668 leader_election.cc:400] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999) with higher term. Message: Invalid argument: T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:30.330983  4668 leader_election.cc:403] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:15:30.331059  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 125522us
I20260709 13:15:30.271576  5434 leader_election.cc:290] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:30.333751  4519 leader_election.cc:304] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:30.333668  4517 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:30.242780  4770 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:30.323997  5485 raft_consensus.cc:697] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:30.345924  4666 leader_election.cc:400] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:30.346956  4666 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:30.256021  4594 leader_election.cc:400] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999) with higher term. Message: Invalid argument: T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3.
I20260709 13:15:30.345331  4741 leader_election.cc:304] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
W20260709 13:15:30.353263  4666 leader_election.cc:343] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:30.353993  4666 leader_election.cc:304] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:30.332020  5391 raft_consensus.cc:2749] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999) with higher term. Message: Invalid argument: T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:30.338711  5451 raft_consensus.cc:2804] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:30.366567  5065 raft_consensus.cc:2749] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:30.367101  4666 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:30.361331  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:30.328315  5514 raft_consensus.cc:2804] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:30.365067  4701 raft_consensus.cc:1275] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:30.340555  5486 consensus_queue.cc:1048] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:15:30.367919  5065 raft_consensus.cc:2804] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:30.383533  4740 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:30.371306  5454 consensus_queue.cc:1048] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:30.340708  5352 consensus_queue.cc:1059] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:15:30.372181  5485 consensus_queue.cc:237] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:30.372658  4822 tablet_bootstrap.cc:492] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:30.341979  5512 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:30.345573  5397 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:30.356541  4594 leader_election.cc:403] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:30.383790  5065 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.392472  5065 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:30.369880  5337 raft_consensus.cc:2804] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:30.360466  5433 raft_consensus.cc:2749] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.360824  5443 raft_consensus.cc:2749] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:30.377185  5472 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
W20260709 13:15:30.393761  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 49837.6us
I20260709 13:15:30.408440  4519 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
W20260709 13:15:30.411909  4740 leader_election.cc:343] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:30.380313  5514 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:30.412544  4740 leader_election.cc:304] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:30.387336  4547 raft_consensus.cc:2468] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:30.386891  5451 raft_consensus.cc:697] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:30.375660  5403 consensus_queue.cc:1048] T 82b6a1da7d62472aa62e9be6aa348f2e P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:15:30.398104  5512 raft_consensus.cc:697] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:30.387634  4773 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:30.390745  5477 raft_consensus.cc:2749] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.397473  5352 consensus_queue.cc:1243] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer 978db1e31e16405182e65e88f3b9bebc log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:15:30.402022  5397 raft_consensus.cc:697] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
W20260709 13:15:30.430462  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 54285.2us
I20260709 13:15:30.415385  5212 raft_consensus.cc:2749] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999) with higher term. Message: Invalid argument: T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3.
I20260709 13:15:30.410463  5337 raft_consensus.cc:697] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:30.402371  5348 consensus_queue.cc:1048] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:15:30.436632  4666 leader_election.cc:400] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:30.421470  5270 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:30.439419  5065 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.417167  5514 raft_consensus.cc:3060] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.428639  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:30.443499  4698 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53246 (request call id 156) took 1254 ms (client timeout 1514 ms). Trace:
I20260709 13:15:30.443670  5451 consensus_queue.cc:237] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:30.443986  4698 rpcz_store.cc:269] 0709 13:15:29.188740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.198286 (+  9546us) service_pool.cc:224] Handling call
0709 13:15:30.420766 (+1222480us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b580029a398. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:15:30.443431 (+ 22665us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8812288}
I20260709 13:15:30.445734  5065 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:30.422665  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:30.449527  5065 consensus_queue.cc:1048] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:15:30.429229  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 173368us
I20260709 13:15:30.450793  5519 raft_consensus.cc:2749] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.466437  4821 tablet_bootstrap.cc:492] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:30.467310  4821 ts_tablet_manager.cc:1403] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.356s	user 0.023s	sys 0.012s
I20260709 13:15:30.449712  5270 raft_consensus.cc:697] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:30.454254  5212 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
W20260709 13:15:30.554291  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 142391us
I20260709 13:15:30.546922  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:30.551246  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:30.518738  5337 consensus_queue.cc:237] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:30.515496  5212 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.561024  4773 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.566318  4740 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:30.509634  5335 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.571374  4821 raft_consensus.cc:359] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.572093  4821 raft_consensus.cc:385] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:30.572404  4821 raft_consensus.cc:740] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:30.573124  4821 consensus_queue.cc:260] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:30.574512  5516 log.cc:927] Time spent T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:15:30.576627  5373 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:30.577209  5373 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.580981  5270 consensus_queue.cc:237] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:30.585328  5522 log.cc:927] Time spent T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:15:30.585719  4519 leader_election.cc:343] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:30.588450  4741 leader_election.cc:304] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:30.589977  4821 ts_tablet_manager.cc:1434] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.122s	user 0.004s	sys 0.003s
I20260709 13:15:30.591180  5259 raft_consensus.cc:2804] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:30.591480  4772 raft_consensus.cc:2468] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:30.593631  5514 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:30.594417  5212 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:30.589996  4822 tablet_bootstrap.cc:654] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.601528  5472 raft_consensus.cc:493] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.605211  5472 raft_consensus.cc:515] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:30.606164  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:30.607376  5524 raft_consensus.cc:2764] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:15:30.608021  5514 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:30.609040  5435 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:30.609653  5435 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.584108  5275 raft_consensus.cc:515] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.612372  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:30.614137  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:30.614464  5259 raft_consensus.cc:493] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.615031  5259 raft_consensus.cc:3060] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:30.622140  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 500880us
I20260709 13:15:30.634697  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:30.635784  4701 raft_consensus.cc:2393] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:30.603300  4740 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:30.604035  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:30.624722  4593 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.513600  5512 consensus_queue.cc:237] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:30.650280  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 47039.9us
I20260709 13:15:30.652349  4547 raft_consensus.cc:1240] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:30.654789  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:30.655821  4695 raft_consensus.cc:2393] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:30.652108  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0059f848a34f47b5a3fe29efe38914b3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:30.657202  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:30.657508  4698 raft_consensus.cc:2393] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:30.658100  4767 raft_consensus.cc:2393] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 3: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:30.652171  5528 raft_consensus.cc:2804] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:30.659404  5528 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.659667  4627 raft_consensus.cc:1240] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:30.659799  5528 raft_consensus.cc:3060] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:30.664652  4821 tablet_bootstrap.cc:492] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512: Bootstrap starting.
I20260709 13:15:30.513305  5397 consensus_queue.cc:237] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:30.636070  5435 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:30.677597  5454 consensus_queue.cc:1059] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:30.634964  5335 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.679167  4546 raft_consensus.cc:2393] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:30.671763  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0c73f991a42e4817d8916fa0a7c30" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:30.675853  5472 leader_election.cc:290] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:30.672921  4773 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:30.689819  5275 leader_election.cc:290] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:30.689738  5212 raft_consensus.cc:493] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:30.769026  4775 raft_consensus.cc:1240] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:30.777823  4770 raft_consensus.cc:1240] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:30.757838  5391 raft_consensus.cc:493] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:30.782924  4593 leader_election.cc:304] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:30.762933  5472 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:30.781934  4549 raft_consensus.cc:1275] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:30.790243  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:30.790887  4773 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:30.791386  4700 raft_consensus.cc:1240] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:15:30.786811  5485 raft_consensus.cc:2749] T f2f0c73f991a42e4817d8916fa0a7c30 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:30.789151  5391 raft_consensus.cc:515] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.796680  5335 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:30.797832  5520 consensus_queue.cc:1059] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:15:30.805531  4698 raft_consensus.cc:1275] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:30.803125  5521 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.802984  5391 leader_election.cc:290] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:30.775133  5212 raft_consensus.cc:515] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:30.818737  4821 tablet_bootstrap.cc:654] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:30.828631  4544 raft_consensus.cc:2468] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:30.830596  5212 leader_election.cc:290] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:30.814154  5443 raft_consensus.cc:3055] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:30.803237  5472 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:30.834491  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:30.842825  5513 log.cc:927] Time spent T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.477s	user 0.002s	sys 0.000s
I20260709 13:15:30.845142  4548 raft_consensus.cc:1240] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:30.847865  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:30.849076  4625 raft_consensus.cc:2393] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 1: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:30.849399  5524 consensus_queue.cc:1048] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:30.824092  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:30.853488  4700 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
W20260709 13:15:30.856321  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 80087.1us
I20260709 13:15:30.856765  4741 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:30.861819  5065 consensus_queue.cc:1059] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:30.837610  5443 raft_consensus.cc:740] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:30.852890  5457 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:30.883630  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:30.884603  4550 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:30.885639  5443 consensus_queue.cc:260] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:30.886090  4544 raft_consensus.cc:1275] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:30.856168  5212 raft_consensus.cc:493] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:30.824072  5521 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:30.869112  5524 raft_consensus.cc:2804] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:30.883940  4548 raft_consensus.cc:1240] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:15:30.896347  5373 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:30.897689  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:30.897475  5212 raft_consensus.cc:515] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:30.834681  5259 raft_consensus.cc:515] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:30.882189  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:30.900399  5521 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:30.921504  5472 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:30.921938  5212 leader_election.cc:290] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:30.922338  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:30.905682  5524 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.925839  4693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53246 (request call id 171) took 1198 ms. Trace:
I20260709 13:15:30.923530  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0c53efa0ee4dfbba4c3f1922027f50" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:30.926296  4693 rpcz_store.cc:276] 0709 13:15:29.727711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.940843 (+213132us) service_pool.cc:224] Handling call
0709 13:15:29.941309 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:30.290849 (+349540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.925660 (+634811us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54006c93d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:30.925794 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649985408}
I20260709 13:15:30.926496  4592 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:30.929112  4592 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:30.880038  5523 consensus_queue.cc:1059] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:15:30.930423  4592 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:30.935658  4592 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:30.927327  5524 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:30.931114  5212 raft_consensus.cc:2749] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:15:30.956293  4668 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 90593.9us
I20260709 13:15:30.962325  4822 tablet_bootstrap.cc:492] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:30.963443  4822 ts_tablet_manager.cc:1403] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.614s	user 0.023s	sys 0.033s
I20260709 13:15:30.942760  4594 leader_election.cc:304] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:30.934401  5472 raft_consensus.cc:2804] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:30.940668  5514 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:30.956152  5523 raft_consensus.cc:3055] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:30.974198  5065 raft_consensus.cc:493] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:30.981880  5065 raft_consensus.cc:515] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:30.938993  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0c53efa0ee4dfbba4c3f1922027f50" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:30.968938  5212 raft_consensus.cc:2804] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:30.970468  5451 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:30.939483  5528 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:30.988440  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 52907.4us
I20260709 13:15:30.979441  5523 raft_consensus.cc:740] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
W20260709 13:15:30.992938  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 118644us
I20260709 13:15:31.002357  5065 leader_election.cc:290] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:31.020336  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.025575  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.027338  4668 leader_election.cc:304] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:31.027850  5547 raft_consensus.cc:493] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.028342  5065 raft_consensus.cc:2749] T 290a017cf81042b6a4e64319563864c5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:31.032506  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 142767us
I20260709 13:15:31.037875  5547 raft_consensus.cc:515] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.038390  5545 raft_consensus.cc:2749] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:31.050614  5547 leader_election.cc:290] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
W20260709 13:15:31.036118  5466 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.233s	user 0.005s	sys 0.005s
I20260709 13:15:31.051362  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3079151438004a9a975fdbe1b5b54378" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:31.051515  5466 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.251s	user 0.005s	sys 0.005s
I20260709 13:15:31.054224  4548 raft_consensus.cc:2468] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 0.
I20260709 13:15:31.054792  5259 leader_election.cc:290] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:31.034150  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:30.995651  5451 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:31.063176  5054 raft_consensus.cc:3055] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:31.063637  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:31.073578  4821 tablet_bootstrap.cc:492] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512: No bootstrap required, opened a new log
I20260709 13:15:31.074184  5054 raft_consensus.cc:740] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:31.074747  4821 ts_tablet_manager.cc:1403] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512: Time spent bootstrapping tablet: real 0.410s	user 0.016s	sys 0.004s
I20260709 13:15:31.059080  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:31.076294  4698 raft_consensus.cc:2468] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
W20260709 13:15:31.076782  5538 log.cc:927] Time spent T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 13:15:31.078896  5530 log.cc:927] Time spent T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.286s	user 0.002s	sys 0.000s
I20260709 13:15:31.083292  4774 raft_consensus.cc:2468] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:31.088979  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:31.004516  5523 consensus_queue.cc:260] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.092051  5528 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:31.093879  4822 raft_consensus.cc:359] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.094635  4822 raft_consensus.cc:385] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:31.094913  4822 raft_consensus.cc:740] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:31.075554  4695 raft_consensus.cc:3060] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:31.095885  4740 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:31.096174  4668 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:31.089934  4625 raft_consensus.cc:2393] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 3: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:31.075551  4593 leader_election.cc:304] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:31.096911  5065 raft_consensus.cc:2804] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:31.098197  5065 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.098541  5065 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.060461  5553 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.058005  5373 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:31.056254  5434 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:31.103323  5054 consensus_queue.cc:260] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.056496  5472 raft_consensus.cc:697] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:31.101892  5259 raft_consensus.cc:2749] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:31.105468  4770 raft_consensus.cc:1275] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.105739  4773 raft_consensus.cc:2468] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 5.
I20260709 13:15:31.106920  4822 consensus_queue.cc:260] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.108630  5054 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.111792  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.121603  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.127430  5065 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.117993  5477 consensus_queue.cc:1048] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:15:31.130239  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:31.130880  4698 raft_consensus.cc:1240] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:31.131382  5477 consensus_queue.cc:1048] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:31.132439  4776 raft_consensus.cc:1240] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:31.133556  5451 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:31.133689  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.134917  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:31.135543  4548 raft_consensus.cc:3060] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.137234  4546 raft_consensus.cc:1240] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:31.138389  5505 log.cc:927] Time spent T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 1.164s	user 0.000s	sys 0.006s
W20260709 13:15:31.141403  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 65923us
I20260709 13:15:31.147017  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:31.147886  4701 raft_consensus.cc:2393] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 3: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:31.155630  4695 raft_consensus.cc:2468] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:31.114820  5212 raft_consensus.cc:697] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:31.159432  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:31.159685  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:31.161471  4625 raft_consensus.cc:3060] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:31.163640  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
W20260709 13:15:31.171908  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 60533.8us
I20260709 13:15:31.173542  5065 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:31.142095  5403 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.117043  5434 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:31.181241  4822 ts_tablet_manager.cc:1434] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.217s	user 0.005s	sys 0.003s
I20260709 13:15:31.202586  4740 leader_election.cc:304] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:31.206384  4668 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:31.116241  5553 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:31.207609  5065 raft_consensus.cc:2804] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:31.208374  5065 raft_consensus.cc:697] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [term 5 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:31.212332  5545 consensus_queue.cc:1048] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:31.221172  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:31.223097  5456 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.223567  4774 raft_consensus.cc:1240] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:31.223716  5456 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:31.224875  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:31.225561  4776 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.231349  5553 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:31.232038  5373 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.205662  4771 raft_consensus.cc:1275] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:15:31.233091  4771 pending_rounds.cc:85] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512: Aborting all ops after (but not including) 2
I20260709 13:15:31.205276  4696 raft_consensus.cc:3060] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:31.233579  4771 pending_rounds.cc:107] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:15:31.234251  4771 raft_consensus.cc:2889] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:31.232682  5373 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.207355  5403 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:31.198714  5549 raft_consensus.cc:2749] T 3a0c53efa0ee4dfbba4c3f1922027f50 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:31.242170  5457 consensus_queue.cc:1059] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:15:31.233057  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:31.248883  4775 raft_consensus.cc:1275] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:15:31.249595  4775 pending_rounds.cc:85] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512: Aborting all ops after (but not including) 1
I20260709 13:15:31.249948  4775 pending_rounds.cc:107] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:15:31.250275  4775 raft_consensus.cc:2889] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:31.250438  5391 raft_consensus.cc:3055] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:31.251003  5391 raft_consensus.cc:740] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:31.250698  5065 consensus_queue.cc:237] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.252153  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:31.262539  4593 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.251962  5391 consensus_queue.cc:260] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.177815  5512 raft_consensus.cc:493] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:31.176517  5451 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
W20260709 13:15:31.326380  4667 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.283517  5509 log.cc:927] Time spent T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:15:31.331067  4695 raft_consensus.cc:1275] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.210562  5259 raft_consensus.cc:2804] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:31.327883  4821 raft_consensus.cc:359] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.333931  4821 raft_consensus.cc:385] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:31.335960  4821 raft_consensus.cc:740] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Initialized, Role: FOLLOWER
I20260709 13:15:31.182766  5551 consensus_queue.cc:1059] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.152s
I20260709 13:15:31.202122  5524 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.293177  5419 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.332427  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.333280  5434 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:31.334311  5259 raft_consensus.cc:493] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.372769  5553 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:31.373433  5553 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:31.374280  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:31.374622  4741 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:31.375504  5553 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:31.375902  4741 leader_election.cc:304] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:31.375521  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:31.377028  4770 raft_consensus.cc:2393] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 3: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:31.377887  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:31.378074  5520 raft_consensus.cc:2804] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:31.378564  5520 raft_consensus.cc:697] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:31.378665  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.379482  5520 consensus_queue.cc:237] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:31.377930  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:31.379774  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3079151438004a9a975fdbe1b5b54378" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:31.380336  4774 raft_consensus.cc:1240] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:31.379814  4740 leader_election.cc:343] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:31.381223  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:31.382557  4696 raft_consensus.cc:2468] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:31.352464  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:31.384989  4694 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:31.386348  5403 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:31.364832  5259 raft_consensus.cc:3060] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:31.387405  5486 consensus_queue.cc:1048] T 5b40eb24527a4f8bbc5da80339a74c26 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:31.389046  4593 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:31.200052  5472 consensus_queue.cc:237] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.336323  5511 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:31.197834  5212 consensus_queue.cc:237] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:31.339359  5391 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:31.392103  4740 leader_election.cc:343] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:31.333189  5451 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:31.396407  5524 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:31.397150  5451 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:31.354437  5419 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:31.335964  5512 raft_consensus.cc:515] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:31.401124  5419 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:31.401822  4702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53246 (request call id 177) took 1223 ms. Trace:
I20260709 13:15:31.402626  5512 leader_election.cc:290] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:31.359645  5373 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:31.404954  4775 raft_consensus.cc:2368] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for earlier term 3. Current term is 4.
I20260709 13:15:31.405222  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:31.406487  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:31.365612  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:31.418351  4668 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.356984  5546 consensus_queue.cc:1048] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.042s
I20260709 13:15:31.394007  5553 raft_consensus.cc:2804] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:31.400401  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.409372  5456 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:31.356458  4821 consensus_queue.cc:260] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.418043  5541 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.467841  5553 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.443790  4699 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:31.472741  4770 raft_consensus.cc:3060] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:31.402261  4702 rpcz_store.cc:276] 0709 13:15:30.178573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.195531 (+ 16958us) service_pool.cc:224] Handling call
0709 13:15:31.400994 (+1205463us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b580029a398. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:31.401227 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:31.401468 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:31.401500 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:31.401543 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:31.401590 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:31.401653 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:31.401763 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30947072}
I20260709 13:15:31.476473  4771 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 3.
I20260709 13:15:31.443470  5373 consensus_queue.cc:1048] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 13:15:31.466866  5397 consensus_queue.cc:1059] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:15:31.472841  4776 raft_consensus.cc:2468] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 3.
I20260709 13:15:31.497099  4821 ts_tablet_manager.cc:1434] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512: Time spent starting tablet: real 0.422s	user 0.015s	sys 0.004s
I20260709 13:15:31.449004  5486 consensus_queue.cc:1048] T 779c16a6075e4893824178771ade71a1 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:15:31.492498  5553 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.485718  5512 raft_consensus.cc:493] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:31.484754  5541 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:31.507997  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 157857us
I20260709 13:15:31.504643  5528 raft_consensus.cc:2764] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:15:31.504240  5451 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:31.505970  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.531867  4668 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:31.512053  5519 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.536921  5065 raft_consensus.cc:2804] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:31.539748  5065 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.540118  5065 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.531502  4774 raft_consensus.cc:1275] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.529299  5394 raft_consensus.cc:3055] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:31.531507  5551 raft_consensus.cc:493] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:31.551210  5283 consensus_queue.cc:1048] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:15:31.546052  4768 raft_consensus.cc:2376] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3. Re-sending same reply.
I20260709 13:15:31.516830  5512 raft_consensus.cc:515] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.538483  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.572542  5065 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:31.556294  5394 raft_consensus.cc:740] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
W20260709 13:15:31.578048  4768 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57066 (request call id 163) took 1366 ms (client timeout 1788 ms). Trace:
W20260709 13:15:31.584575  4768 rpcz_store.cc:269] 0709 13:15:30.207634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.208865 (+  1231us) service_pool.cc:224] Handling call
0709 13:15:31.498437 (+1289572us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b58009fb798. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:15:31.573546 (+ 75109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45241216}
W20260709 13:15:31.585237  5568 log.cc:927] Time spent T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:15:31.586007  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.587393  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:31.546335  5519 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.588775  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:31.590580  5519 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
W20260709 13:15:31.598953  5514 consensus_meta.cc:287] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.002s
I20260709 13:15:31.599493  5514 raft_consensus.cc:2749] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:31.604631  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0059f848a34f47b5a3fe29efe38914b3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.605423  4626 raft_consensus.cc:2468] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
W20260709 13:15:31.608769  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 78463.6us
I20260709 13:15:31.539351  5451 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.610669  4594 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:31.614956  4594 leader_election.cc:304] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:31.615983  5546 raft_consensus.cc:3060] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:31.612932  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 163333us
I20260709 13:15:31.571487  5554 raft_consensus.cc:493] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:31.626510  5065 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:31.607273  4548 raft_consensus.cc:2468] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:31.627018  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:31.632835  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:31.633535  4546 raft_consensus.cc:1240] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:31.633893  5512 leader_election.cc:290] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:31.634838  4774 raft_consensus.cc:1275] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.635973  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:31.636268  5547 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.636572  4774 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.636837  5547 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.638041  4767 raft_consensus.cc:1275] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.639266  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.645701  5451 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:31.647265  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.631384  5554 raft_consensus.cc:515] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:31.627471  5394 consensus_queue.cc:260] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.571451  5551 raft_consensus.cc:515] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.640060  5541 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:31.620639  5514 raft_consensus.cc:2749] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:31.633527  4628 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.657516  5259 raft_consensus.cc:515] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.666265  5554 leader_election.cc:290] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:31.669051  4619 raft_consensus.cc:1275] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:31.670418  5259 leader_election.cc:290] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:31.671654  5443 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.665714  4767 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 4.
I20260709 13:15:31.673770  4767 raft_consensus.cc:1275] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.675310  4593 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:31.674532  4770 raft_consensus.cc:2468] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:31.676976  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:31.677562  5457 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:31.677870  5259 raft_consensus.cc:493] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:31.661801  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.686200  4593 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:31.663574  5547 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:31.700326  4628 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 4.
W20260709 13:15:31.706804  4668 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.687772  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:31.709250  4666 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:31.702033  5554 raft_consensus.cc:2749] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:31.708402  5511 raft_consensus.cc:2764] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:15:31.711589  4702 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:31.689414  5259 raft_consensus.cc:515] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.721256  5259 leader_election.cc:290] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:31.726491  4702 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:31.729800  4627 raft_consensus.cc:1240] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:31.704500  4771 raft_consensus.cc:2468] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:31.712965  5541 raft_consensus.cc:2804] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:31.732553  4767 raft_consensus.cc:1240] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:31.701465  5457 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:31.737260  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:31.737752  4769 raft_consensus.cc:1240] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 2. Current term is 3. Ops: [2.1-2.1]
I20260709 13:15:31.738756  5553 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:31.739199  5551 leader_election.cc:290] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:31.744005  5472 consensus_queue.cc:1048] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:31.750910  4594 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:31.766358  4772 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57060 (request call id 154) took 1489 ms. Trace:
I20260709 13:15:31.746840  4822 tablet_bootstrap.cc:492] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:31.766927  4772 rpcz_store.cc:276] 0709 13:15:30.277312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.287138 (+  9826us) service_pool.cc:224] Handling call
0709 13:15:31.766195 (+1479057us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b5400383454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:31.766303 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30429568}
I20260709 13:15:31.746850  4621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48792 (request call id 146) took 1102 ms. Trace:
I20260709 13:15:31.747412  5541 raft_consensus.cc:697] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 4 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:31.769711  5551 raft_consensus.cc:2804] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:31.770157  5551 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:31.770542  5551 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:15:31.756153  4665 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.771967  4627 raft_consensus.cc:1275] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.770182  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.773043  4547 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:31.754678  4769 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57066 (request call id 170) took 1243 ms. Trace:
I20260709 13:15:31.773801  5511 consensus_queue.cc:1059] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:31.760301  5335 consensus_queue.cc:1048] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:15:31.773556  4769 rpcz_store.cc:276] 0709 13:15:30.511410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.516134 (+  4724us) service_pool.cc:224] Handling call
0709 13:15:31.737151 (+1221017us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b58009fb798. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:31.737425 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:31.754323 (+ 16898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:31.754608 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7260928}
I20260709 13:15:31.756038  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:31.751019  5457 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:31.749177  4771 raft_consensus.cc:1275] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:31.782040  5553 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:31.770550  5541 consensus_queue.cc:237] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:31.753491  4774 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 4.
I20260709 13:15:31.756091  5546 raft_consensus.cc:2749] T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:31.769419  4621 rpcz_store.cc:276] 0709 13:15:30.644055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.645525 (+  1470us) service_pool.cc:224] Handling call
0709 13:15:30.959429 (+313904us) spinlock_profiling.cc:243] Waited 9.28 ms on lock 0x7b5800003f98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:31.745845 (+786416us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b5800003f98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552e9c 00007f59cb552a5b 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:31.746099 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:31.746355 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:31.746385 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:31.746550 (+   165us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:31.746596 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:31.746669 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:31.746790 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31592704}
I20260709 13:15:31.791970  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:31.801340  4622 raft_consensus.cc:2376] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 4. Re-sending same reply.
I20260709 13:15:31.744668  4624 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:31.786476  5551 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:31.794567  5523 raft_consensus.cc:3060] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.785318  4625 raft_consensus.cc:2393] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 3: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:31.795379  4822 tablet_bootstrap.cc:654] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:31.853891  4668 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.873447  5583 log.cc:927] Time spent T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:15:31.877144  5560 log.cc:927] Time spent T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.394s	user 0.000s	sys 0.003s
I20260709 13:15:31.869522  5488 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.886283  5551 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:31.889603  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:31.890509  5488 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.892357  4701 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:31.893329  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
W20260709 13:15:31.899183  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 311627us
I20260709 13:15:31.900858  4592 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:31.905472  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 295985us
I20260709 13:15:31.908489  5434 consensus_queue.cc:1048] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:15:31.910658  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 285248us
I20260709 13:15:31.911666  4518 leader_election.cc:304] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:31.915066  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:31.917275  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.918318  5579 raft_consensus.cc:2804] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:31.918908  5579 raft_consensus.cc:697] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:31.920100  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.921703  4666 leader_election.cc:304] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:31.916536  4553 raft_consensus.cc:2393] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:31.927500  4695 raft_consensus.cc:1275] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:31.914959  4517 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:31.906208  5546 raft_consensus.cc:2749] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:31.931519  4552 raft_consensus.cc:1275] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.932421  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:31.933588  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:31.934257  4545 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:31.915220  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.937053  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.930130  4666 leader_election.cc:304] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:31.942279  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.944603  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.942930  4553 raft_consensus.cc:2468] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:31.907253  5511 consensus_queue.cc:1059] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:15:31.948278  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:31.950049  4822 tablet_bootstrap.cc:492] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:31.950583  5460 consensus_queue.cc:1048] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:31.950927  5372 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.939605  5525 raft_consensus.cc:2749] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:31.951579  5372 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.953320  5488 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:31.953598  4702 raft_consensus.cc:1275] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:31.954016  5372 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:31.954936  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:31.950615  4822 ts_tablet_manager.cc:1403] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.204s	user 0.014s	sys 0.008s
I20260709 13:15:31.957028  4700 raft_consensus.cc:1240] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 2. Current term is 3. Ops: [2.1-2.1]
W20260709 13:15:31.957623  4666 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.83.194:35607 timed out after 1.770s (SENT) [suppressed 7 similar messages]
I20260709 13:15:31.957122  5579 consensus_queue.cc:237] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:31.958366  5397 raft_consensus.cc:2804] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:32.006426  4701 raft_consensus.cc:1275] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:15:32.007074  4701 pending_rounds.cc:85] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a: Aborting all ops after (but not including) 1
I20260709 13:15:32.007351  4701 pending_rounds.cc:107] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:15:32.007639  4701 raft_consensus.cc:2889] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:31.939136  5541 raft_consensus.cc:2749] T 179cf9478a1642929379f46e4311d9a5 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:32.132822  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 175214us
I20260709 13:15:32.131907  5396 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:32.135207  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 176525us
W20260709 13:15:32.134609  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 178346us
I20260709 13:15:32.131076  5397 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.132443  5546 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.920356  5486 consensus_queue.cc:1059] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:15:32.147903  4666 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:32.139083  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:32.144457  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:32.152689  5397 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.155157  5565 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:32.182524  4666 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:32.188391  4822 raft_consensus.cc:359] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:32.189248  4822 raft_consensus.cc:385] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:32.189541  4822 raft_consensus.cc:740] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:32.171080  5434 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.152321  5396 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:32.161181  5546 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:32.186625  5541 raft_consensus.cc:2804] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:32.217593  4822 consensus_queue.cc:260] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:32.200189  5443 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.204155  5545 raft_consensus.cc:3055] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:32.202919  5454 raft_consensus.cc:493] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:32.227978  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:32.205026  4620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48792 (request call id 145) took 1558 ms. Trace:
I20260709 13:15:32.260432  4822 ts_tablet_manager.cc:1434] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.305s	user 0.009s	sys 0.000s
I20260709 13:15:32.224943  5541 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.208104  5434 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:32.201401  5565 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:32.262061  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 120702us
I20260709 13:15:32.232573  5545 raft_consensus.cc:740] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:32.269351  5541 raft_consensus.cc:3060] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.284188  4519 leader_election.cc:400] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Vote denied by peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999) with higher term. Message: Invalid argument: T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for earlier term 3. Current term is 4.
I20260709 13:15:32.284667  4519 leader_election.cc:403] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:32.266525  4770 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 3.
I20260709 13:15:32.289119  4518 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:32.288769  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 327790us
I20260709 13:15:32.270996  4775 raft_consensus.cc:2468] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:32.254329  5454 raft_consensus.cc:515] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:32.288565  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:32.282799  5283 consensus_queue.cc:1048] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:15:32.259357  4620 rpcz_store.cc:276] 0709 13:15:30.643752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.645867 (+  2115us) service_pool.cc:224] Handling call
0709 13:15:32.132343 (+1486476us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b5800003f98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:15:32.202029 (+ 69686us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29623168}
I20260709 13:15:32.319949  5564 consensus_queue.cc:1059] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:32.331192  4594 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:32.289711  5578 consensus_queue.cc:1048] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:15:32.290894  5280 consensus_queue.cc:1048] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:15:32.308861  5373 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:15:32.339551  4822 tablet_bootstrap.cc:492] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
I20260709 13:15:32.324540  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:32.356858  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 58153.1us
I20260709 13:15:32.335790  5511 raft_consensus.cc:2804] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:32.340483  5457 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:32.367254  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 78215.5us
I20260709 13:15:32.345832  5596 raft_consensus.cc:2804] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:15:32.372073  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 75670.3us
I20260709 13:15:32.378376  4517 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:32.312005  5545 consensus_queue.cc:260] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:32.390022  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:32.396354  4740 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:32.349680  5602 consensus_queue.cc:1048] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:15:32.369820  5511 raft_consensus.cc:697] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:32.396886  4772 raft_consensus.cc:1275] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:32.375288  5603 raft_consensus.cc:493] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:32.378057  5283 raft_consensus.cc:493] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:32.380898  5396 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:32.400151  5546 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:32.402045  5571 raft_consensus.cc:2804] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.408354  4666 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:32.490341  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 82094us
W20260709 13:15:32.493129  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 70347.8us
I20260709 13:15:32.498025  4551 raft_consensus.cc:1240] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:32.370817  5597 consensus_queue.cc:1048] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:15:32.371351  5457 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:32.412038  5454 leader_election.cc:290] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:32.418047  5435 raft_consensus.cc:3055] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:32.488384  5551 raft_consensus.cc:493] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.422385  5283 raft_consensus.cc:515] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:32.534677  4549 raft_consensus.cc:1275] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:32.547066  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:32.560178  5551 raft_consensus.cc:515] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:32.508167  5434 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:32.538736  5541 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:32.563251  5551 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:32.563742  5541 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:32.534793  5607 consensus_queue.cc:1059] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
W20260709 13:15:32.542044  4545 consensus_meta.cc:287] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.000s
I20260709 13:15:32.565016  4545 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:32.541913  4592 leader_election.cc:304] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:32.410619  5593 log.cc:927] Time spent T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:15:32.522519  5435 raft_consensus.cc:740] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:32.574426  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:32.575197  4822 tablet_bootstrap.cc:654] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:32.493266  5600 raft_consensus.cc:2749] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:32.503969  5565 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
W20260709 13:15:32.507619  5590 log.cc:927] Time spent T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 13:15:32.504812  5571 raft_consensus.cc:697] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:32.505041  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:32.579118  4592 leader_election.cc:304] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:32.536274  5280 raft_consensus.cc:2749] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:32.410368  4624 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48792 (request call id 142) took 1767 ms (client timeout 1770 ms). Trace:
I20260709 13:15:32.604242  4771 raft_consensus.cc:2468] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 2.
I20260709 13:15:32.606187  4592 leader_election.cc:304] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:32.573755  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:32.558842  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:32.563764  5457 leader_election.cc:290] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:32.524915  5596 raft_consensus.cc:697] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:32.600106  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:32.576690  5551 raft_consensus.cc:2749] T ae70b5bde8ec47b1931014ebfe03ea2a P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:32.600044  5546 consensus_queue.cc:1048] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:15:32.413378  5603 raft_consensus.cc:515] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:32.609215  5523 consensus_meta.cc:287] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a: Time spent flushing consensus metadata: real 0.742s	user 0.004s	sys 0.009s
I20260709 13:15:32.607553  5472 raft_consensus.cc:2749] T 3b05eadf0e794d249a3ee1e016745baf P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:32.623642  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 224220us
I20260709 13:15:32.612059  5553 raft_consensus.cc:2749] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:32.558066  5511 consensus_queue.cc:237] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:32.641484  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:32.641811  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:32.645581  4741 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:32.675766  4693 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:32.643131  5394 raft_consensus.cc:3060] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:15:32.678211  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 66648.5us
I20260709 13:15:32.679579  5519 raft_consensus.cc:2804] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:32.684183  5606 raft_consensus.cc:493] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:32.604522  5571 consensus_queue.cc:237] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:32.684906  5606 raft_consensus.cc:515] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:32.687129  5606 leader_election.cc:290] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:32.689471  5600 raft_consensus.cc:493] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:32.691109  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:32.640445  4698 raft_consensus.cc:1275] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:15:32.679047  4592 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:32.637159  5283 leader_election.cc:290] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:32.643226  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:32.636011  5603 leader_election.cc:290] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:32.714785  4519 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:32.715967  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:32.717131  4668 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:32.631101  5551 consensus_queue.cc:1048] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.082s
I20260709 13:15:32.640115  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:32.680387  5611 raft_consensus.cc:493] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:32.731279  4668 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:32.717442  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:32.711514  4698 pending_rounds.cc:85] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a: Aborting all ops after (but not including) 1
I20260709 13:15:32.618361  5435 consensus_queue.cc:260] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:32.697438  5525 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:32.728106  5514 raft_consensus.cc:2804] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.697618  5610 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.719065  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:32.743450  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 44970.4us
W20260709 13:15:32.732052  5397 consensus_meta.cc:287] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc: Time spent flushing consensus metadata: real 0.533s	user 0.004s	sys 0.007s
I20260709 13:15:32.735909  5488 consensus_queue.cc:1048] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:15:32.744561  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:32.741183  5541 raft_consensus.cc:2804] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:32.759721  5454 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:32.628087  5612 consensus_queue.cc:1048] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 13:15:32.677930  5466 consensus_queue.cc:1048] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.047s
I20260709 13:15:32.720009  5565 raft_consensus.cc:493] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.762394  5283 raft_consensus.cc:2804] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:32.693399  5519 raft_consensus.cc:697] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:32.764942  4668 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:32.729624  5396 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:32.692830  5600 raft_consensus.cc:515] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:32.769302  4741 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
W20260709 13:15:32.772681  5443 consensus_meta.cc:287] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a: Time spent flushing consensus metadata: real 0.540s	user 0.006s	sys 0.006s
I20260709 13:15:32.762972  4698 pending_rounds.cc:107] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260709 13:15:32.766896  4516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.765952  5599 consensus_queue.cc:1048] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
W20260709 13:15:32.777667  5608 log.cc:927] Time spent T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.266s	user 0.004s	sys 0.000s
W20260709 13:15:32.779922  5615 log.cc:927] Time spent T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:15:32.713690  5605 consensus_queue.cc:1048] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:15:32.783785  4627 raft_consensus.cc:3060] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:32.782058  5373 raft_consensus.cc:2749] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999) with higher term. Message: Invalid argument: T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for earlier term 3. Current term is 4.
W20260709 13:15:32.609603  4624 rpcz_store.cc:269] 0709 13:15:30.642733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.645396 (+  2663us) service_pool.cc:224] Handling call
0709 13:15:31.220919 (+575523us) spinlock_profiling.cc:243] Waited 7.65 ms on lock 0x7b5800003f98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:15:32.410156 (+1189237us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400659354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:32.410292 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1077834240}
W20260709 13:15:32.791213  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 76522us
I20260709 13:15:32.785725  5514 raft_consensus.cc:697] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:32.797326  4698 raft_consensus.cc:2889] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:32.837556  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59401c42fe8430aaed4b49ef7df0372" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:32.693078  5596 consensus_queue.cc:237] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:32.804008  5552 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:32.846920  5552 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.847376  5435 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.847003  5514 consensus_queue.cc:237] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:32.850734  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:32.760108  5611 raft_consensus.cc:515] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:32.855118  5611 leader_election.cc:290] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:32.789053  5454 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.855921  5454 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:32.806926  5601 raft_consensus.cc:2804] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:32.857061  5601 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:32.857349  5598 log.cc:927] Time spent T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.345s	user 0.000s	sys 0.003s
I20260709 13:15:32.857518  5601 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:32.782958  5397 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:32.795661  5541 raft_consensus.cc:697] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 3 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:32.840111  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:32.768114  5525 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:32.845742  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae70b5bde8ec47b1931014ebfe03ea2a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:32.845606  5519 consensus_queue.cc:237] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:32.847484  5552 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:32.852892  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:32.794049  5283 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.799170  5443 raft_consensus.cc:2749] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:32.862764  4666 leader_election.cc:304] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:32.797461  5546 consensus_queue.cc:1048] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.065s
I20260709 13:15:32.799535  5396 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:32.795538  5565 raft_consensus.cc:515] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:32.782714  5610 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:32.890640  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:32.877957  5457 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.885603  5283 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:32.907888  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:32.915736  4519 leader_election.cc:304] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:32.898923  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:32.907104  5511 raft_consensus.cc:493] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:32.911063  5457 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:32.930922  4698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53246 (request call id 187) took 1785 ms. Trace:
I20260709 13:15:32.933127  5511 raft_consensus.cc:515] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:32.917140  4544 raft_consensus.cc:1240] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:32.917971  5599 raft_consensus.cc:2749] T c59401c42fe8430aaed4b49ef7df0372 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:32.956506  5624 log.cc:927] Time spent T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:15:32.938812  4698 rpcz_store.cc:276] 0709 13:15:31.145591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.152555 (+  6964us) service_pool.cc:224] Handling call
0709 13:15:31.160318 (+  7763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.872811 (+1712493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.930625 (+ 57814us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540005a5d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:32.930848 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3362897792}
I20260709 13:15:32.920250  5443 raft_consensus.cc:493] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:32.924696  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:32.973798  5623 raft_consensus.cc:2749] T 672896bdac10415b85afc981d3c6eac9 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:32.972842  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 57226.7us
I20260709 13:15:32.951186  5466 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.988220  5466 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:32.988220  5554 raft_consensus.cc:493] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:32.982801  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:33.013893  5525 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:33.016400  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3079151438004a9a975fdbe1b5b54378" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:33.017292  5397 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:33.017685  4822 tablet_bootstrap.cc:492] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:33.018188  4822 ts_tablet_manager.cc:1403] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.679s	user 0.035s	sys 0.015s
I20260709 13:15:32.997994  5554 raft_consensus.cc:515] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:33.018489  4767 raft_consensus.cc:1240] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [term 3 LEADER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:33.023610  5524 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.982801  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:32.976986  5443 raft_consensus.cc:515] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:33.017246  4548 raft_consensus.cc:3060] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:32.976025  5523 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:33.025113  4698 raft_consensus.cc:2410] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:33.028424  5443 leader_election.cc:290] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:33.050483  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3079151438004a9a975fdbe1b5b54378" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:32.926779  5541 consensus_queue.cc:237] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:33.045845  5523 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:33.102360  4519 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.106760  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 241341us
I20260709 13:15:33.024209  5524 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.108408  5212 raft_consensus.cc:493] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:33.123117  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 262652us
I20260709 13:15:33.120299  4593 leader_election.cc:304] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:33.127429  4593 leader_election.cc:304] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:33.128324  4695 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:33.131585  4593 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:33.129791  4822 raft_consensus.cc:359] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.135555  4822 raft_consensus.cc:385] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.135874  4822 raft_consensus.cc:740] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.108970  5521 raft_consensus.cc:493] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:33.138942  4822 consensus_queue.cc:260] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.138875  4519 leader_election.cc:304] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:33.161288  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:33.172400  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 309729us
I20260709 13:15:33.134616  5553 raft_consensus.cc:2749] T bd4347b508bf4002a95a198ba9bcf072 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.129446  5551 raft_consensus.cc:2749] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:33.191627  4519 leader_election.cc:304] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
W20260709 13:15:33.289130  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 97590.9us
I20260709 13:15:33.124133  5212 raft_consensus.cc:515] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:33.151041  5616 raft_consensus.cc:2749] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:15:33.318807  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 169950us
W20260709 13:15:33.334659  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 44824.3us
I20260709 13:15:33.311867  5485 raft_consensus.cc:493] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.337272  5485 raft_consensus.cc:515] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.339583  5485 leader_election.cc:290] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.184108  5612 raft_consensus.cc:2749] T ae70b5bde8ec47b1931014ebfe03ea2a P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:33.369143  4593 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.154240  5521 raft_consensus.cc:515] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.310887  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:33.331431  5580 raft_consensus.cc:2749] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:33.398718  4822 ts_tablet_manager.cc:1434] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.380s	user 0.029s	sys 0.021s
I20260709 13:15:33.179713  5621 consensus_queue.cc:1048] T 92fea640a4c24068bf9e2805a83ee8b6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.195s
I20260709 13:15:33.343428  5466 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:33.309562  5600 leader_election.cc:290] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.348263  5514 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:33.372644  5396 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:33.372867  5616 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:33.345468  5551 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:33.415973  5551 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:33.416815  5521 leader_election.cc:290] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:33.402820  5546 raft_consensus.cc:493] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:33.418524  5546 raft_consensus.cc:515] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:33.419560  4822 tablet_bootstrap.cc:492] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82: Bootstrap starting.
W20260709 13:15:33.407006  5394 consensus_meta.cc:287] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc: Time spent flushing consensus metadata: real 0.727s	user 0.013s	sys 0.008s
I20260709 13:15:33.392064  5554 leader_election.cc:290] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:33.391057  5610 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:33.422976  5280 consensus_queue.cc:1048] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:33.419747  5514 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:33.414440  4546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51880 (request call id 178) took 1142 ms. Trace:
I20260709 13:15:33.425937  5632 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.424674  4546 rpcz_store.cc:276] 0709 13:15:32.268388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.269709 (+  1321us) service_pool.cc:224] Handling call
0709 13:15:32.781563 (+511854us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b58002f2298. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552eee 0000000000375939 00007f59d35c61e0 00007f59d368f0c4 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:32.781735 (+   172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.781939 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:15:32.882484 (+100545us) log.cc:844] Serialized 14 byte log entry
0709 13:15:33.372521 (+490037us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:33.378492 (+  5971us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:33.389169 (+ 10677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.402511 (+ 13342us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.410816 (+  8305us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":165745,"prepare.queue_time_us":157649,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":39,"raft.queue_time_us":156128,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":9918,"spinlock_wait_cycles":23146240,"thread_start_us":372510,"threads_started":2,"wal-append.queue_time_us":499883}
I20260709 13:15:33.426159  5523 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:33.426532  5632 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:33.427320  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:33.410552  5565 leader_election.cc:290] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:33.450898  4547 raft_consensus.cc:1275] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:15:33.451423  4547 pending_rounds.cc:85] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc: Aborting all ops after (but not including) 2
I20260709 13:15:33.451682  4547 pending_rounds.cc:107] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:15:33.451958  4547 raft_consensus.cc:2889] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:33.444506  5524 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:33.413336  5511 leader_election.cc:290] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
W20260709 13:15:33.421491  5454 consensus_meta.cc:287] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a: Time spent flushing consensus metadata: real 0.565s	user 0.005s	sys 0.008s
I20260709 13:15:33.454507  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:33.467850  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:33.432364  5641 log.cc:927] Time spent T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:15:33.432529  5633 log.cc:927] Time spent T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:15:33.435519  5644 log.cc:927] Time spent T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
W20260709 13:15:33.435956  5601 consensus_meta.cc:287] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.006s
I20260709 13:15:33.442744  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:33.431023  5280 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:33.428921  5543 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:33.432466  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:33.435679  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.442267  5514 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:33.450443  4548 raft_consensus.cc:2468] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:33.458142  5632 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.455154  5433 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.494669  5472 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
W20260709 13:15:33.510385  4519 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.485271  5596 raft_consensus.cc:493] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:33.495324  5554 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:33.499408  5546 leader_election.cc:290] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:33.421233  5616 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:33.482542  5454 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:33.506033  4620 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:33.501557  5601 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:33.506398  5280 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.517858  5433 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:33.569773  5604 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.570432  5604 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:33.562640  5394 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.577257  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:33.518102  5472 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.532369  5596 raft_consensus.cc:515] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:33.576690  5435 consensus_meta.cc:287] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc: Time spent flushing consensus metadata: real 0.729s	user 0.014s	sys 0.007s
I20260709 13:15:33.518989  5543 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:33.585521  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 248767us
W20260709 13:15:33.581069  5552 consensus_meta.cc:287] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512: Time spent flushing consensus metadata: real 0.691s	user 0.019s	sys 0.000s
I20260709 13:15:33.577025  5647 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.591737  5647 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:33.592363  5524 consensus_queue.cc:1048] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:33.594523  5352 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.595407  5352 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:33.597018  5647 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:33.606436  4772 raft_consensus.cc:1275] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:33.542572  5554 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:33.615064  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.610433  4694 raft_consensus.cc:1240] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:15:33.598694  5212 leader_election.cc:290] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:33.618126  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.619570  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beeb8c8bf7343c297daf017c542c67b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.621902  5352 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.622483  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:33.623708  4770 raft_consensus.cc:1275] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:33.624020  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:33.625897  5646 log.cc:927] Time spent T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:15:33.626144  5433 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:33.590363  4696 raft_consensus.cc:1240] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:15:33.627561  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.627913  5596 leader_election.cc:290] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:33.628427  4549 raft_consensus.cc:2468] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:33.630654  5601 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.631177  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:33.632575  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:33.633142  4550 consensus_meta.cc:287] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc: Time spent flushing consensus metadata: real 0.761s	user 0.008s	sys 0.010s
I20260709 13:15:33.633574  4550 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:33.633925  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:33.634869  4620 raft_consensus.cc:2393] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 2: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:33.635615  5554 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:33.636013  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0059f848a34f47b5a3fe29efe38914b3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:33.635443  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.637104  5454 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.638556  5280 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:33.639174  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.639425  5212 consensus_queue.cc:1048] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:33.639632  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3079151438004a9a975fdbe1b5b54378" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.639890  4772 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:33.633162  4740 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:33.577804  5394 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.641635  4740 leader_election.cc:304] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:33.642004  4822 tablet_bootstrap.cc:654] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.642637  5394 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:33.642735  4594 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:33.643404  5655 raft_consensus.cc:2749] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.643775  5647 raft_consensus.cc:2804] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:33.644381  5647 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.644753  5647 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:33.645139  5551 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:33.645483  4740 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:33.640511  4771 raft_consensus.cc:2393] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:33.589502  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6beeb8c8bf7343c297daf017c542c67b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:33.603117  5552 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:33.676141  4740 leader_election.cc:304] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:33.662969  4544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51884 (request call id 170) took 1268 ms. Trace:
W20260709 13:15:33.664171  5651 log.cc:927] Time spent T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:15:33.685611  5545 raft_consensus.cc:3060] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:33.686810  4740 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:33.688047  5656 raft_consensus.cc:2804] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:33.688900  5656 raft_consensus.cc:697] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:33.690213  5656 consensus_queue.cc:237] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.685490  5454 raft_consensus.cc:493] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:33.655877  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0059f848a34f47b5a3fe29efe38914b3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.652627  5653 raft_consensus.cc:2749] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:33.695329  4666 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:33.710533  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.711447  4666 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:33.712366  5565 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:33.712560  5541 raft_consensus.cc:2749] T 463b2b829b9b474b969be53a8778e42b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.712833  5565 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:33.713322  5565 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:33.681833  4544 rpcz_store.cc:276] 0709 13:15:32.395285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.397564 (+  2279us) service_pool.cc:224] Handling call
0709 13:15:32.503366 (+105802us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.953057 (+449691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.653460 (+700403us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b540006e0d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:33.662110 (+  8650us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890049920}
I20260709 13:15:33.704174  5543 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:33.704909  5601 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:33.715372  5601 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:33.705668  5454 raft_consensus.cc:515] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:33.718485  5454 leader_election.cc:290] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.706988  5616 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:33.708413  5472 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
W20260709 13:15:33.723073  4666 leader_election.cc:343] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.723699  4666 leader_election.cc:304] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:33.724534  5454 raft_consensus.cc:2749] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:33.725692  5601 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.726459  4666 leader_election.cc:304] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:33.729985  4776 raft_consensus.cc:1240] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:15:33.707794  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
W20260709 13:15:33.708806  5660 log.cc:927] Time spent T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:15:33.710233  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:33.663722  5283 consensus_meta.cc:287] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc: Time spent flushing consensus metadata: real 0.741s	user 0.011s	sys 0.011s
I20260709 13:15:33.752720  4668 leader_election.cc:304] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:33.736433  4546 raft_consensus.cc:1275] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:33.757089  5521 consensus_queue.cc:1059] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:33.663110  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.764878  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:33.765383  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:33.766000  5627 log.cc:927] Time spent T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.812s	user 0.003s	sys 0.011s
I20260709 13:15:33.766933  4700 raft_consensus.cc:1275] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:33.752417  5283 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.767340  4619 raft_consensus.cc:3060] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.769007  5283 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:33.769809  4553 raft_consensus.cc:2393] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 3: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:33.760824  5352 raft_consensus.cc:3055] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:33.770448  5352 raft_consensus.cc:740] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:33.771284  5352 consensus_queue.cc:260] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.772116  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:33.772498  5352 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:33.752854  5632 consensus_queue.cc:1048] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:15:33.749886  5601 raft_consensus.cc:2749] T 6beeb8c8bf7343c297daf017c542c67b P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:33.777005  4822 tablet_bootstrap.cc:492] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82: No bootstrap required, opened a new log
I20260709 13:15:33.776345  5653 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.777644  4822 ts_tablet_manager.cc:1403] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82: Time spent bootstrapping tablet: real 0.365s	user 0.022s	sys 0.016s
I20260709 13:15:33.777679  5653 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:33.780079  5653 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.781401  5457 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.781019  4822 raft_consensus.cc:359] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:33.782027  4822 raft_consensus.cc:385] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.782425  4822 raft_consensus.cc:740] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.782624  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.783082  4822 consensus_queue.cc:260] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:33.784891  4822 ts_tablet_manager.cc:1434] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:33.785684  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:33.761124  5655 raft_consensus.cc:2749] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.786728  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd4347b508bf4002a95a198ba9bcf072" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:33.789058  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:33.790320  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:33.774698  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:33.758496  5565 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:33.792068  4623 raft_consensus.cc:1275] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:33.792807  5565 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:33.755509  5543 raft_consensus.cc:2749] T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:33.759007  4621 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:33.795830  4774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57066 (request call id 193) took 1390 ms. Trace:
I20260709 13:15:33.796366  4774 rpcz_store.cc:276] 0709 13:15:32.405433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.407033 (+  1600us) service_pool.cc:224] Handling call
0709 13:15:33.116692 (+709659us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b58009fb798. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552eee 0000000000375939 00007f59d35c61e0 00007f59d368f0c4 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:33.133352 (+ 16660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.545900 (+412548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.551826 (+  5926us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.555197 (+  3371us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.751966 (+196769us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:33.758260 (+  6294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.761376 (+  3116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.795424 (+ 34048us) raft_consensus.cc:1685] finished
0709 13:15:33.795460 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.795753 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11038,"raft.queue_time_us":218154,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":10252,"spinlock_wait_cycles":6625664,"thread_start_us":472902,"threads_started":2,"wal-append.queue_time_us":230020}
I20260709 13:15:33.798454  5616 raft_consensus.cc:493] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.798986  5616 raft_consensus.cc:515] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:33.774791  5552 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:33.801808  5616 leader_election.cc:290] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:33.802968  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:33.806288  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.808207  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:33.813750  4740 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:33.814406  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:33.814952  5620 raft_consensus.cc:2749] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.818460  4666 leader_election.cc:304] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:33.829221  4544 raft_consensus.cc:2410] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for term 4 because replica has last-logged OpId of term: 2 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:15:33.829705  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:33.830791  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:33.833014  5630 raft_consensus.cc:2804] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:33.833973  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:33.834081  5571 consensus_queue.cc:1048] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:33.834779  4620 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:33.836859  4740 leader_election.cc:304] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:33.869812  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 52674us
I20260709 13:15:33.871819  4666 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:33.879591  4619 raft_consensus.cc:2468] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 1.
I20260709 13:15:33.875696  5541 raft_consensus.cc:2749] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:33.867168  5552 raft_consensus.cc:2749] T 26a8a7249f2c46db84e421ff58ed474e P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:33.871570  5543 raft_consensus.cc:2749] T bd4347b508bf4002a95a198ba9bcf072 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.880386  4619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48802 (request call id 187) took 1161 ms. Trace:
I20260709 13:15:33.829424  5647 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:33.887917  4666 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:33.894938  4625 raft_consensus.cc:1275] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:15:33.895418  4625 pending_rounds.cc:85] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82: Aborting all ops after (but not including) 1
I20260709 13:15:33.895675  4625 pending_rounds.cc:107] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:15:33.895928  4625 raft_consensus.cc:2889] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:33.873471  5565 consensus_queue.cc:1048] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:15:33.896462  5601 raft_consensus.cc:2804] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:33.897011  5601 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:33.897418  5601 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:33.883904  4619 rpcz_store.cc:276] 0709 13:15:32.718982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.721583 (+  2601us) service_pool.cc:224] Handling call
0709 13:15:33.050247 (+328664us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b5800003f98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:15:33.767150 (+716903us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b54002e96a8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d4454924 00007f59d363d227 00007f59d363cab8 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:33.880307 (+113157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120558208}
I20260709 13:15:33.897526  5616 raft_consensus.cc:493] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.915900  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:33.915776  5616 raft_consensus.cc:515] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:33.919111  4624 raft_consensus.cc:2368] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3.
I20260709 13:15:33.942087  5647 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:33.943203  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:33.943881  4772 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:15:33.947232  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:33.953944  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:33.955899  4772 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:33.963083  4594 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:33.966069  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:33.964414  5647 raft_consensus.cc:2804] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:33.983556  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:33.985569  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:33.987414  4668 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:33.971359  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:33.996898  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:33.991199  5565 raft_consensus.cc:2749] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:33.991894  5647 raft_consensus.cc:697] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:34.006316  5435 consensus_queue.cc:1048] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:34.006367  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:33.991222  5606 consensus_queue.cc:1048] T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:15:33.996091  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
W20260709 13:15:34.037562  4550 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51884 (request call id 168) took 1653 ms (client timeout 1815 ms). Trace:
W20260709 13:15:34.045357  4550 rpcz_store.cc:269] 0709 13:15:32.389410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.389742 (+   332us) service_pool.cc:224] Handling call
0709 13:15:34.008973 (+1619231us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b540065ead4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:34.035145 (+ 26172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1022897536}
I20260709 13:15:34.046975  4696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53246 (request call id 208) took 1413 ms. Trace:
I20260709 13:15:34.047487  4696 rpcz_store.cc:276] 0709 13:15:32.633997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.638105 (+  4108us) service_pool.cc:224] Handling call
0709 13:15:33.025106 (+387001us) spinlock_profiling.cc:243] Waited 8.51 ms on lock 0x7b580029a398. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:33.025342 (+   236us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.628163 (+602821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.046648 (+418485us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54001def54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:34.046904 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1291739904}
I20260709 13:15:34.058568  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.059381  4769 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57060 (request call id 187) took 1106 ms. Trace:
W20260709 13:15:34.062338  5673 log.cc:927] Time spent T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:15:34.064281  4695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53238 (request call id 134) took 1376 ms. Trace:
I20260709 13:15:34.064767  4695 rpcz_store.cc:276] 0709 13:15:32.687464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.690816 (+  3352us) service_pool.cc:224] Handling call
0709 13:15:34.064102 (+1373286us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54004b0d54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:34.064220 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492633344}
I20260709 13:15:34.065686  5616 leader_election.cc:290] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
W20260709 13:15:34.067255  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 54167.9us
I20260709 13:15:34.070525  4552 raft_consensus.cc:1275] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:15:34.071219  5435 consensus_queue.cc:1059] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:34.072110  5435 raft_consensus.cc:3055] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:34.072053  5613 consensus_queue.cc:1059] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:34.072634  5435 raft_consensus.cc:740] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:34.073540  5435 consensus_queue.cc:260] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:34.078926  4666 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.083345  4693 raft_consensus.cc:1240] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:34.115403  4666 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.116415  4666 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.117775  5523 raft_consensus.cc:2749] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:34.118768  5565 raft_consensus.cc:2749] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:34.119705  5565 raft_consensus.cc:2749] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:34.122243  5435 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:34.125077  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.126566  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.127802  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:34.128396  5672 log.cc:927] Time spent T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:15:34.131093  4620 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 3.
I20260709 13:15:34.113583  5472 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.132630  5472 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:34.133407  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.134672  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.135274  4695 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:34.092397  5488 consensus_queue.cc:1048] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:15:34.114779  5632 raft_consensus.cc:493] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.159502  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.160651  4593 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:34.102833  5456 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.059890  4769 rpcz_store.cc:276] 0709 13:15:32.953449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.958910 (+  5461us) service_pool.cc:224] Handling call
0709 13:15:33.095100 (+136190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.611019 (+515919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.620897 (+  9878us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.802050 (+181153us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b2c00048cc8. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ccf70a11 00007f59ccf80372 00007f59ce6b2a01 00007f59ce67743a 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2
0709 13:15:33.802566 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.813238 (+ 10672us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:15:33.813275 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.058972 (+245697us) spinlock_profiling.cc:243] Waited 50.2 ms on lock 0x7b5400383454. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:34.059061 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.059119 (+    58us) raft_consensus.cc:1685] finished
0709 13:15:34.059134 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.059314 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41814,"raft.queue_time_us":218211,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":4873,"spinlock_wait_cycles":149878784,"thread_start_us":309139,"threads_started":2,"wal-append.queue_time_us":10714}
I20260709 13:15:34.182152  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:34.199848  4695 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.171288  5546 raft_consensus.cc:2749] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.179278  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.166320  5616 consensus_queue.cc:1059] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:15:34.153371  5632 raft_consensus.cc:515] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:34.241143  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.241552  5616 raft_consensus.cc:3055] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:34.242234  5616 raft_consensus.cc:740] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:34.242825  5632 leader_election.cc:290] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:34.164605  5456 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:34.217976  4698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53252 (request call id 242) took 1108 ms. Trace:
I20260709 13:15:34.243507  5616 consensus_queue.cc:260] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:34.243700  4698 rpcz_store.cc:276] 0709 13:15:33.109805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.112777 (+  2972us) service_pool.cc:224] Handling call
0709 13:15:33.937474 (+824697us) spinlock_profiling.cc:243] Waited 5.39 ms on lock 0x7b580029a398. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:33.937706 (+   232us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.941651 (+  3945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.941687 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.942293 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 13:15:34.001676 (+ 59383us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:15:34.002651 (+   975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.007291 (+  4640us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.206196 (+198905us) raft_consensus.cc:1685] finished
0709 13:15:34.206524 (+   328us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.217881 (+ 11357us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3480,"raft.queue_time_us":11486,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":12394368,"thread_start_us":51517,"threads_started":1,"wal-append.queue_time_us":149269}
I20260709 13:15:34.245187  5456 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:34.246090  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.247478  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.219168  5630 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:34.173952  5647 consensus_queue.cc:237] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:34.248700  5630 raft_consensus.cc:3060] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.252128  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.253141  4517 leader_election.cc:304] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:34.253759  5486 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:34.254196  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.254532  5486 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:34.255331  5472 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:34.255476  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3079151438004a9a975fdbe1b5b54378" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.256384  4548 raft_consensus.cc:2393] T 3079151438004a9a975fdbe1b5b54378 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 1: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:34.257851  4768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57066 (request call id 206) took 1126 ms. Trace:
I20260709 13:15:34.258718  4624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48792 (request call id 166) took 1116 ms. Trace:
I20260709 13:15:34.259186  4624 rpcz_store.cc:276] 0709 13:15:33.142049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.144369 (+  2320us) service_pool.cc:224] Handling call
0709 13:15:33.771957 (+627588us) spinlock_profiling.cc:243] Waited 8.97 ms on lock 0x7b5800003f98. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d363d0a0 00007f59d363c984 00007f59ccbb76d4 00007f59ccbb7662
0709 13:15:34.258510 (+486553us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400659354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:34.258655 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321842304}
I20260709 13:15:34.260211  5454 raft_consensus.cc:493] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:34.260766  5454 raft_consensus.cc:515] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:34.261570  5630 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:34.262801  5397 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.263361  5397 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:34.263828  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.264940  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.265201  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:34.265566  5621 raft_consensus.cc:2749] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.222718  5600 raft_consensus.cc:493] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:34.224879  5283 consensus_queue.cc:1059] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:34.266923  4696 raft_consensus.cc:1275] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:15:34.227123  4695 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:34.253276  5580 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:34.267253  5600 raft_consensus.cc:515] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:34.222129  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.270823  4593 leader_election.cc:304] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:34.272120  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.286433  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.248449  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.302495  5609 raft_consensus.cc:493] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:34.303110  5609 raft_consensus.cc:515] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:34.275445  5616 raft_consensus.cc:3060] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.262801  5604 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:34.276211  5434 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.272703  5486 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:34.280027  5485 raft_consensus.cc:2749] T 181238be5f794f7188fdf4eb7f368b71 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.316869  5601 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:34.317736  5638 log.cc:927] Time spent T 0059f848a34f47b5a3fe29efe38914b3 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 1.194s	user 0.000s	sys 0.006s
I20260709 13:15:34.319211  5601 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:34.281196  5454 leader_election.cc:290] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:34.276017  5472 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.320502  5472 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:34.321761  5682 log.cc:927] Time spent T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:15:34.322896  5472 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:34.267823  5580 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:34.323526  5397 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:34.325533  5580 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:34.326934  5609 leader_election.cc:290] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:34.328420  5434 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:34.330657  5434 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:34.331786  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0c73f991a42e4817d8916fa0a7c30" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.332006  4545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51884 (request call id 179) took 1417 ms. Trace:
I20260709 13:15:34.332422  5604 raft_consensus.cc:493] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:34.333030  5604 raft_consensus.cc:515] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:34.258291  4768 rpcz_store.cc:276] 0709 13:15:33.131386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.133523 (+  2137us) service_pool.cc:224] Handling call
0709 13:15:33.948216 (+814693us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b58009fb798. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:33.948495 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.949005 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.949047 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.949680 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:15:34.033013 (+ 83333us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:15:34.036682 (+  3669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.038066 (+  1384us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.257523 (+219457us) raft_consensus.cc:1685] finished
0709 13:15:34.257545 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.257783 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":14160640,"thread_start_us":53180,"threads_started":1,"wal-append.queue_time_us":250141}
I20260709 13:15:34.314029  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.314407  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.310298  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.315862  4769 raft_consensus.cc:1275] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 1. (index mismatch)
I20260709 13:15:34.289167  5621 raft_consensus.cc:3055] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:34.279253  5630 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:34.342767  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.344215  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.375092  4551 raft_consensus.cc:2468] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
W20260709 13:15:34.345516  4553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51884 (request call id 173) took 1449 ms (client timeout 1594 ms). Trace:
I20260709 13:15:34.349234  4552 raft_consensus.cc:1240] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:34.300236  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:34.358657  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:34.362627  4547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51880 (request call id 177) took 2408 ms. Trace:
I20260709 13:15:34.361075  5621 raft_consensus.cc:740] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
W20260709 13:15:34.376158  4553 rpcz_store.cc:269] 0709 13:15:32.895955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.907619 (+ 11664us) service_pool.cc:224] Handling call
0709 13:15:34.345223 (+1437604us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b540006e354. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce67b2fa 00007f59d363cb17 00007f59ccbb76d4 00007f59ccbb7662 00007f59ccbb7591 00007f59ccbb750c 00007f59ccbb67c2 00007f59cc9b248c 00007f59cc9b18c6 00007f59cc9b126f
0709 13:15:34.345443 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1971508608}
I20260709 13:15:34.371994  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.366973  4622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48802 (request call id 194) took 1592 ms. Trace:
I20260709 13:15:34.377475  4547 rpcz_store.cc:276] 0709 13:15:31.946183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.947802 (+  1619us) service_pool.cc:224] Handling call
0709 13:15:31.948171 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:33.452540 (+1504369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.354060 (+901520us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540078e1d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:34.354295 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3452301184}
I20260709 13:15:34.373447  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.387316  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.385736  4741 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:34.393306  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 276903us
I20260709 13:15:34.394559  4666 leader_election.cc:304] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.395910  5687 raft_consensus.cc:2749] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.373569  5600 leader_election.cc:290] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:34.332684  4545 rpcz_store.cc:276] 0709 13:15:32.914248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.574564 (+660316us) service_pool.cc:224] Handling call
0709 13:15:34.330928 (+756364us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b58002f2298. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:34.331238 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:34.331494 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:34.331543 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:34.331603 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:34.331656 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.331754 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.331914 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10449792}
I20260709 13:15:34.389972  5648 raft_consensus.cc:493] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.390532  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.416136  5648 raft_consensus.cc:515] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:34.383500  4622 rpcz_store.cc:276] 0709 13:15:32.774125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.777961 (+  3836us) service_pool.cc:224] Handling call
0709 13:15:32.778322 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.793336 (+ 15014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.797868 (+  4532us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:32.812195 (+ 14327us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.123390 (+311195us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:15:33.123454 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.123713 (+   259us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.317060 (+1193347us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b4400022b4c. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb4b0c85 00007f59cb4b16a2 00007f59ce2231db 00007f59ce21eaaf 00007f59ce21286e 00007f59ce211bb6 00007f59ce2117d8 00007f59ce21b221 00007f59ce21b1d9 00007f59ce21b169
0709 13:15:34.335033 (+ 17973us) raft_consensus.cc:1685] finished
0709 13:15:34.338108 (+  3075us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.366890 (+ 28782us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8387,"raft.queue_time_us":13149,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":3305,"spinlock_wait_cycles":3619712,"thread_start_us":310701,"threads_started":1,"wal-append.queue_time_us":311080}
I20260709 13:15:34.396878  4697 raft_consensus.cc:2393] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 3: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:34.402561  4594 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:34.418380  5648 leader_election.cc:290] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:34.418704  5472 raft_consensus.cc:2749] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:34.419411  5604 leader_election.cc:290] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:34.419497  5472 raft_consensus.cc:493] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.397214  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.420063  5472 raft_consensus.cc:515] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:34.397992  4666 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.418586  4448 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42678 (request call id 50) took 7799 ms. Trace:
I20260709 13:15:34.397119  4767 raft_consensus.cc:1240] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:34.414803  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:34.398890  4741 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.425365  4448 rpcz_store.cc:276] 0709 13:15:26.619407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.619825 (+   418us) service_pool.cc:224] Handling call
0709 13:15:34.418497 (+7798672us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1449382,"reported_tablets":5}
I20260709 13:15:34.432861  4592 leader_election.cc:304] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:34.382020  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 103391us
I20260709 13:15:34.425345  5485 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.445304  5485 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:34.446298  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:34.447748  5485 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:34.448500  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.405220  5621 consensus_queue.cc:260] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:34.449543  5621 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:34.436882  5647 consensus_queue.cc:1048] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:15:34.451340  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.402733  5466 raft_consensus.cc:2749] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:34.457068  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 58295.9us
W20260709 13:15:34.457305  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 60492.2us
I20260709 13:15:34.437776  4592 leader_election.cc:304] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.457444  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.458496  4666 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.458199  4741 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.460026  4551 raft_consensus.cc:2393] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 3: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:34.455396  4593 leader_election.cc:304] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.462080  4666 leader_election.cc:304] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.462677  5647 raft_consensus.cc:2749] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.484099  4741 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.490564  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
W20260709 13:15:34.492949  4666 leader_election.cc:336] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607): Timed out: RequestConsensusVote RPC to 127.26.83.194:35607 timed out after 1.770s (SENT)
I20260709 13:15:34.493498  4666 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:34.487154  5686 raft_consensus.cc:2749] T 5b40eb24527a4f8bbc5da80339a74c26 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:34.487958  5454 raft_consensus.cc:2749] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.434214  5546 raft_consensus.cc:493] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
W20260709 13:15:34.498288  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 60632.3us
I20260709 13:15:34.439248  5545 raft_consensus.cc:2749] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.442545  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.502007  4592 leader_election.cc:304] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.483947  5600 raft_consensus.cc:2749] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:34.503564  4592 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.461053  5652 raft_consensus.cc:2749] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:34.437871  5523 consensus_queue.cc:1048] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
I20260709 13:15:34.497947  5565 raft_consensus.cc:2749] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.457716  5485 consensus_queue.cc:1059] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.046s
I20260709 13:15:34.512142  5488 consensus_queue.cc:1048] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:34.496354  4666 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.516244  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.527385  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.496188  5571 raft_consensus.cc:2749] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:34.501243  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86791c9adcc64babb9560e224fd9abd9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.540839  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.541527  4697 raft_consensus.cc:1275] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:34.500815  5472 leader_election.cc:290] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:34.538275  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.538805  5648 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:34.546357  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.557237  4666 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:34.558625  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.559176  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.526094  5545 raft_consensus.cc:2749] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.504585  5692 raft_consensus.cc:2749] T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:34.517678  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:34.527894  5466 raft_consensus.cc:2749] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:34.501617  5649 consensus_queue.cc:1048] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 13:15:34.503772  5546 raft_consensus.cc:515] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:34.594637  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.601336  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.557776  4770 raft_consensus.cc:2468] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:34.595882  5551 raft_consensus.cc:3055] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:34.624469  5551 raft_consensus.cc:740] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:34.625327  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:34.625520  5551 consensus_queue.cc:260] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:34.626288  4548 raft_consensus.cc:3060] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.596814  5637 consensus_queue.cc:1048] T e4e79648a69946a0b61b094c6737070a P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:15:34.620808  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86791c9adcc64babb9560e224fd9abd9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.557341  5472 raft_consensus.cc:2749] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:34.563712  5648 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:34.629371  4772 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:34.603791  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:34.633847  4619 raft_consensus.cc:2468] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 1.
I20260709 13:15:34.626749  5551 raft_consensus.cc:3060] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.600922  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:34.624117  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.606334  5647 consensus_queue.cc:1048] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:15:34.593988  5653 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.647908  5653 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:34.649561  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.650585  4694 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:34.593505  5601 raft_consensus.cc:2749] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.656224  5601 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.656973  5601 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:34.661800  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:34.664441  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 51024.5us
I20260709 13:15:34.640549  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "463b2b829b9b474b969be53a8778e42b" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.670735  5546 leader_election.cc:290] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:34.675837  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.686189  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0c73f991a42e4817d8916fa0a7c30" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:34.686579  5653 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:34.692119  4450 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42684 (request call id 48) took 9493 ms. Trace:
I20260709 13:15:34.697789  5601 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:34.701225  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:34.696956  4450 rpcz_store.cc:276] 0709 13:15:25.198575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.198705 (+   130us) service_pool.cc:224] Handling call
0709 13:15:25.451902 (+253197us) spinlock_profiling.cc:243] Waited 8.37 ms on lock 0x7b5400a190d4. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e1090 00007f59ce67ff3a 00007f59d44361cc 00007f59d453437e 00007f59d29f52a4 00007f59d29f5232 00007f59d29f5161 00007f59d29f50dc 00007f59d29f4392 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:34.692023 (+9240121us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:28.850891 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:29.083610 (+232719us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:29.083648 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:15:29.083665 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:29.211985 (+128320us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:29.218416 (+  6431us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:29.223299 (+  4883us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:15:29.306820 (+ 83521us) tablet.cc:701] Row locks acquired
0709 13:15:29.306841 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:15:29.306990 (+   149us) write_op.cc:270] Start()
0709 13:15:29.307129 (+   139us) write_op.cc:276] Timestamp: P: 1783602929306949 usec, L: 0
0709 13:15:29.307152 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:15:29.307870 (+   718us) log.cc:844] Serialized 928 byte log entry
0709 13:15:29.754616 (+446746us) op_driver.cc:464] REPLICATION: finished
0709 13:15:29.900019 (+145403us) write_op.cc:301] APPLY: starting
0709 13:15:29.900124 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:29.900267 (+   143us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:29.900403 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:29.900862 (+   459us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:29.901075 (+   213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:15:29.901135 (+    60us) write_op.cc:312] APPLY: finished
0709 13:15:29.901373 (+   238us) log.cc:844] Serialized 28 byte log entry
0709 13:15:29.902369 (+   996us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:29.902481 (+   112us) write_op.cc:454] Released schema lock
0709 13:15:29.902534 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:29.903171 (+   637us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:29.903204 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":868817,"reported_tablets":16,"spinlock_wait_cycles":19259648,"child_traces":[["op",{"apply.queue_time_us":139446,"apply.run_cpu_time_us":2225,"apply.run_wall_time_us":3016,"num_ops":2,"prepare.queue_time_us":212539,"prepare.run_cpu_time_us":43177,"prepare.run_wall_time_us":636737,"raft.queue_time_us":162236,"raft.run_cpu_time_us":13323,"raft.run_wall_time_us":106136,"replication_time_us":437335,"thread_start_us":378852,"threads_started":4,"wal-append.queue_time_us":210525,"wal-append.run_cpu_time_us":23993,"wal-append.run_wall_time_us":1378420}]]}
I20260709 13:15:34.714898  4451 rpcz_store.cc:275] Call kudu.master.MasterService.IsCreateTableDone from 127.0.0.1:42688 (request call id 40) took 1311 ms. Trace:
I20260709 13:15:34.715409  4451 rpcz_store.cc:276] 0709 13:15:33.414560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.429135 (+ 14575us) service_pool.cc:224] Handling call
0709 13:15:33.429495 (+   360us) catalog_manager.cc:2419] Looking up, locking, and authorizing table
0709 13:15:34.692668 (+1263173us) catalog_manager.cc:2317] Verify if the table creation is in progress for test-workload [id=a4b690bde17245419f655f8ee4ad32e2]
0709 13:15:34.714817 (+ 22149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1283861}
I20260709 13:15:34.722122  5648 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:34.724517  4548 raft_consensus.cc:2468] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:34.731586  4622 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 3.
I20260709 13:15:34.545192  5600 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:34.734720  5637 raft_consensus.cc:493] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:34.735409  5637 raft_consensus.cc:515] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:34.737795  5600 raft_consensus.cc:697] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 3 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:34.739113  5600 consensus_queue.cc:237] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
W20260709 13:15:34.741760  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 68068.9us
I20260709 13:15:34.723868  4449 catalog_manager.cc:5697] T 26a8a7249f2c46db84e421ff58ed474e P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 1 to 2, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 2 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:34.751686  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 44502.4us
W20260709 13:15:34.764477  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 69303us
I20260709 13:15:34.766798  4518 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:34.767602  4519 leader_election.cc:304] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
W20260709 13:15:34.770807  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 222880us
I20260709 13:15:34.777105  5692 raft_consensus.cc:493] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.777675  5692 raft_consensus.cc:515] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:34.779884  5692 leader_election.cc:290] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:34.781442  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.784921  4700 raft_consensus.cc:1240] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:15:34.785976  5637 leader_election.cc:290] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:34.788015  5486 raft_consensus.cc:2804] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:34.788558  5486 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.789011  5486 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:34.791709  4593 leader_election.cc:304] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:34.792657  4666 leader_election.cc:304] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:34.794518  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 48121us
I20260709 13:15:34.787277  4449 catalog_manager.cc:5697] T 179cf9478a1642929379f46e4311d9a5 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:34.803449  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 307761us
I20260709 13:15:34.808825  5692 raft_consensus.cc:2749] T 463b2b829b9b474b969be53a8778e42b P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:34.802790  5454 raft_consensus.cc:493] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:34.802866  5613 raft_consensus.cc:2749] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:34.813753  5454 raft_consensus.cc:515] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:34.817358  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:34.818182  4621 raft_consensus.cc:2468] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:34.820084  5623 raft_consensus.cc:3055] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:34.820591  5623 raft_consensus.cc:740] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:34.823550  4666 leader_election.cc:304] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.821425  5623 consensus_queue.cc:260] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:34.817875  4519 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:34.828877  4666 leader_election.cc:304] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:34.830925  4666 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.817718  5454 leader_election.cc:290] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:34.806380  4449 catalog_manager.cc:5697] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 1 to 2, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 2 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:34.825131  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 105756us
I20260709 13:15:34.843719  4449 catalog_manager.cc:5697] T fb2de72a119a4bb2b88f82a3713d8f7c P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 3 to 4, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 4 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:34.848596  5648 consensus_queue.cc:1048] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.851639  5456 raft_consensus.cc:2804] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:34.831048  5457 raft_consensus.cc:2749] T 54dd8f3fe16647c5b841897841d7206b P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:34.853302  4666 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:34.855367  4519 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:34.832034  5541 raft_consensus.cc:2804] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:34.838532  5634 raft_consensus.cc:2749] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.864938  4740 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:34.856235  5623 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:34.841804  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:34.861028  5693 raft_consensus.cc:2804] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:34.874078  4449 catalog_manager.cc:5697] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 1 to 2, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 2 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:34.876650  4740 leader_election.cc:304] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.879746  4519 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
W20260709 13:15:34.870981  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 367511us
I20260709 13:15:34.903445  4592 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:34.884670  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.907107  4592 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.907863  4740 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:34.880692  5454 raft_consensus.cc:2804] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:34.869483  5456 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.924055  4592 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.882752  5655 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.928565  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:34.915284  5454 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.929205  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:34.933095  4449 catalog_manager.cc:5697] T 86791c9adcc64babb9560e224fd9abd9 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:34.889093  5693 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.878291  5541 raft_consensus.cc:493] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:34.880915  5580 raft_consensus.cc:2749] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:34.938795  4449 catalog_manager.cc:5697] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 1 to 2, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 2 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:34.942029  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0c53efa0ee4dfbba4c3f1922027f50" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:34.942471  5616 raft_consensus.cc:2749] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:34.943408  5454 raft_consensus.cc:3060] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:34.933810  4592 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.905591  5283 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:34.965338  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0c53efa0ee4dfbba4c3f1922027f50" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.935899  4625 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:34.933766  5551 raft_consensus.cc:2749] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.916774  5706 raft_consensus.cc:2804] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:34.972991  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "498cf28e26b0420fa93d10f75cedb4b4" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:34.932998  5456 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:34.977579  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 43848.2us
I20260709 13:15:34.936643  5521 raft_consensus.cc:2804] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:34.916632  5705 raft_consensus.cc:2749] T 86791c9adcc64babb9560e224fd9abd9 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:34.985344  4449 catalog_manager.cc:5697] T c59401c42fe8430aaed4b49ef7df0372 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 1 to 2, leader changed from 117be35051554b3b837c42a5dfcfa38a (127.26.83.195) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 2 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:34.994325  4592 leader_election.cc:304] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.962666  5693 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.022460  4592 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:34.968892  5541 raft_consensus.cc:3060] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:35.009639  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:35.011616  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:34.916687  5631 consensus_queue.cc:1048] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.072s
I20260709 13:15:34.983582  5283 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:34.995440  5706 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:34.987741  5546 raft_consensus.cc:2749] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:35.031683  5551 raft_consensus.cc:2749] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:35.070063  4699 raft_consensus.cc:1240] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 3 LEADER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:35.110050  4701 raft_consensus.cc:1240] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:35.074985  5283 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.006271  5486 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:35.081923  5706 raft_consensus.cc:3060] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:15:35.130101  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 106155us
W20260709 13:15:35.124354  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 116577us
I20260709 13:15:35.133296  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:35.134244  4627 raft_consensus.cc:2410] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:15:35.071720  5604 consensus_queue.cc:1059] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.053s
I20260709 13:15:35.143425  4627 raft_consensus.cc:3060] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:35.144663  4740 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:35.169166  4623 raft_consensus.cc:1275] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:15:35.169687  5708 log.cc:927] Time spent T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:15:35.186296  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:35.193012  4666 leader_election.cc:304] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:35.197901  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eeb7a8868de44466ab91f3972a95eb22" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:35.216706  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:35.156932  5485 raft_consensus.cc:3055] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:35.169605  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:35.197041  5460 raft_consensus.cc:2804] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:35.207779  5693 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:35.208961  5521 raft_consensus.cc:697] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [term 2 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:35.197402  5631 consensus_queue.cc:1059] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:15:35.228652  4621 raft_consensus.cc:1240] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 2. Current term is 3. Ops: []
W20260709 13:15:35.253717  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 60842.4us
I20260709 13:15:35.226363  5485 raft_consensus.cc:740] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:35.241842  5460 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:35.246529  5637 raft_consensus.cc:2749] T 498cf28e26b0420fa93d10f75cedb4b4 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:35.282888  4666 leader_election.cc:304] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:35.282733  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 53031.5us
W20260709 13:15:35.269297  5709 log.cc:927] Time spent T c4e396207b384d2aac3bf87ded2fd828 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:15:35.265383  5352 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.242743  5283 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:35.270705  4549 raft_consensus.cc:1240] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:35.281149  5460 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.304317  5631 raft_consensus.cc:3055] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:35.304581  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:35.305411  4628 raft_consensus.cc:2468] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
W20260709 13:15:35.309939  5713 log.cc:927] Time spent T 0059f848a34f47b5a3fe29efe38914b3 P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:15:35.304829  5631 raft_consensus.cc:740] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:35.314539  5631 consensus_queue.cc:260] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:35.316991  4702 raft_consensus.cc:1275] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:15:35.318241  5714 log.cc:927] Time spent T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:15:35.319054  5723 consensus_queue.cc:1048] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:35.289253  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:35.283759  5552 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:35.292636  5687 raft_consensus.cc:2749] T 3a0c53efa0ee4dfbba4c3f1922027f50 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:35.330067  4547 raft_consensus.cc:2393] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 4: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
W20260709 13:15:35.330317  5715 log.cc:927] Time spent T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.122s	user 0.000s	sys 0.004s
I20260709 13:15:35.289119  5456 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
W20260709 13:15:35.332751  5717 log.cc:927] Time spent T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:15:35.359679  4553 raft_consensus.cc:3055] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:35.360324  4553 raft_consensus.cc:740] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:35.361573  4553 consensus_queue.cc:260] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:35.363027  4553 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.365185  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:35.339282  5552 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:35.367213  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:35.326030  5485 consensus_queue.cc:260] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:35.300562  5352 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:35.379839  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 74967.5us
I20260709 13:15:35.342347  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:35.383065  4593 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.391613  4693 raft_consensus.cc:1240] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:35.370117  5485 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.396457  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:35.397902  4553 raft_consensus.cc:1275] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:35.397221  4552 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:35.368687  5552 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
W20260709 13:15:35.402364  5722 log.cc:927] Time spent T 3079151438004a9a975fdbe1b5b54378 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 13:15:35.409767  4740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.414073  5456 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:35.415374  5693 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
W20260709 13:15:35.420277  5712 log.cc:927] Time spent T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.264s	user 0.001s	sys 0.001s
W20260709 13:15:35.425910  5716 log.cc:927] Time spent T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.281s	user 0.002s	sys 0.000s
I20260709 13:15:35.429979  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:35.435549  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
W20260709 13:15:35.439572  5623 consensus_meta.cc:287] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.000s
I20260709 13:15:35.329655  5521 consensus_queue.cc:237] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:35.443522  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:35.458168  4625 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.443877  4623 raft_consensus.cc:1275] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:15:35.462697  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:15:35.466804  5707 log.cc:927] Time spent T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.495s	user 0.001s	sys 0.001s
I20260709 13:15:35.469053  5283 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:35.469606  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:35.470276  4627 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.471657  4625 raft_consensus.cc:2468] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 4.
I20260709 13:15:35.459832  4519 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
W20260709 13:15:35.476972  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 61358.1us
I20260709 13:15:35.477384  4666 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:35.387372  5706 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:35.482582  5283 raft_consensus.cc:2749] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:35.483135  5352 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:35.483985  5637 raft_consensus.cc:2804] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:35.483870  5454 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:35.492655  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:35.484655  5637 raft_consensus.cc:697] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [term 4 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:35.495837  5637 consensus_queue.cc:237] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:35.497117  4695 raft_consensus.cc:1240] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:35.407291  5687 consensus_queue.cc:1048] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.106s
I20260709 13:15:35.515664  5460 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:35.523196  5706 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:35.524221  4773 raft_consensus.cc:1240] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:35.457803  5616 raft_consensus.cc:2749] T eeb7a8868de44466ab91f3972a95eb22 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:35.526633  4619 raft_consensus.cc:1240] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:35.526836  5454 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:35.530555  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:35.531602  4517 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:35.534933  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:35.535384  5610 raft_consensus.cc:2749] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:35.537582  5602 consensus_queue.cc:1048] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.538197  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:35.540632  4627 raft_consensus.cc:2410] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:35.542300  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:35.539850  4517 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:35.545055  4517 leader_election.cc:304] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:35.519220  5729 consensus_queue.cc:1048] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:15:35.559512  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 78547.4us
I20260709 13:15:35.561980  4517 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:35.562999  5693 raft_consensus.cc:2749] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:35.419780  5541 raft_consensus.cc:515] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:35.542035  5460 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:35.559633  5602 raft_consensus.cc:2749] T e2c78af3c856414e866ff06aa637ada1 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:35.559202  5283 raft_consensus.cc:2749] T eeb7a8868de44466ab91f3972a95eb22 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:35.583552  4699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53246 (request call id 241) took 1058 ms. Trace:
I20260709 13:15:35.584026  4699 rpcz_store.cc:276] 0709 13:15:34.524991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.540861 (+ 15870us) service_pool.cc:224] Handling call
0709 13:15:34.541193 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:35.124928 (+583735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.583338 (+458410us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54003c6c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:35.583496 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1096841600}
I20260709 13:15:35.586704  4696 raft_consensus.cc:1240] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:35.587306  5541 leader_election.cc:290] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:35.588059  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:35.558907  5687 consensus_queue.cc:1059] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:15:35.582253  4740 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:35.592265  5687 consensus_queue.cc:1243] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer 978db1e31e16405182e65e88f3b9bebc log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
W20260709 13:15:35.596151  5655 consensus_meta.cc:287] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.001s
I20260709 13:15:35.596666  5655 raft_consensus.cc:2749] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:35.597730  4776 raft_consensus.cc:1240] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:35.594311  5732 raft_consensus.cc:2804] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:35.600363  5732 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:35.603561  5732 raft_consensus.cc:3060] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:35.604949  5486 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
W20260709 13:15:35.698035  5735 log.cc:927] Time spent T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.095s	user 0.001s	sys 0.003s
I20260709 13:15:35.702302  5541 raft_consensus.cc:3055] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:15:35.753223  4592 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.765161  5486 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.820585  5486 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:35.716405  5732 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:35.564484  4517 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:35.835086  5434 consensus_queue.cc:1059] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:35.836290  4773 raft_consensus.cc:1275] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:35.767472  5283 consensus_queue.cc:1059] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:15:35.789588  5454 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:35.788333  5631 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:35.816062  5565 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.843003  5454 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:35.837378  5434 raft_consensus.cc:2804] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:35.840307  5613 raft_consensus.cc:3055] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:35.850188  5613 raft_consensus.cc:740] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:35.851629  5706 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
W20260709 13:15:35.837384  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 272831us
I20260709 13:15:35.852694  5706 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:35.856197  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:35.862308  4545 raft_consensus.cc:2468] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:35.862697  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:35.863860  4619 raft_consensus.cc:2468] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:35.867272  5732 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:35.847720  5434 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:35.872531  5434 raft_consensus.cc:3060] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:35.869746  5485 consensus_queue.cc:1048] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:15:35.864447  5728 consensus_queue.cc:1059] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:15:35.878363  5728 raft_consensus.cc:3055] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:35.878906  5728 raft_consensus.cc:740] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:35.852701  5486 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:35.858057  5454 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:35.852538  5613 consensus_queue.cc:260] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:35.890424  5719 consensus_queue.cc:1059] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:15:35.889066  5541 raft_consensus.cc:740] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:35.899983  5706 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:35.912781  4517 leader_election.cc:304] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:35.912949  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:35.913975  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:35.915081  5719 raft_consensus.cc:2749] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:35.897560  5565 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:35.923585  5565 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:35.925932  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:35.937108  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:35.944293  5728 consensus_queue.cc:260] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:35.946518  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:35.957257  5744 log.cc:927] Time spent T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:15:35.957536  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:35.958861  5609 raft_consensus.cc:3055] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:35.959462  5609 raft_consensus.cc:740] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:35.998348  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:35.999248  4551 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 4.
I20260709 13:15:36.001389  4552 raft_consensus.cc:1240] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 3. Current term is 4. Ops: [3.1-3.1]
I20260709 13:15:35.980005  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:36.002751  4694 raft_consensus.cc:2393] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:36.003217  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.007534  5609 consensus_queue.cc:260] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:36.004662  4550 raft_consensus.cc:2410] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:15:36.011915  5613 raft_consensus.cc:3060] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:36.013940  5728 raft_consensus.cc:3060] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.019160  5606 raft_consensus.cc:3055] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:36.019098  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:36.020056  5606 raft_consensus.cc:740] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:36.020608  4699 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:36.022773  5606 consensus_queue.cc:260] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
W20260709 13:15:36.025025  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 106210us
I20260709 13:15:36.027249  5606 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.037048  5609 raft_consensus.cc:3060] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.029407  5541 consensus_queue.cc:260] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 3.1, Last appended by leader: 1, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
W20260709 13:15:36.047916  4740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.049633  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:36.050830  4693 raft_consensus.cc:2393] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 4: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
W20260709 13:15:36.064860  5746 log.cc:927] Time spent T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:15:36.068990  5735 log.cc:927] Time spent T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.135s	user 0.001s	sys 0.001s
I20260709 13:15:36.072142  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:36.072533  4666 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:36.073380  4776 raft_consensus.cc:2393] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 4: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:36.073982  5454 raft_consensus.cc:2749] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:36.077454  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:36.075273  5434 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:36.081949  5434 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:36.086805  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
W20260709 13:15:36.089413  4518 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.091930  5719 consensus_queue.cc:1059] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:36.096129  5610 raft_consensus.cc:3055] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:36.096689  5610 raft_consensus.cc:740] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:36.097955  5610 consensus_queue.cc:260] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:36.098625  4518 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:36.099685  5719 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:36.100261  5719 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.100286  5610 raft_consensus.cc:3060] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.100867  5719 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:36.107965  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:36.119236  4694 raft_consensus.cc:2393] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 3: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:36.114514  5434 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.136315  4666 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:36.124387  5541 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.143584  5565 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:36.144915  5565 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:36.145399  5565 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.142922  4740 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:36.154647  4773 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:36.160014  4594 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:36.161028  5485 raft_consensus.cc:2804] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:36.161489  5485 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:36.162066  5485 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:36.165345  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:36.166456  4544 raft_consensus.cc:2468] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:36.166653  4694 raft_consensus.cc:2393] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 4: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
W20260709 13:15:36.176054  5749 log.cc:927] Time spent T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.084s	user 0.001s	sys 0.001s
I20260709 13:15:36.186049  5434 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:36.188453  4451 catalog_manager.cc:5697] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 4, leader changed from 117be35051554b3b837c42a5dfcfa38a (127.26.83.195) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 4 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:36.195276  4517 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:36.200531  4517 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:36.202423  4517 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:36.203236  5397 raft_consensus.cc:2749] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:15:36.208927  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 66061.9us
I20260709 13:15:36.209708  5397 raft_consensus.cc:2749] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:36.210045  4740 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:36.211339  5649 raft_consensus.cc:2804] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:36.211752  5649 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.212126  5649 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.212925  5486 raft_consensus.cc:2749] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:36.225073  5652 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.227622  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:36.230326  4695 raft_consensus.cc:2468] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:36.228457  4697 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 5.
I20260709 13:15:36.190886  4451 catalog_manager.cc:5697] T 9885b7c627e24d63a651c2cbc9e0e24d P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.236945  4451 catalog_manager.cc:5697] T 6beeb8c8bf7343c297daf017c542c67b P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.241595  5519 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:36.263731  5519 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:36.266476  5519 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:36.262619  4518 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
W20260709 13:15:36.250252  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 47961us
I20260709 13:15:36.251655  5565 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:36.252801  5434 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:36.257750  5649 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:36.233520  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:36.262241  4740 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:36.272428  5397 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:36.276305  4666 leader_election.cc:304] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:36.279418  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.280006  4545 raft_consensus.cc:1240] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:36.280031  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.280747  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.281643  4547 raft_consensus.cc:1240] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:15:36.281525  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.282620  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.293957  5397 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.301878  4620 raft_consensus.cc:1240] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 3. Current term is 4. Ops: [2.2-3.3]
I20260709 13:15:36.305936  4549 raft_consensus.cc:1275] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:36.308460  4627 raft_consensus.cc:1240] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:36.310544  5454 raft_consensus.cc:2804] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:36.311591  5454 raft_consensus.cc:697] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:36.309942  4544 raft_consensus.cc:1240] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:36.287330  5719 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:36.300983  4773 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:36.320120  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:36.324625  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:36.307663  5397 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:36.317710  5631 raft_consensus.cc:2749] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:15:36.332590  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 70434.7us
I20260709 13:15:36.327481  4451 catalog_manager.cc:5697] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 2, leader changed from <none> to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } }
I20260709 13:15:36.321602  5454 consensus_queue.cc:237] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:36.344617  4451 catalog_manager.cc:5697] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } }
W20260709 13:15:36.354085  4519 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.352245  5649 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:36.367206  4593 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
W20260709 13:15:36.368139  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 83554.3us
I20260709 13:15:36.350265  4546 raft_consensus.cc:2468] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:36.379860  5565 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:36.370899  5652 raft_consensus.cc:2749] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:36.385506  5719 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:36.384620  4451 catalog_manager.cc:5697] T 463b2b829b9b474b969be53a8778e42b P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 2 to 3, leader changed from 117be35051554b3b837c42a5dfcfa38a (127.26.83.195) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 3 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.375447  5485 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:36.388226  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 57523.6us
I20260709 13:15:36.391233  4627 raft_consensus.cc:1275] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:36.379765  4592 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:36.393688  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:36.394320  4695 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:36.376150  5545 raft_consensus.cc:2804] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:36.400599  4551 raft_consensus.cc:1275] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:36.406788  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:36.406433  4451 catalog_manager.cc:5697] T f2f0c73f991a42e4817d8916fa0a7c30 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.407648  4702 raft_consensus.cc:2393] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:36.408782  4620 raft_consensus.cc:1275] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:36.409101  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:36.409708  4624 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.409829  4774 raft_consensus.cc:1240] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:36.410625  5521 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:36.411298  5521 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:36.408289  4451 catalog_manager.cc:5697] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } }
I20260709 13:15:36.415845  4740 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:36.417167  5754 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.421993  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:36.423038  4767 raft_consensus.cc:2393] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 4: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:36.423206  5647 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.423547  5521 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:36.423785  5647 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:36.424225  5706 raft_consensus.cc:2749] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:36.424786  5755 consensus_queue.cc:1048] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:15:36.426481  4668 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:36.427806  5565 raft_consensus.cc:2749] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:36.428386  5647 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:36.428776  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:36.427809  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:36.429225  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 49691.7us
I20260709 13:15:36.429515  4544 raft_consensus.cc:3060] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.430153  4592 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:36.440881  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.422034  5754 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:36.443774  4619 raft_consensus.cc:2468] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:36.445993  5757 consensus_queue.cc:1048] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:15:36.446206  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 65967.9us
I20260709 13:15:36.448814  5732 raft_consensus.cc:2804] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:36.449450  5732 raft_consensus.cc:697] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [term 4 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:36.450477  5732 consensus_queue.cc:237] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:36.451483  5488 consensus_queue.cc:1048] T b34f76f541974e5490d222ce8856de10 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:36.420845  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:36.453315  4550 raft_consensus.cc:2368] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 4. Current term is 5.
I20260709 13:15:36.453471  5616 raft_consensus.cc:2749] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:36.434477  4451 catalog_manager.cc:5697] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 2, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 2 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.457306  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:36.434149  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.458315  4694 raft_consensus.cc:2393] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 5: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:36.460256  4592 leader_election.cc:304] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:36.461795  5647 raft_consensus.cc:2749] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:36.456239  4451 catalog_manager.cc:5697] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } }
I20260709 13:15:36.466933  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:36.467360  4451 catalog_manager.cc:5697] T 54dd8f3fe16647c5b841897841d7206b P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.469431  4451 catalog_manager.cc:5697] T 181238be5f794f7188fdf4eb7f368b71 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.471000  4451 catalog_manager.cc:5697] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 2 to 4, leader changed from 117be35051554b3b837c42a5dfcfa38a (127.26.83.195) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 4 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } }
I20260709 13:15:36.460597  5485 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:36.477098  4449 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42670 (request call id 55) took 8477 ms. Trace:
I20260709 13:15:36.477711  4449 rpcz_store.cc:276] 0709 13:15:27.924912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.925124 (+   212us) service_pool.cc:224] Handling call
0709 13:15:36.398436 (+8473312us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:35.299632 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:35.402798 (+103166us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:35.405175 (+  2377us) write_op.cc:435] Acquired schema lock
0709 13:15:35.407383 (+  2208us) tablet.cc:662] Decoding operations
0709 13:15:35.473913 (+ 66530us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:35.474013 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:35.474060 (+    47us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:15:35.484787 (+ 10727us) tablet.cc:701] Row locks acquired
0709 13:15:35.484817 (+    30us) write_op.cc:260] PREPARE: finished
0709 13:15:35.485013 (+   196us) write_op.cc:270] Start()
0709 13:15:35.485175 (+   162us) write_op.cc:276] Timestamp: P: 1783602935484956 usec, L: 0
0709 13:15:35.485209 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:15:35.486170 (+   961us) log.cc:844] Serialized 3257 byte log entry
0709 13:15:35.797571 (+311401us) op_driver.cc:464] REPLICATION: finished
0709 13:15:35.906664 (+109093us) write_op.cc:301] APPLY: starting
0709 13:15:35.909858 (+  3194us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:35.910729 (+   871us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:35.910906 (+   177us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:36.002849 (+ 91943us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:36.003276 (+   427us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:15:36.003360 (+    84us) write_op.cc:312] APPLY: finished
0709 13:15:36.003819 (+   459us) log.cc:844] Serialized 64 byte log entry
0709 13:15:36.026419 (+ 22600us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:36.026854 (+   435us) write_op.cc:454] Released schema lock
0709 13:15:36.026939 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:36.027764 (+   825us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:36.027803 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":358730,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":72194,"apply.run_cpu_time_us":6100,"apply.run_wall_time_us":138378,"num_ops":8,"prepare.queue_time_us":206541,"prepare.run_cpu_time_us":13036,"prepare.run_wall_time_us":305959,"raft.queue_time_us":84520,"raft.run_cpu_time_us":7232,"raft.run_wall_time_us":41976,"replication_time_us":312034,"thread_start_us":313285,"threads_started":4,"wal-append.queue_time_us":112635}]]}
I20260709 13:15:36.483592  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:36.485661  4624 raft_consensus.cc:2468] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:36.487149  5485 consensus_queue.cc:1059] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:36.493109  5754 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:36.497193  4773 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:36.501643  4594 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:36.502110  4740 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:36.502905  5616 raft_consensus.cc:2804] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:36.503510  5616 raft_consensus.cc:697] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:36.504558  5616 consensus_queue.cc:237] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:36.491766  4695 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 5.
I20260709 13:15:36.498693  5485 raft_consensus.cc:3055] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:36.498603  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:36.503036  5397 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:36.461949  4592 leader_election.cc:304] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:36.514359  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:36.515115  4767 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:36.515501  5755 consensus_queue.cc:1048] T 8d23781a09a142888526ed9cbcd30d04 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:36.512241  4518 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:36.518453  5757 raft_consensus.cc:2804] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:36.522603  5757 raft_consensus.cc:697] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [term 4 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:36.498056  4451 catalog_manager.cc:5697] T e4e79648a69946a0b61b094c6737070a P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 3 to 5, leader changed from 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196) to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 5 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } }
I20260709 13:15:36.508589  5485 raft_consensus.cc:740] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:36.508348  5545 raft_consensus.cc:697] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:36.538071  4666 leader_election.cc:400] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 4. Current term is 5.
I20260709 13:15:36.538667  4666 leader_election.cc:403] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:15:36.527058  5719 raft_consensus.cc:2804] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:36.542714  4741 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:36.526304  5706 raft_consensus.cc:2749] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:36.518580  4774 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 2.
I20260709 13:15:36.544317  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:36.547536  5521 raft_consensus.cc:2749] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:36.552847  4546 raft_consensus.cc:2368] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 5. Current term is 6.
W20260709 13:15:36.548349  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 86525.9us
I20260709 13:15:36.558748  4592 leader_election.cc:304] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:36.549106  5757 consensus_queue.cc:237] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:36.559705  5647 consensus_queue.cc:1059] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:36.560390  4740 leader_election.cc:400] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 5. Current term is 6.
I20260709 13:15:36.560784  4740 leader_election.cc:403] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:36.560907  5760 consensus_queue.cc:1059] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:36.561954  5760 raft_consensus.cc:2749] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:36.562311  5521 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:15:36.549093  5729 raft_consensus.cc:2749] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 4. Current term is 5.
I20260709 13:15:36.552908  4769 raft_consensus.cc:3060] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:36.545565  4451 catalog_manager.cc:5697] T bd4347b508bf4002a95a198ba9bcf072 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.575766  5485 consensus_queue.cc:260] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:36.576952  5719 raft_consensus.cc:697] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [term 5 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:36.577997  5719 consensus_queue.cc:237] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:36.578759  4544 raft_consensus.cc:2468] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:36.582931  5545 consensus_queue.cc:237] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:36.594134  5397 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 6 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:36.597081  5760 raft_consensus.cc:3055] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:36.598798  5760 raft_consensus.cc:740] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:36.604936  4668 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:36.610685  4769 raft_consensus.cc:2468] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 5.
I20260709 13:15:36.613242  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:36.618713  5758 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:36.619465  5758 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:36.622632  5758 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:36.619499  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:36.632861  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:36.633592  4699 raft_consensus.cc:3060] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:36.625522  5729 raft_consensus.cc:2804] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:36.638870  5729 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.639267  5729 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.641322  4517 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:36.641153  4451 catalog_manager.cc:5697] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 3, leader changed from 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 3 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } }
I20260709 13:15:36.653234  4451 catalog_manager.cc:5697] T 3079151438004a9a975fdbe1b5b54378 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:36.647002  5434 raft_consensus.cc:2804] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:36.656422  5434 raft_consensus.cc:697] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:36.657521  5434 consensus_queue.cc:237] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:36.663091  5760 consensus_queue.cc:260] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:36.664626  5760 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.667798  4699 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 6.
I20260709 13:15:36.640785  5485 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.628430  5521 raft_consensus.cc:2749] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 5. Current term is 6.
I20260709 13:15:36.690773  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.702546  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.699584  4451 catalog_manager.cc:5697] T 0059f848a34f47b5a3fe29efe38914b3 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 1 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.706818  4547 raft_consensus.cc:2393] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 4: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:36.724397  5728 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.725029  5728 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:36.727300  5728 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:36.728509  5545 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:36.729180  5545 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:36.745453  4769 raft_consensus.cc:2468] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 6.
W20260709 13:15:36.750773  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 54824.9us
I20260709 13:15:36.737293  4451 catalog_manager.cc:5697] T 92fea640a4c24068bf9e2805a83ee8b6 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 3, leader changed from 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 3 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } }
I20260709 13:15:36.705687  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:36.773419  5634 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.774060  5634 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:36.786726  5634 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:36.783304  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:36.789016  4592 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:36.790259  4451 catalog_manager.cc:5697] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } }
I20260709 13:15:36.791771  4451 catalog_manager.cc:5697] T eeb7a8868de44466ab91f3972a95eb22 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 3, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 3 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:36.792078  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.794123  4547 raft_consensus.cc:2468] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:36.782859  4694 raft_consensus.cc:2468] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:36.809942  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:36.824064  4627 raft_consensus.cc:2410] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:36.796800  4451 catalog_manager.cc:5697] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 3, leader changed from 117be35051554b3b837c42a5dfcfa38a (127.26.83.195) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 3 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } }
W20260709 13:15:36.838124  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 49209.1us
W20260709 13:15:36.841617  5764 log.cc:927] Time spent T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:15:36.857632  5545 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:36.866130  4592 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:36.867928  5706 raft_consensus.cc:2749] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:36.893941  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 219155us
I20260709 13:15:36.896318  4741 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:36.912595  5773 raft_consensus.cc:2804] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:36.913162  5773 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:36.913576  5773 raft_consensus.cc:3060] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:36.920917  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.921921  4547 raft_consensus.cc:2410] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:15:36.924381  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:36.926846  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:36.927956  4518 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:36.909816  5647 raft_consensus.cc:2804] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:36.934928  5647 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:36.935366  5647 raft_consensus.cc:3060] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:36.936895  4666 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:36.937948  5770 raft_consensus.cc:2804] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:36.957984  4666 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:36.958956  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 62721.9us
I20260709 13:15:36.950103  5541 raft_consensus.cc:2749] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:36.975471  4625 raft_consensus.cc:3060] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:36.983943  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:36.984954  4694 raft_consensus.cc:2393] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 3: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:36.990749  5729 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:36.974835  5634 raft_consensus.cc:2749] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:37.040354  5770 raft_consensus.cc:697] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [term 6 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:37.076294  4625 raft_consensus.cc:1275] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:37.100183  5729 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:37.104291  4769 raft_consensus.cc:1275] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:37.085213  5545 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.078531  5485 raft_consensus.cc:493] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.105607  5485 raft_consensus.cc:515] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:37.106554  5786 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.098810  5634 consensus_queue.cc:1048] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:15:37.107285  5786 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:37.107888  5773 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:37.110342  5773 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:37.112802  5786 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:37.117496  4625 raft_consensus.cc:1275] T bd2cefb52b43430cbf06fb122ce9c1d8 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:37.123409  5606 consensus_queue.cc:1048] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:37.118541  5770 consensus_queue.cc:237] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:37.120361  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:37.126969  4776 raft_consensus.cc:2410] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for term 5 because replica has last-logged OpId of term: 3 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:15:37.133317  5760 consensus_queue.cc:1048] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:37.105535  5545 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:37.070111  5647 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
W20260709 13:15:37.155035  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 207258us
I20260709 13:15:37.171200  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:37.202160  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:37.203626  4700 raft_consensus.cc:2410] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc for term 5 because replica has last-logged OpId of term: 3 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:15:37.217376  4518 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:37.259284  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:37.267865  4668 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:37.214532  5728 raft_consensus.cc:493] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:37.292169  5791 raft_consensus.cc:2749] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:37.280810  5778 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.284134  5755 raft_consensus.cc:2749] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:15:37.300947  5732 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.322695  5647 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:37.285493  5728 raft_consensus.cc:515] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:37.323251  4544 raft_consensus.cc:3060] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:37.322417  5732 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:37.293026  4773 raft_consensus.cc:1275] T 5171793d105d4f9b9070c7bf79d17a74 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:37.336561  4695 raft_consensus.cc:1240] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 2. Current term is 3. Ops: []
I20260709 13:15:37.344677  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:37.308539  5778 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:37.355087  4548 raft_consensus.cc:1275] T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:37.355300  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:37.356240  4545 raft_consensus.cc:2393] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 4: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:37.363832  5545 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:37.372712  4619 raft_consensus.cc:1275] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:37.379590  4544 raft_consensus.cc:1275] T 4b2841fd64294d6b9762df32ed1bb63e P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:37.382298  5732 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:37.389451  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:37.390187  4545 raft_consensus.cc:1275] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:37.385800  5778 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:37.391775  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:37.393693  4544 raft_consensus.cc:2468] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:37.397121  4592 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:37.402984  4551 raft_consensus.cc:2468] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:37.404480  4592 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:37.407253  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:37.408175  4773 raft_consensus.cc:2468] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:37.411616  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:37.412370  4769 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 3.
I20260709 13:15:37.415580  5760 raft_consensus.cc:2804] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:37.416109  5760 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.416460  5760 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.423681  5647 raft_consensus.cc:2804] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:37.424455  5647 raft_consensus.cc:697] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:37.425505  5647 consensus_queue.cc:237] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:37.433009  4550 raft_consensus.cc:1275] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:37.441067  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:37.443514  5719 consensus_queue.cc:1048] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:37.452924  5765 consensus_queue.cc:1048] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:15:37.458940  5485 leader_election.cc:290] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:37.502697  5631 consensus_queue.cc:1048] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:37.505029  4694 raft_consensus.cc:1275] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:15:37.513854  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 75607us
W20260709 13:15:37.514099  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 90087.7us
I20260709 13:15:37.517798  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:37.518496  4546 raft_consensus.cc:2468] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:37.520916  4592 leader_election.cc:304] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:37.522219  4519 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:37.525560  5647 raft_consensus.cc:2804] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:37.526181  5647 raft_consensus.cc:493] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:37.526595  5647 raft_consensus.cc:3060] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:37.527366  5719 raft_consensus.cc:2804] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:37.527947  5719 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.528332  5719 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.537886  5602 consensus_queue.cc:1048] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:37.563280  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:37.570545  4695 raft_consensus.cc:1275] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:37.573686  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:37.580334  4696 raft_consensus.cc:2393] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 3: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
I20260709 13:15:37.575028  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:37.579293  4698 raft_consensus.cc:1275] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:37.577803  5773 consensus_queue.cc:1048] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:37.582242  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:37.583035  4546 raft_consensus.cc:2468] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:37.590085  4740 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:37.596297  4776 raft_consensus.cc:1275] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 6 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:37.595805  5652 consensus_queue.cc:1048] T 84197a75d2854ab286a8a19349bf7cc9 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:15:37.605628  5486 consensus_queue.cc:1048] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:15:37.601866  5488 raft_consensus.cc:2804] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:37.587855  5541 consensus_queue.cc:1048] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:15:37.621348  5488 raft_consensus.cc:493] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.650975  4695 raft_consensus.cc:1275] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 6 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:37.621079  5602 consensus_queue.cc:1048] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:15:37.648167  5488 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.676849  5613 consensus_queue.cc:1048] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:37.679769  5719 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:37.706148  5728 leader_election.cc:290] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:37.657915  5647 raft_consensus.cc:515] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:37.710094  5647 leader_election.cc:290] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:37.681937  5706 consensus_queue.cc:1059] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:15:37.686452  5760 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:37.742336  4694 raft_consensus.cc:1275] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:37.745000  4624 raft_consensus.cc:2393] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 4: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:37.739113  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:37.751258  5760 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:37.731300  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:37.752429  4544 raft_consensus.cc:3060] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:37.754784  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:37.755609  4545 raft_consensus.cc:3060] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.736003  5809 raft_consensus.cc:3055] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:37.763576  5719 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:37.765206  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:37.767294  4699 raft_consensus.cc:2468] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:37.758917  5647 consensus_queue.cc:1048] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:15:37.770290  4544 raft_consensus.cc:2468] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
W20260709 13:15:37.771030  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 73755.4us
W20260709 13:15:37.774346  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 58865.5us
I20260709 13:15:37.775413  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:37.787396  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:37.799701  4552 raft_consensus.cc:1240] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:15:37.813388  5652 consensus_queue.cc:1048] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:37.817387  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:37.786106  5809 raft_consensus.cc:740] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:37.838512  4592 leader_election.cc:304] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:37.851061  5616 raft_consensus.cc:2804] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:37.858566  4741 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:37.871994  5812 log.cc:927] Time spent T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:15:37.878947  5545 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:37.879731  5616 raft_consensus.cc:697] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:37.879673  5545 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:37.882071  5545 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:37.882153  5616 consensus_queue.cc:237] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:37.884023  5753 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.884634  5753 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:37.886881  5753 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:37.887403  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:37.895183  5755 consensus_queue.cc:1059] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:37.896523  4666 leader_election.cc:304] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:37.897149  5809 consensus_queue.cc:260] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:37.899245  5810 log.cc:927] Time spent T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:15:37.900050  5809 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:37.901129  5637 raft_consensus.cc:2749] T 3fb194bf63a4451f8c4d28502acdad60 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:37.902911  5808 log.cc:927] Time spent T 8be6a3412d5a45a3a84b4b39e54c977a P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:15:37.903947  4547 raft_consensus.cc:2368] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4.
I20260709 13:15:37.898285  5728 raft_consensus.cc:3055] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:37.905218  5728 raft_consensus.cc:740] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:37.906095  5728 consensus_queue.cc:260] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:37.856417  4545 raft_consensus.cc:2468] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 4.
I20260709 13:15:37.910452  5434 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.911019  5434 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:37.913132  4592 leader_election.cc:400] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4.
I20260709 13:15:37.913496  4592 leader_election.cc:403] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:37.914265  4592 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:37.946666  4693 raft_consensus.cc:1275] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:37.946846  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:37.952952  4450 catalog_manager.cc:5697] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 4, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 4 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } }
I20260709 13:15:37.955679  4545 raft_consensus.cc:1275] T a153a8986132450aa579bc00e57dd1c7 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:37.957805  5728 raft_consensus.cc:3060] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:37.962488  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:37.921209  5706 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:15:37.972534  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 48106.4us
I20260709 13:15:37.979822  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:37.942289  5773 raft_consensus.cc:2804] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:37.980484  4776 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:37.980988  5773 raft_consensus.cc:697] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [term 4 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:38.009765  4666 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:37.941989  5545 raft_consensus.cc:2804] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:37.970832  5434 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:37.981910  5773 consensus_queue.cc:237] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 3.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:38.018038  4450 catalog_manager.cc:5697] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 2 to 3, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 3 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } }
I20260709 13:15:38.008975  5809 consensus_queue.cc:1048] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:15:38.033807  5488 raft_consensus.cc:515] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:38.039330  5488 leader_election.cc:290] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:38.030431  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:38.017604  5545 raft_consensus.cc:697] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [term 4 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:38.041091  4767 raft_consensus.cc:2393] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 4: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:38.061985  4450 catalog_manager.cc:5697] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 1 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } }
I20260709 13:15:38.074041  5545 consensus_queue.cc:237] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:38.083287  4776 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
W20260709 13:15:38.090047  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:38.057175  5647 consensus_queue.cc:1048] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:15:38.100839  5706 raft_consensus.cc:2749] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4.
I20260709 13:15:38.081032  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:38.103263  4619 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:38.104914  4517 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:38.105949  5770 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:38.106338  5770 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.106644  5770 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.130417  4519 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:38.134150  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 108021us
I20260709 13:15:38.142765  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:38.146485  4450 catalog_manager.cc:5697] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 1 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } }
I20260709 13:15:38.146355  5786 raft_consensus.cc:2804] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:38.158692  4450 catalog_manager.cc:5697] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 1 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } }
I20260709 13:15:38.161247  4450 catalog_manager.cc:5697] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:38.163906  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bcbea185b8c48d7989bb050470948a0" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:38.164813  4551 raft_consensus.cc:2393] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 4: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:38.155323  4451 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42686 (request call id 42) took 3301 ms. Trace:
I20260709 13:15:38.168403  4450 catalog_manager.cc:5697] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 3, leader changed from 117be35051554b3b837c42a5dfcfa38a (127.26.83.195) to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.175223  4450 catalog_manager.cc:5697] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } }
I20260709 13:15:38.176810  5755 raft_consensus.cc:2749] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:38.173262  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:38.198746  4767 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:38.157334  5786 raft_consensus.cc:697] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:38.169039  4451 rpcz_store.cc:276] 0709 13:15:34.854139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.854324 (+   185us) service_pool.cc:224] Handling call
0709 13:15:38.155216 (+3300892us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:37.055546 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:37.056369 (+   823us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:37.056431 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:15:37.056454 (+    23us) tablet.cc:662] Decoding operations
0709 13:15:37.058672 (+  2218us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:37.066851 (+  8179us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:37.069580 (+  2729us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:15:37.092881 (+ 23301us) tablet.cc:701] Row locks acquired
0709 13:15:37.092961 (+    80us) write_op.cc:260] PREPARE: finished
0709 13:15:37.093163 (+   202us) write_op.cc:270] Start()
0709 13:15:37.093341 (+   178us) write_op.cc:276] Timestamp: P: 1783602937093090 usec, L: 0
0709 13:15:37.093368 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:15:37.094334 (+   966us) log.cc:844] Serialized 8685 byte log entry
0709 13:15:37.532032 (+437698us) op_driver.cc:464] REPLICATION: finished
0709 13:15:37.560699 (+ 28667us) write_op.cc:301] APPLY: starting
0709 13:15:37.560820 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:37.561258 (+   438us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:37.561349 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:37.613923 (+ 52574us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:37.671182 (+ 57259us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 13:15:37.684582 (+ 13400us) write_op.cc:312] APPLY: finished
0709 13:15:37.707216 (+ 22634us) log.cc:844] Serialized 151 byte log entry
0709 13:15:37.707388 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:37.707579 (+   191us) write_op.cc:454] Released schema lock
0709 13:15:37.707647 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:37.708051 (+   404us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:37.708080 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":25199,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":28357,"apply.run_cpu_time_us":5520,"apply.run_wall_time_us":147342,"num_ops":22,"prepare.queue_time_us":109542,"prepare.run_cpu_time_us":49854,"prepare.run_wall_time_us":232972,"raft.queue_time_us":271145,"raft.run_cpu_time_us":19253,"raft.run_wall_time_us":28562,"replication_time_us":438520,"thread_start_us":453679,"threads_started":4,"wal-append.queue_time_us":162695}]]}
I20260709 13:15:38.210000  4740 leader_election.cc:304] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:38.229125  5824 raft_consensus.cc:2749] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:38.233920  5786 consensus_queue.cc:237] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:38.281388  5809 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.281961  5809 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:38.281626  5770 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:38.284299  5809 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:38.291050  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:38.293073  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:38.293855  4545 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 4.
I20260709 13:15:38.297154  4592 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:38.298303  5485 raft_consensus.cc:2804] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:38.298964  5485 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.299331  5485 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.298131  5770 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:38.301911  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:38.302582  4627 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.307546  5728 raft_consensus.cc:493] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:38.313341  5728 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:38.316563  5728 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:38.318037  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:38.316820  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:38.333060  4550 raft_consensus.cc:1275] T f21085005d7842b9aaa3c27849f36d63 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:38.337077  4627 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 5.
I20260709 13:15:38.338754  4517 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:38.340538  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:38.343008  4767 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.343809  5647 consensus_queue.cc:1048] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:38.344908  5770 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:38.345513  5770 raft_consensus.cc:697] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 5 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:38.346524  5770 consensus_queue.cc:237] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:38.350708  4699 raft_consensus.cc:1275] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:38.356952  4545 raft_consensus.cc:2393] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 4: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:38.359483  5647 consensus_queue.cc:1048] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:38.368363  4767 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 5.
I20260709 13:15:38.364180  5485 raft_consensus.cc:515] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:38.412629  4625 raft_consensus.cc:2368] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 4. Current term is 5.
I20260709 13:15:38.415988  4666 leader_election.cc:400] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Vote denied by peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607) with higher term. Message: Invalid argument: T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 4. Current term is 5.
I20260709 13:15:38.416373  4666 leader_election.cc:403] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:38.417321  5637 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.421639  4544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:38.431233  4544 raft_consensus.cc:3060] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:38.451862  5485 leader_election.cc:290] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:38.453508  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a4681c16b4942faaea1ca21bc374bee" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:38.462760  5637 raft_consensus.cc:2749] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607) with higher term. Message: Invalid argument: T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a for earlier term 4. Current term is 5.
I20260709 13:15:38.469187  4544 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 5.
I20260709 13:15:38.471004  4592 leader_election.cc:304] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:38.471774  5485 raft_consensus.cc:2804] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:38.472448  5485 raft_consensus.cc:697] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [term 5 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:38.473614  5485 consensus_queue.cc:237] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:38.509996  4694 raft_consensus.cc:1275] T 007ae9d1d319436394e39965fef13375 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 3 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:15:38.519517  4698 raft_consensus.cc:2468] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 5.
I20260709 13:15:38.523989  5519 consensus_queue.cc:1048] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:38.543231  4448 catalog_manager.cc:5697] T bd2cefb52b43430cbf06fb122ce9c1d8 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:38.551359  4448 catalog_manager.cc:5697] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 1 to 4, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 4 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } }
I20260709 13:15:38.554234  4450 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42678 (request call id 51) took 2278 ms. Trace:
I20260709 13:15:38.555279  4450 rpcz_store.cc:276] 0709 13:15:36.275604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.275967 (+   363us) service_pool.cc:224] Handling call
0709 13:15:38.554151 (+2278184us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:38.341459 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:38.343443 (+  1984us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:38.343479 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:15:38.343496 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:38.344421 (+   925us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:38.344479 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:38.344508 (+    29us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:15:38.345187 (+   679us) tablet.cc:701] Row locks acquired
0709 13:15:38.345204 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:15:38.345360 (+   156us) write_op.cc:270] Start()
0709 13:15:38.345463 (+   103us) write_op.cc:276] Timestamp: P: 1783602938345315 usec, L: 0
0709 13:15:38.345486 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:15:38.358947 (+ 13461us) log.cc:844] Serialized 3257 byte log entry
0709 13:15:38.426701 (+ 67754us) op_driver.cc:464] REPLICATION: finished
0709 13:15:38.463323 (+ 36622us) write_op.cc:301] APPLY: starting
0709 13:15:38.463412 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:38.463622 (+   210us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:38.463760 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:38.465573 (+  1813us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:38.465805 (+   232us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:15:38.465863 (+    58us) write_op.cc:312] APPLY: finished
0709 13:15:38.466142 (+   279us) log.cc:844] Serialized 64 byte log entry
0709 13:15:38.466328 (+   186us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:38.466455 (+   127us) write_op.cc:454] Released schema lock
0709 13:15:38.466504 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:38.466935 (+   431us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:38.466964 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":199458,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":36323,"apply.run_cpu_time_us":3586,"apply.run_wall_time_us":3586,"num_ops":8,"prepare.queue_time_us":118515,"prepare.run_cpu_time_us":3697,"prepare.run_wall_time_us":21670,"replication_time_us":79949,"thread_start_us":92156,"threads_started":2}]]}
I20260709 13:15:38.562597  4448 catalog_manager.cc:5697] T c88b6c7c1b3345b297004b802687432b P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:38.622246  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 43499.7us
I20260709 13:15:38.633258  4545 raft_consensus.cc:1275] T 1bcbea185b8c48d7989bb050470948a0 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:38.647902  4550 raft_consensus.cc:1275] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:15:38.725030  5545 consensus_queue.cc:1048] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 0, Time since last communication: 0.163s
I20260709 13:15:38.873523  4771 raft_consensus.cc:1275] T 1bcbea185b8c48d7989bb050470948a0 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:38.949882  5786 raft_consensus.cc:493] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.950603  5786 raft_consensus.cc:515] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:38.953510  5786 leader_election.cc:290] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:39.106477  5780 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:39.107280  5780 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:39.111212  5780 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:39.491673  5545 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.492556  5545 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:39.495666  5545 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
F20260709 13:15:39.504980 26959 test_workload.cc:382] Timed out: Timed out waiting for Table Creation
I20260709 13:15:38.564492  4448 catalog_manager.cc:5697] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 0 to 2, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 2 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } }
I20260709 13:15:39.541664  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:39.634162  4620 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
*** Check failure stack trace: ***
*** Aborted at 1783602939 (unix time) try "date -d @1783602939" if you are using GNU date ***
I20260709 13:15:39.617326  4627 raft_consensus.cc:1275] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:39.635859  4448 catalog_manager.cc:5697] T 5171793d105d4f9b9070c7bf79d17a74 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 1 to 5, leader changed from 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 5 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } }
PC: @                0x0 (unknown)
I20260709 13:15:39.610978  5488 consensus_queue.cc:1048] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:39.811947  5860 raft_consensus.cc:493] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:39.812711  5860 raft_consensus.cc:515] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:39.814947  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:39.815676  4770 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:39.607661  4773 raft_consensus.cc:1275] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:39.820032  4773 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57066 (request call id 320) took 1087 ms. Trace:
I20260709 13:15:39.607326  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:39.570983  5732 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:39.618631  5786 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:39.646353  5847 consensus_queue.cc:1048] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:39.898336  5786 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:39.653615  5843 raft_consensus.cc:493] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:39.702730  5846 raft_consensus.cc:493] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:39.899857  5755 raft_consensus.cc:493] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:39.900179  5846 raft_consensus.cc:515] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:39.703517  5850 raft_consensus.cc:493] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
W20260709 13:15:39.688320  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 46919.9us
W20260709 13:15:39.724190  4666 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.998s	user 0.011s	sys 0.023s
I20260709 13:15:39.729854  5849 raft_consensus.cc:493] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:39.732563  5780 raft_consensus.cc:493] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:39.732872  5842 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:39.747638  5851 raft_consensus.cc:493] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:39.754912  5852 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:39.769248  5829 raft_consensus.cc:493] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:39.769832  5728 raft_consensus.cc:493] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:39.783044  5753 raft_consensus.cc:493] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:39.783479  5545 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.786444  5848 raft_consensus.cc:493] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:39.790067  5862 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:39.824177  5860 leader_election.cc:290] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:39.825644  4594 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:39.825016  5854 raft_consensus.cc:493] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:39.831591  5706 consensus_queue.cc:1048] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:39.832419  5647 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:39.841594  5868 raft_consensus.cc:493] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:39.820485  4773 rpcz_store.cc:276] 0709 13:15:38.732875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.815789 (+ 82914us) service_pool.cc:224] Handling call
0709 13:15:39.607051 (+791262us) spinlock_profiling.cc:243] Waited 7.65 ms on lock 0x7b58009fb798. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:39.607268 (+   217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.819652 (+212384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.819937 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17604864}
I20260709 13:15:39.847154  5861 raft_consensus.cc:493] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
*** SIGABRT (@0x3e80000694f) received by PID 26959 (TID 0x7f59c71bd840) from PID 26959; stack trace: ***
I20260709 13:15:39.850471  5866 raft_consensus.cc:493] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:39.812350  5488 consensus_queue.cc:1243] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer 978db1e31e16405182e65e88f3b9bebc log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 13:15:39.859768  5773 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
W20260709 13:15:39.860901  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 49731.4us
I20260709 13:15:39.872426  5864 raft_consensus.cc:493] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
W20260709 13:15:39.874441  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 49708.8us
I20260709 13:15:39.897419  5732 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:39.911147  4666 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.185s	user 0.012s	sys 0.023s
    @           0x2d8530 __tsan::CallUserSignalHandler() at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
I20260709 13:15:39.900601  5786 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
    @           0x2daa13 rtl_sigaction() at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
I20260709 13:15:39.978425  4740 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:39.945139  4550 raft_consensus.cc:1275] T 1a4681c16b4942faaea1ca21bc374bee P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:15:40.014595  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 189049us
I20260709 13:15:39.945289  5847 raft_consensus.cc:493] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:39.920476  5753 raft_consensus.cc:515] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:39.920907  5860 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:39.946648  5874 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:39.920568  5780 raft_consensus.cc:515] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:39.911633  5851 raft_consensus.cc:515] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:40.042191  5780 leader_election.cc:290] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:40.043069  5780 raft_consensus.cc:2804] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:39.911485  5848 raft_consensus.cc:515] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.040083  5874 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:39.920972  5728 raft_consensus.cc:515] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:39.943840  5871 raft_consensus.cc:493] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:39.909958  5647 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:39.919241  5545 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:39.911913  5850 raft_consensus.cc:515] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:39.947605  5842 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
W20260709 13:15:40.067973  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 72826.8us
I20260709 13:15:39.900553  5755 raft_consensus.cc:515] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.043715  5780 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:39.947894  5868 raft_consensus.cc:515] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:39.980185  5773 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.059093  5631 raft_consensus.cc:493] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.039932  5876 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:40.043767  5851 leader_election.cc:290] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:40.053715  5706 raft_consensus.cc:493] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:39.924134  5862 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:40.147548  5706 raft_consensus.cc:515] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.149767  5862 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.150228  5706 leader_election.cc:290] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.069036  5875 raft_consensus.cc:2804] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:40.152941  5875 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.153023  5842 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.153434  5875 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:40.076462  5848 leader_election.cc:290] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.076570  5545 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:39.923537  5854 raft_consensus.cc:515] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:39.947368  5861 raft_consensus.cc:515] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:39.974865  5864 raft_consensus.cc:515] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:39.923751  5852 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:39.899937  5843 raft_consensus.cc:515] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:40.190241  5843 leader_election.cc:290] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.191792  5843 raft_consensus.cc:493] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:39.924031  5849 raft_consensus.cc:515] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.194301  5753 leader_election.cc:290] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.194530  5849 leader_election.cc:290] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:39.911753  5829 raft_consensus.cc:515] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.194927  5753 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:40.195513  5753 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:40.196743  5829 leader_election.cc:290] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
W20260709 13:15:40.068897  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 76624.1us
I20260709 13:15:40.122128  5870 raft_consensus.cc:493] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.197613  5846 leader_election.cc:290] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.198091  5545 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:40.197897  5870 raft_consensus.cc:515] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.198253  5846 consensus_queue.cc:1048] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:40.198715  5545 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.121536  5755 leader_election.cc:290] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:40.201450  5868 leader_election.cc:290] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.201850  4768 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57060 (request call id 244) took 1470 ms. Trace:
I20260709 13:15:40.119963  5732 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.202072  5868 raft_consensus.cc:493] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.119509  5877 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.202839  5868 raft_consensus.cc:515] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:39.995455  5866 raft_consensus.cc:515] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.205683  5866 leader_election.cc:290] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:40.205806  5728 leader_election.cc:290] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.206220  5852 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.206314  5866 raft_consensus.cc:493] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.206993  5866 raft_consensus.cc:515] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.207376  5728 raft_consensus.cc:493] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.207945  5728 raft_consensus.cc:515] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.078509  5780 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:40.030500  5847 raft_consensus.cc:515] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:40.121564  5647 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.088136  5769 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.220611  5769 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:40.202157  5755 raft_consensus.cc:493] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.222191  5755 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:40.222805  5769 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:40.224205  5755 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.192317  5843 raft_consensus.cc:515] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.182637  5848 raft_consensus.cc:493] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:40.225446  5848 raft_consensus.cc:515] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.226536  5843 leader_election.cc:290] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:40.227362  5848 leader_election.cc:290] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.156152  5706 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:40.229019  5773 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.229120  5706 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:40.230247  5861 leader_election.cc:290] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.231802  5706 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.066356  5871 raft_consensus.cc:515] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.234125  5728 leader_election.cc:290] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.234862  5871 leader_election.cc:290] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:40.238024  5863 consensus_queue.cc:1048] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:40.240012  5863 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:40.241245  5786 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:40.241987  5786 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.243294  5753 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.244330  5786 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.186990  5850 leader_election.cc:290] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.187326  5864 leader_election.cc:290] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.188241  5874 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.046520  5860 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.202906  5877 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:40.280220  5860 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.280292  5877 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.124764  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.200359  5870 leader_election.cc:290] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.215610  5854 leader_election.cc:290] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.202260  4768 rpcz_store.cc:276] 0709 13:15:38.731690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.815639 (+ 83949us) service_pool.cc:224] Handling call
0709 13:15:39.572891 (+757252us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b58009fb798. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59cb412db1 00007f59cb552b71 00007f59cb552a22 00007f59d446bc09 00007f59d35c5ee0 00007f59d368f0ab 00007f59d368f131 00007f59d3650fc0 00007f59d363b929 00007f59d363b3d2 00007f59ccbb2bd4
0709 13:15:39.573105 (+   214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:40.030335 (+457230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:40.037378 (+  7043us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:40.039827 (+  2449us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:40.039881 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.201422 (+161541us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b5400385c54. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59ce5e0571 00007f59ce677e01 00007f59ce67683e 00007f59d363b419 00007f59ccbb2bd4 00007f59ccbb2b62 00007f59ccbb2a91 00007f59ccbb2a0c 00007f59ccbb1cc2 00007f59cc9b248c 00007f59cc9b18c6
0709 13:15:40.201480 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.201627 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043007488}
I20260709 13:15:40.260118  5868 leader_election.cc:290] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.281395  4695 raft_consensus.cc:2393] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 3: Already voted for candidate 117be35051554b3b837c42a5dfcfa38a in this term.
W20260709 13:15:40.178179  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 54957.4us
I20260709 13:15:40.173139  5631 raft_consensus.cc:515] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.283545  5631 leader_election.cc:290] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.248158  5875 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:40.286388  5875 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
W20260709 13:15:40.300942  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 221760us
W20260709 13:15:40.301877  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 323501us
I20260709 13:15:40.332320  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "007ae9d1d319436394e39965fef13375" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.293476  5850 raft_consensus.cc:493] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:40.240554  5863 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:40.348675  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.352770  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.330168  5706 raft_consensus.cc:493] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.355441  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.355690  5903 raft_consensus.cc:493] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.356307  5890 raft_consensus.cc:493] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:40.357004  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.356958  5890 raft_consensus.cc:515] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.357049  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.357740  4770 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 4.
I20260709 13:15:40.359383  5890 leader_election.cc:290] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.360316  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.351493  5854 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.363045  5854 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:40.369107  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.371047  5866 leader_election.cc:290] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.371677  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.355676  5899 raft_consensus.cc:493] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.355676  5706 raft_consensus.cc:515] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.373234  5899 raft_consensus.cc:515] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.371677  4544 pending_rounds.cc:85] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc: Aborting all ops after (but not including) 0
I20260709 13:15:40.356309  5903 raft_consensus.cc:515] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.379987  5899 leader_election.cc:290] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.381017  5903 leader_election.cc:290] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.380052  5706 leader_election.cc:290] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.382031  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.382325  5854 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.383308  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.383582  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.384150  4770 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 4.
I20260709 13:15:40.381186  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.385008  4767 raft_consensus.cc:2468] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 1.
I20260709 13:15:40.387769  4594 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:40.388569  5868 raft_consensus.cc:493] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:40.389101  5868 raft_consensus.cc:515] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
    @     0x7f59cea7f980 (unknown) at ??:0
I20260709 13:15:40.390709  4773 raft_consensus.cc:1275] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:40.390905  5867 raft_consensus.cc:493] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.391337  5868 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.379083  4544 pending_rounds.cc:107] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20260709 13:15:40.392827  4544 raft_consensus.cc:2889] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:40.393180  5863 leader_election.cc:290] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:40.351433  5850 raft_consensus.cc:515] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.396075  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.397099  5850 leader_election.cc:290] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.397594  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.398763  5847 leader_election.cc:290] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:40.177526  5876 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.399966  5545 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.398814  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.391525  5867 raft_consensus.cc:515] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:40.407202  4770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:40.410777  4594 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:40.413376  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.368460  5780 raft_consensus.cc:515] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:40.415499  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.416196  5888 raft_consensus.cc:493] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:40.416901  5780 leader_election.cc:290] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.417536  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.418247  5866 raft_consensus.cc:493] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:40.418814  5753 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:40.419054  5847 consensus_queue.cc:1048] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:40.419317  5545 raft_consensus.cc:2804] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:40.420560  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.541488  5888 raft_consensus.cc:515] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.543900  5866 raft_consensus.cc:515] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:40.412370  5854 raft_consensus.cc:2804] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:40.422976  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.423866  4770 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:40.425202  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.546283  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.546793  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.429481  4594 leader_election.cc:304] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:40.547446  4553 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:40.547549  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.434496  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0c73f991a42e4817d8916fa0a7c30" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.548342  5875 raft_consensus.cc:2804] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:40.548841  5875 raft_consensus.cc:493] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.549221  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c78af3c856414e866ff06aa637ada1" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.544957  5854 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.549455  5753 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:40.550096  5854 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:40.550594  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.551149  5888 leader_election.cc:290] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.551702  5753 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.551908  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.543560  4769 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 5.
I20260709 13:15:40.553390  4592 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:40.443894  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.444347  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.447479  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.548386  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.447940  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4e79648a69946a0b61b094c6737070a" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.448271  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.449682  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.452622  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.452821  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.455600  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.556433  4545 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 4.
I20260709 13:15:40.557441  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.557901  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.558640  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.559095  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.559698  4693 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:40.559788  4545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.560534  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd411637f5c84d50b6c9ba5cae086a07" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.456148  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.456575  5867 leader_election.cc:290] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.463508  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84197a75d2854ab286a8a19349bf7cc9" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.561430  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26a8a7249f2c46db84e421ff58ed474e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.561779  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.562070  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2841fd64294d6b9762df32ed1bb63e" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.562593  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.563030  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.466121  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.563349  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.563711  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.465113  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dd8f3fe16647c5b841897841d7206b" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.564563  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.565539  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.565874  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.566216  4697 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:40.566659  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.567114  4546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.567206  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.482048  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.567942  4553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.568167  4699 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:40.478929  4622 raft_consensus.cc:1275] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Refusing update from remote peer 978db1e31e16405182e65e88f3b9bebc: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:40.568706  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.557044  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.569202  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290a017cf81042b6a4e64319563864c5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.569885  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:40.570291  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.570367  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.485178  5899 raft_consensus.cc:493] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:40.571142  5899 raft_consensus.cc:515] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.571514  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.479678  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.573276  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.573498  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.573556  4517 leader_election.cc:304] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:40.574755  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.575289  5848 raft_consensus.cc:2749] T 84197a75d2854ab286a8a19349bf7cc9 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:40.575938  5848 consensus_queue.cc:1048] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:40.494863  4448 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42670 (request call id 56) took 2821 ms. Trace:
I20260709 13:15:40.491935  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.557441  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2cefb52b43430cbf06fb122ce9c1d8" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.577754  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.578135  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34f76f541974e5490d222ce8856de10" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.578301  4623 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 3.
I20260709 13:15:40.579236  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b05eadf0e794d249a3ee1e016745baf" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.579324  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.579629  4698 raft_consensus.cc:1275] T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
    @     0x7f59c7e7de87 gsignal at ??:0
I20260709 13:15:40.580327  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.505820  4696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.510613  4549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.510634  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.511943  5868 raft_consensus.cc:493] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.638228  5868 raft_consensus.cc:515] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.638561  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.639892  4623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.640506  5868 leader_election.cc:290] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:40.512279  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.513096  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.516584  5892 raft_consensus.cc:493] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.643973  5892 raft_consensus.cc:515] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:40.647684  5854 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:40.646978  5892 leader_election.cc:290] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.529753  4518 leader_election.cc:304] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:40.421826  5876 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
W20260709 13:15:40.547683  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 118292us
I20260709 13:15:40.433703  4627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "556306e5b2024ff493e111028a819954" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.654173  4623 raft_consensus.cc:1240] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:15:40.438213  4624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d23781a09a142888526ed9cbcd30d04" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.656060  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.555693  4625 raft_consensus.cc:2468] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 1.
I20260709 13:15:40.563143  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82b6a1da7d62472aa62e9be6aa348f2e" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.475325  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181238be5f794f7188fdf4eb7f368b71" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.658866  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9885b7c627e24d63a651c2cbc9e0e24d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.660192  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.564851  5850 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:40.660943  5850 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:40.661584  5850 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:40.567430  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.666174  5854 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:40.570673  4622 raft_consensus.cc:2368] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 3. Current term is 4.
I20260709 13:15:40.484829  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0c73f991a42e4817d8916fa0a7c30" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.565874  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:40.669200  4700 raft_consensus.cc:3060] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:40.575982  4767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5171793d105d4f9b9070c7bf79d17a74" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.484629  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.489269  4621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a153a8986132450aa579bc00e57dd1c7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.491370  4628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.495414  4450 catalog_manager.cc:5697] T 4b2841fd64294d6b9762df32ed1bb63e P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 3, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:40.497443  4498 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:40.572649  4694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.622697  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:40.627749  4723 debug-util.cc:398] Leaking SignalData structure 0x7b080015a280 after lost signal to thread 26962
I20260709 13:15:40.625313  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:40.573729  4592 leader_election.cc:304] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:40.668393  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.589890  4626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.568617  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fb194bf63a4451f8c4d28502acdad60" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.687013  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.582355  4768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8be6a3412d5a45a3a84b4b39e54c977a" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.588757  5866 leader_election.cc:290] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:40.549239  5875 raft_consensus.cc:3060] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:40.583827  4769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c88b6c7c1b3345b297004b802687432b" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.564531  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.567874  5545 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:40.583678  5899 leader_election.cc:290] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:40.576656  4448 rpcz_store.cc:276] 0709 13:15:37.673741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.673936 (+   195us) service_pool.cc:224] Handling call
0709 13:15:40.494797 (+2820861us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:39.680159 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:39.681014 (+   855us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:39.681063 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:15:39.681091 (+    28us) tablet.cc:662] Decoding operations
0709 13:15:39.682023 (+   932us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:39.682120 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:39.682154 (+    34us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:15:39.682795 (+   641us) tablet.cc:701] Row locks acquired
0709 13:15:39.682820 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:15:39.682991 (+   171us) write_op.cc:270] Start()
0709 13:15:39.683126 (+   135us) write_op.cc:276] Timestamp: P: 1783602939682938 usec, L: 0
0709 13:15:39.683145 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:15:39.683915 (+   770us) log.cc:844] Serialized 2093 byte log entry
0709 13:15:39.840567 (+156652us) op_driver.cc:464] REPLICATION: finished
0709 13:15:40.154791 (+314224us) write_op.cc:301] APPLY: starting
0709 13:15:40.173857 (+ 19066us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:40.220357 (+ 46500us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:40.220496 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:40.221124 (+   628us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:40.221329 (+   205us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:15:40.221373 (+    44us) write_op.cc:312] APPLY: finished
0709 13:15:40.221609 (+   236us) log.cc:844] Serialized 46 byte log entry
0709 13:15:40.221802 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:40.221907 (+   105us) write_op.cc:454] Released schema lock
0709 13:15:40.221952 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:40.222316 (+   364us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:40.222351 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35467,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":246886,"apply.run_cpu_time_us":2530,"apply.run_wall_time_us":104693,"num_ops":5,"prepare.queue_time_us":21663,"prepare.run_cpu_time_us":19871,"prepare.run_wall_time_us":55759,"raft.queue_time_us":90811,"raft.run_cpu_time_us":22487,"raft.run_wall_time_us":167061,"replication_time_us":157235,"thread_start_us":322509,"threads_started":4,"wal-append.queue_time_us":62275}]]}
I20260709 13:15:40.635186  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b40eb24527a4f8bbc5da80339a74c26" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:40.580489  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d318f5fdde647228f87449d2cf0201d" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:40.589509  4740 leader_election.cc:304] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:40.582726  4770 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 4.
I20260709 13:15:40.590410  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "779c16a6075e4893824178771ade71a1" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.610896  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92fea640a4c24068bf9e2805a83ee8b6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.649704  4741 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:40.683514  4450 catalog_manager.cc:5697] T 84197a75d2854ab286a8a19349bf7cc9 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 4, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 4 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } }
W20260709 13:15:40.684172  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 110510us
I20260709 13:15:40.582806  4772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d356c2066af241e4a89fad36578a7f89" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.652221  4594 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:40.581233  4773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e396207b384d2aac3bf87ded2fd828" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
W20260709 13:15:40.688867  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 159200us
I20260709 13:15:40.688812  4593 leader_election.cc:304] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:40.689216  5848 raft_consensus.cc:2749] T 007ae9d1d319436394e39965fef13375 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:40.735819  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 47148us
W20260709 13:15:40.691545  4723 debug-util.cc:398] Leaking SignalData structure 0x7b080015a960 after lost signal to thread 4485
I20260709 13:15:40.685664  5854 raft_consensus.cc:2749] T 26a8a7249f2c46db84e421ff58ed474e P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:40.728338  4592 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:40.702348  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 112922us
I20260709 13:15:40.729579  4450 catalog_manager.cc:5697] T f21085005d7842b9aaa3c27849f36d63 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 3, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:40.733755  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 84132.8us
I20260709 13:15:40.736708  4593 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:40.745451  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 93303.2us
I20260709 13:15:40.762013  4740 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:40.763018  4450 catalog_manager.cc:5697] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:40.766773  4593 leader_election.cc:304] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:40.701161  5545 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:40.773041  4594 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:40.774228  4740 leader_election.cc:400] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Vote denied by peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607) with higher term. Message: Invalid argument: T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 3. Current term is 4.
I20260709 13:15:40.774610  4740 leader_election.cc:403] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:15:40.769382  4450 catalog_manager.cc:5697] T a153a8986132450aa579bc00e57dd1c7 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 2 to 3, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:40.761142  5876 raft_consensus.cc:2749] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:40.774693  4741 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:40.781086  4741 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:40.776232  4450 catalog_manager.cc:5697] T 1bcbea185b8c48d7989bb050470948a0 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:40.780076  4517 leader_election.cc:304] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:40.737660  5647 raft_consensus.cc:2749] T cd411637f5c84d50b6c9ba5cae086a07 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:40.784282  4741 leader_election.cc:304] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:40.763844  5892 raft_consensus.cc:2749] T b34f76f541974e5490d222ce8856de10 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:40.778705  4594 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:40.767323  5706 raft_consensus.cc:2749] T 40db247913f74803b44a6878860081b7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:40.771916  4620 raft_consensus.cc:2393] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 117be35051554b3b837c42a5dfcfa38a in current term 4: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:40.788062  4741 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:40.773073  5842 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:40.703189  5732 raft_consensus.cc:2749] T e2c78af3c856414e866ff06aa637ada1 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:40.735020  5867 raft_consensus.cc:2749] T c88b6c7c1b3345b297004b802687432b P 098b8827550f475e8c8fa8a20de8f512 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:40.770561  4700 raft_consensus.cc:2468] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 4.
I20260709 13:15:40.799772  5631 raft_consensus.cc:2749] T 290a017cf81042b6a4e64319563864c5 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:40.787360  5895 raft_consensus.cc:2749] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607) with higher term. Message: Invalid argument: T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 3. Current term is 4.
I20260709 13:15:40.818437  5631 raft_consensus.cc:2749] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:40.793143  5902 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:40.819342  5902 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.793437  5886 raft_consensus.cc:2804] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:40.819692  5902 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:40.819907  5886 raft_consensus.cc:493] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:40.820343  5886 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:40.829160  4723 debug-util.cc:398] Leaking SignalData structure 0x7b080028c560 after lost signal to thread 4577
W20260709 13:15:40.829581  4723 debug-util.cc:398] Leaking SignalData structure 0x7b0800151f40 after lost signal to thread 4578
W20260709 13:15:40.830763  4723 debug-util.cc:398] Leaking SignalData structure 0x7b080034c300 after lost signal to thread 4652
W20260709 13:15:40.831084  4723 debug-util.cc:398] Leaking SignalData structure 0x7b08003bf920 after lost signal to thread 4653
I20260709 13:15:40.787621  5885 raft_consensus.cc:2804] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:40.801734  5842 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:40.815960  5892 raft_consensus.cc:2749] T 556306e5b2024ff493e111028a819954 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:40.849997  5854 raft_consensus.cc:2804] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:40.839169  5885 raft_consensus.cc:493] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Starting leader election (detected failure of leader fb69c26178814cacb4f3abd0750d4b82)
I20260709 13:15:40.841516  5888 raft_consensus.cc:2749] T b34f76f541974e5490d222ce8856de10 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:15:40.868661  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 101947us
I20260709 13:15:40.857909  5842 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:40.870694  4593 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:40.862807  4723 debug-util.cc:398] Leaking SignalData structure 0x7b08001229a0 after lost signal to thread 4726
W20260709 13:15:40.876034  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 96015.2us
I20260709 13:15:40.867101  5885 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:40.872026  4723 debug-util.cc:398] Leaking SignalData structure 0x7b08002af000 after lost signal to thread 4727
I20260709 13:15:40.870808  5854 raft_consensus.cc:697] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [term 4 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
W20260709 13:15:40.925768  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 152741us
W20260709 13:15:40.873227  4518 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 89403us
W20260709 13:15:40.933601  4723 debug-util.cc:398] Leaking SignalData structure 0x7b08002ed2c0 after lost signal to thread 4800
W20260709 13:15:40.928304  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 57703.9us
I20260709 13:15:40.930178  5876 raft_consensus.cc:2749] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:15:40.934669  4723 debug-util.cc:398] Leaking SignalData structure 0x7b08000fd920 after lost signal to thread 4801
I20260709 13:15:40.931644  4518 leader_election.cc:304] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:40.950456  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 162462us
W20260709 13:15:40.953802  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 160335us
I20260709 13:15:40.955232  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb2de72a119a4bb2b88f82a3713d8f7c" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:40.957183  4517 leader_election.cc:304] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:40.957243  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:40.959669  5876 consensus_queue.cc:1048] T 1a4681c16b4942faaea1ca21bc374bee P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:40.967540  4518 leader_election.cc:304] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
W20260709 13:15:40.969033  4594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 190382us
I20260709 13:15:40.948369  5854 consensus_queue.cc:237] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:40.973399  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:40.968300  5888 raft_consensus.cc:2749] T 82b6a1da7d62472aa62e9be6aa348f2e P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
    @     0x7f59c7e7f7f1 abort at ??:0
I20260709 13:15:40.963848  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "179cf9478a1642929379f46e4311d9a5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
    @           0x2de526 abort at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
I20260709 13:15:40.984700  5850 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:40.994096  4619 raft_consensus.cc:1240] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:40.987387  5902 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:40.996096  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:40.998311  5902 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 6 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
W20260709 13:15:41.000164  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 47380.5us
I20260709 13:15:41.000824  4518 leader_election.cc:304] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:40.965055  4593 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.004146  4593 leader_election.cc:304] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:41.004642  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 52432.8us
I20260709 13:15:41.005904  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:41.009490  4593 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:41.013486  4593 leader_election.cc:304] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.017403  5902 raft_consensus.cc:2749] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:41.019661  5850 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:41.020995  5847 raft_consensus.cc:2749] T 54dd8f3fe16647c5b841897841d7206b P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.021569  4518 leader_election.cc:304] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.022421  5860 raft_consensus.cc:2749] T 556306e5b2024ff493e111028a819954 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.023626  4518 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:41.026432  4668 leader_election.cc:304] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.026564  4620 raft_consensus.cc:1240] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 6 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:15:41.027371  5878 raft_consensus.cc:2749] T 181238be5f794f7188fdf4eb7f368b71 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.028502  5860 consensus_queue.cc:1059] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:41.030427  5860 raft_consensus.cc:3055] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:41.030838  5860 raft_consensus.cc:740] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:41.031673  5860 consensus_queue.cc:260] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:41.036937  5860 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.034621  4668 leader_election.cc:304] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.008101  5888 raft_consensus.cc:2749] T 3b05eadf0e794d249a3ee1e016745baf P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.009724  5916 raft_consensus.cc:2749] T e4e79648a69946a0b61b094c6737070a P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:41.007797  5877 raft_consensus.cc:2749] T 8d23781a09a142888526ed9cbcd30d04 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.014084  5917 raft_consensus.cc:2749] T 5e52269d0b2241c6b96a578f3ab61d72 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.027271  5919 raft_consensus.cc:2749] T 82b6a1da7d62472aa62e9be6aa348f2e P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.042769  5878 raft_consensus.cc:2749] T f2f0c73f991a42e4817d8916fa0a7c30 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.042027  5886 raft_consensus.cc:515] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:41.045322  4775 raft_consensus.cc:1240] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:41.047891  5918 consensus_queue.cc:1059] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:41.049413  4741 leader_election.cc:304] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.057950  4518 leader_election.cc:304] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:41.056207  5920 raft_consensus.cc:2749] T c4e396207b384d2aac3bf87ded2fd828 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.048646  5842 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:41.062719  4668 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.063653  5842 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:41.033161  4517 leader_election.cc:304] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.065313  5888 raft_consensus.cc:2749] T 8d23781a09a142888526ed9cbcd30d04 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.065768  5878 raft_consensus.cc:2749] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.066087  5545 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:41.066671  4517 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:41.067641  5888 raft_consensus.cc:2804] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:41.068107  5888 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:41.032864  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c7d2adc93614f2ea7a7e5f63f4efab7" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:41.068480  5888 raft_consensus.cc:3060] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:41.053589  5886 leader_election.cc:290] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:41.069931  4776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:41.060374  5885 raft_consensus.cc:515] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:41.071604  4774 raft_consensus.cc:1240] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:41.074411  5919 consensus_queue.cc:1059] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:41.075516  5919 raft_consensus.cc:3055] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:41.075841  5919 raft_consensus.cc:740] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:41.076699  5919 consensus_queue.cc:260] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:41.052560  5915 raft_consensus.cc:2749] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.078040  5919 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:41.079154  5916 consensus_queue.cc:1059] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:41.080574  5849 raft_consensus.cc:3055] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:41.081099  5849 raft_consensus.cc:740] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:41.082226  5885 leader_election.cc:290] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:41.082173  5849 consensus_queue.cc:260] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:41.084383  5915 raft_consensus.cc:2749] T 9885b7c627e24d63a651c2cbc9e0e24d P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.086298  5875 raft_consensus.cc:515] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:41.088104  4666 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:41.088548  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40db247913f74803b44a6878860081b7" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:41.089687  5878 raft_consensus.cc:2804] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:41.090150  5878 raft_consensus.cc:493] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:41.090672  5878 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.090816  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:41.093321  5875 leader_election.cc:290] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:41.093910  5545 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:41.049067  5918 raft_consensus.cc:3055] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:41.096935  5918 raft_consensus.cc:740] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:41.097385  4666 leader_election.cc:304] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.097787  5918 consensus_queue.cc:260] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:41.098631  5871 raft_consensus.cc:2749] T bd2cefb52b43430cbf06fb122ce9c1d8 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.099351  5918 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:41.099777  4666 leader_election.cc:304] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.100862  5871 raft_consensus.cc:2749] T 779c16a6075e4893824178771ade71a1 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.102026  4666 leader_election.cc:304] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.102808  5871 raft_consensus.cc:2749] T 4b2841fd64294d6b9762df32ed1bb63e P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.107623  5878 raft_consensus.cc:515] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:41.113354  5849 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:41.117287  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:41.129875  4592 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.131294  4592 leader_election.cc:304] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.132089  5545 raft_consensus.cc:2749] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.136310  5545 raft_consensus.cc:2749] T fb2de72a119a4bb2b88f82a3713d8f7c P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.016503  4593 leader_election.cc:304] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
W20260709 13:15:41.139294  4593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 122864us
I20260709 13:15:41.140142  4593 leader_election.cc:304] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.135555  4592 leader_election.cc:304] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.148773  4450 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42678 (request call id 52) took 2508 ms. Trace:
I20260709 13:15:41.149276  4450 rpcz_store.cc:276] 0709 13:15:38.639972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.640189 (+   217us) service_pool.cc:224] Handling call
0709 13:15:41.148675 (+2508486us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:40.957321 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:40.957854 (+   533us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:40.957888 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:15:40.957907 (+    19us) tablet.cc:662] Decoding operations
0709 13:15:40.958795 (+   888us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:40.958885 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:40.958919 (+    34us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:15:40.959443 (+   524us) tablet.cc:701] Row locks acquired
0709 13:15:40.959461 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:15:40.959596 (+   135us) write_op.cc:270] Start()
0709 13:15:40.959708 (+   112us) write_op.cc:276] Timestamp: P: 1783602940959552 usec, L: 0
0709 13:15:40.959729 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:15:40.960417 (+   688us) log.cc:844] Serialized 2481 byte log entry
0709 13:15:41.105404 (+144987us) op_driver.cc:464] REPLICATION: finished
0709 13:15:41.108642 (+  3238us) write_op.cc:301] APPLY: starting
0709 13:15:41.108748 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:41.108924 (+   176us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:41.109062 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:41.109725 (+   663us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:41.116130 (+  6405us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:15:41.116179 (+    49us) write_op.cc:312] APPLY: finished
0709 13:15:41.116448 (+   269us) log.cc:844] Serialized 52 byte log entry
0709 13:15:41.116616 (+   168us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:41.116722 (+   106us) write_op.cc:454] Released schema lock
0709 13:15:41.117707 (+   985us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":5295,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":2968,"num_ops":6,"prepare.queue_time_us":166098,"prepare.run_cpu_time_us":3577,"prepare.run_wall_time_us":3576,"replication_time_us":145521,"thread_start_us":177193,"threads_started":2}]]}
I20260709 13:15:41.154510  4668 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.155844  5871 raft_consensus.cc:2749] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.158695  4666 leader_election.cc:304] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.159888  5871 raft_consensus.cc:2749] T 5171793d105d4f9b9070c7bf79d17a74 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:41.161325  4666 leader_election.cc:304] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.161445  5888 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:41.162207  5871 raft_consensus.cc:2749] T 8be6a3412d5a45a3a84b4b39e54c977a P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.163589  4666 leader_election.cc:304] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.169059  5871 raft_consensus.cc:2749] T d356c2066af241e4a89fad36578a7f89 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.171106  4666 leader_election.cc:304] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.173070  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:41.173183  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:41.179512  4695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:41.180761  4694 raft_consensus.cc:1240] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Rejecting Update request from peer 098b8827550f475e8c8fa8a20de8f512 for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:41.185750  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 50287.7us
I20260709 13:15:41.187536  4693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:41.190258  4740 leader_election.cc:304] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.190183  4666 leader_election.cc:304] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.197731  4702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:41.197968  5854 raft_consensus.cc:493] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 098b8827550f475e8c8fa8a20de8f512)
I20260709 13:15:41.198473  5854 raft_consensus.cc:515] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:41.189693  5917 raft_consensus.cc:2749] T 779c16a6075e4893824178771ade71a1 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.190006  5545 raft_consensus.cc:2749] T 3fb194bf63a4451f8c4d28502acdad60 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.199987  5732 raft_consensus.cc:2749] T 5b40eb24527a4f8bbc5da80339a74c26 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.201730  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:41.189833  5871 raft_consensus.cc:2749] T c88b6c7c1b3345b297004b802687432b P 117be35051554b3b837c42a5dfcfa38a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:41.191694  5868 raft_consensus.cc:2749] T 179cf9478a1642929379f46e4311d9a5 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.217787  4622 raft_consensus.cc:1240] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:41.221045  4592 leader_election.cc:304] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.221697  5854 leader_election.cc:290] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:41.221904  5545 raft_consensus.cc:2749] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.225906  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:41.227025  4593 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.188460  5878 leader_election.cc:290] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:41.233343  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "672896bdac10415b85afc981d3c6eac9" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:41.234530  4552 raft_consensus.cc:2410] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:15:41.234694  5545 raft_consensus.cc:2749] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.235814  5899 raft_consensus.cc:2749] T 92fea640a4c24068bf9e2805a83ee8b6 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.236057  4593 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a
W20260709 13:15:41.235893  4666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 45788.8us
I20260709 13:15:41.236981  5545 raft_consensus.cc:2749] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:15:41.238260  4666 leader_election.cc:304] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.239449  5728 raft_consensus.cc:2749] T 9885b7c627e24d63a651c2cbc9e0e24d P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.242213  5769 consensus_queue.cc:1059] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:41.243949  5888 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:41.244709  5842 raft_consensus.cc:3055] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:41.239408  4666 leader_election.cc:304] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.245182  5842 raft_consensus.cc:740] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:41.247611  5842 consensus_queue.cc:260] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:41.249537  5842 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Advancing to term 5
    @     0x7f59cac4fd84 google::LogMessage::SendToLog() at ??:0
I20260709 13:15:41.257385  5878 raft_consensus.cc:2749] T c4e396207b384d2aac3bf87ded2fd828 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.263415  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:41.273121  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d49f8bbce5438badd7576f93fa30fd" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:41.281085  4619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7c8faa6496c4c3dbd2e25d9617b0ffc" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:41.286085  4666 leader_election.cc:304] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.287427  5899 raft_consensus.cc:2749] T a153a8986132450aa579bc00e57dd1c7 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.288864  4666 leader_election.cc:304] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
W20260709 13:15:41.289371  4517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 56369.8us
I20260709 13:15:41.289611  5878 raft_consensus.cc:2749] T 0c7d2adc93614f2ea7a7e5f63f4efab7 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:41.293614  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:41.294432  4666 leader_election.cc:304] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.295038  4592 leader_election.cc:304] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.295217  5878 raft_consensus.cc:2749] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.297276  5868 raft_consensus.cc:2749] T 672896bdac10415b85afc981d3c6eac9 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.299299  4592 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:41.300375  5877 raft_consensus.cc:2749] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:41.301169  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:41.301617  5878 consensus_queue.cc:1059] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:41.302826  5899 raft_consensus.cc:3055] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:41.303192  5899 raft_consensus.cc:740] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:41.304067  5899 consensus_queue.cc:260] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:41.305655  5899 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:41.309412  4774 raft_consensus.cc:2468] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117be35051554b3b837c42a5dfcfa38a in term 5.
I20260709 13:15:41.318288  4451 catalog_manager.cc:5697] T 8be6a3412d5a45a3a84b4b39e54c977a P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 3 to 4, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 4 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.335520  4619 raft_consensus.cc:2468] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:41.336587  4449 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42684 (request call id 49) took 4621 ms. Trace:
I20260709 13:15:41.341683  4449 rpcz_store.cc:276] 0709 13:15:36.715666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.716915 (+  1249us) service_pool.cc:224] Handling call
0709 13:15:41.322513 (+4605598us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b68000c7760. stack: 00007f59cb558343 00007f59cb558240 00007f59caee71d4 00007f59d4c25304 00007f59d4c25299 00007f59d4452de1 00007f59d44300ca 00007f59d442e808 00007f59d442ce40 00007f59d4534c3e 00007f59d29f52a4 00007f59d29f5232 00007f59d29f5161 00007f59d29f50dc 00007f59d29f4392 00007f59cc9b248c
0709 13:15:41.336458 (+ 13945us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":33,"spinlock_wait_cycles":66903808}
I20260709 13:15:41.343329  4740 leader_election.cc:304] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:41.344048  4668 leader_election.cc:304] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 117be35051554b3b837c42a5dfcfa38a; no voters: 
I20260709 13:15:41.330890  4451 catalog_manager.cc:5697] T 007ae9d1d319436394e39965fef13375 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 0 to 4, leader changed from <none> to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 4 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:41.349995  4451 catalog_manager.cc:5697] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 1 to 2, leader changed from 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196) to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } }
I20260709 13:15:41.332456  5866 consensus_queue.cc:1059] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:15:41.361557  5780 raft_consensus.cc:2804] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:41.369954  5728 raft_consensus.cc:3055] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:41.370442  5780 raft_consensus.cc:697] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [term 4 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:41.370571  5728 raft_consensus.cc:740] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:41.371567  5780 consensus_queue.cc:237] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:41.371834  5728 consensus_queue.cc:260] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:41.362175  5871 raft_consensus.cc:2804] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:41.375480  5871 raft_consensus.cc:697] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [term 5 LEADER]: Becoming Leader. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
W20260709 13:15:41.389619  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 193319us
I20260709 13:15:41.384097  4622 raft_consensus.cc:2468] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:41.404989  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f21085005d7842b9aaa3c27849f36d63" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:41.391708  5890 raft_consensus.cc:493] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:41.406100  4698 raft_consensus.cc:1275] T 4d318f5fdde647228f87449d2cf0201d P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:41.406463  5890 raft_consensus.cc:515] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:41.376596  5871 consensus_queue.cc:237] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } }
I20260709 13:15:41.412294  4451 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42686 (request call id 43) took 2905 ms. Trace:
I20260709 13:15:41.415036  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:41.396206  5728 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.412174  4740 leader_election.cc:304] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 117be35051554b3b837c42a5dfcfa38a
I20260709 13:15:41.413724  4451 rpcz_store.cc:276] 0709 13:15:38.507035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.507721 (+   686us) service_pool.cc:224] Handling call
0709 13:15:41.412231 (+2904510us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:41.369067 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:41.369642 (+   575us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:41.369677 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:15:41.369694 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:41.370401 (+   707us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:41.370471 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:41.370503 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:15:41.370823 (+   320us) tablet.cc:701] Row locks acquired
0709 13:15:41.370842 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:15:41.370982 (+   140us) write_op.cc:270] Start()
0709 13:15:41.371107 (+   125us) write_op.cc:276] Timestamp: P: 1783602941370938 usec, L: 0
0709 13:15:41.371138 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:15:41.371801 (+   663us) log.cc:844] Serialized 1316 byte log entry
0709 13:15:41.374095 (+  2294us) op_driver.cc:464] REPLICATION: finished
0709 13:15:41.375012 (+   917us) write_op.cc:301] APPLY: starting
0709 13:15:41.375097 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:41.375226 (+   129us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:41.375305 (+    79us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:41.375648 (+   343us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:41.375873 (+   225us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:15:41.375924 (+    51us) write_op.cc:312] APPLY: finished
0709 13:15:41.376140 (+   216us) log.cc:844] Serialized 34 byte log entry
0709 13:15:41.376321 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:41.376450 (+   129us) write_op.cc:454] Released schema lock
0709 13:15:41.376488 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:41.376889 (+   401us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:41.376916 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20512,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":694,"apply.run_cpu_time_us":1819,"apply.run_wall_time_us":1829,"num_ops":3,"prepare.queue_time_us":145,"prepare.run_cpu_time_us":3067,"prepare.run_wall_time_us":4823,"replication_time_us":2819}]]}
I20260709 13:15:41.409454  4449 catalog_manager.cc:5697] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:41.433104  4622 raft_consensus.cc:1240] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:41.435163  4548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d0edf26f04c6eaef32e7dcc96eec6" candidate_uuid: "117be35051554b3b837c42a5dfcfa38a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:41.429317  5877 consensus_queue.cc:1048] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:15:41.408708  5890 leader_election.cc:290] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
W20260709 13:15:41.440637  5932 log.cc:927] Time spent T 1a4681c16b4942faaea1ca21bc374bee P 117be35051554b3b837c42a5dfcfa38a: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:15:41.444655  5842 raft_consensus.cc:2804] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:41.484501  4666 leader_election.cc:304] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.488924  5866 raft_consensus.cc:2749] T f21085005d7842b9aaa3c27849f36d63 P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.489748  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:41.491385  5842 raft_consensus.cc:697] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [term 4 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:41.492376  5842 consensus_queue.cc:237] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:41.455521  5934 consensus_queue.cc:1059] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:15:41.500049  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e52269d0b2241c6b96a578f3ab61d72" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
W20260709 13:15:41.509107  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 111688us
I20260709 13:15:41.511677  4518 leader_election.cc:304] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.516399  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:41.522138  5914 raft_consensus.cc:2749] T 40db247913f74803b44a6878860081b7 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:15:41.555073  5890 raft_consensus.cc:3055] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:41.555734  5890 raft_consensus.cc:740] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:41.558336  4771 raft_consensus.cc:1275] T 4d318f5fdde647228f87449d2cf0201d P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:41.562704  4518 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.563678  5888 raft_consensus.cc:2749] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.565886  4518 leader_election.cc:304] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc; no voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.566717  5888 raft_consensus.cc:2749] T 5e52269d0b2241c6b96a578f3ab61d72 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:41.569064  5854 consensus_queue.cc:1048] T 4d318f5fdde647228f87449d2cf0201d P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:41.556660  5890 consensus_queue.cc:260] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:41.582580  5890 raft_consensus.cc:3060] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:41.609830  4550 raft_consensus.cc:1240] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Rejecting Update request from peer 117be35051554b3b837c42a5dfcfa38a for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:41.605535  4451 catalog_manager.cc:5697] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } }
I20260709 13:15:41.628693  4740 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512; no voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:41.612527  4451 catalog_manager.cc:5697] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 3 to 5, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 5 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } }
I20260709 13:15:41.631006  5899 consensus_queue.cc:1059] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:15:41.635870  5866 raft_consensus.cc:3055] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:41.636281  5866 raft_consensus.cc:740] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 117be35051554b3b837c42a5dfcfa38a, State: Running, Role: LEADER
I20260709 13:15:41.634116  5895 raft_consensus.cc:2749] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:15:41.637575  5866 consensus_queue.cc:260] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:41.639761  5866 raft_consensus.cc:3060] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 1 FOLLOWER]: Advancing to term 2
    @     0x7f59cac50930 google::LogMessage::Flush() at ??:0
I20260709 13:15:41.636258  4451 catalog_manager.cc:5697] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 3 to 4, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 4 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.671595  4774 raft_consensus.cc:1240] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 3. Current term is 4. Ops: []
I20260709 13:15:41.700570  4449 catalog_manager.cc:5697] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a reported cstate change: term changed from 4 to 5, leader changed from 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196) to 117be35051554b3b837c42a5dfcfa38a (127.26.83.195). New cstate: current_term: 5 leader_uuid: "117be35051554b3b837c42a5dfcfa38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:41.772358  4519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.196:36999, user_credentials={real_user=slave}} blocked reactor thread for 64852.1us
I20260709 13:15:41.707432  5933 consensus_queue.cc:1059] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 13:15:41.791280  5933 raft_consensus.cc:3055] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:41.791713  5933 raft_consensus.cc:740] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:41.792554  5933 consensus_queue.cc:260] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:41.804868  4775 raft_consensus.cc:1275] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:41.808531  5933 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:41.815454  5866 consensus_queue.cc:1048] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:41.846321  4698 raft_consensus.cc:3060] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:41.862586  4698 raft_consensus.cc:1275] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:41.873193  4449 catalog_manager.cc:5697] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 3 to 4, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 4 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.870096  5631 consensus_queue.cc:1048] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:15:41.902918  4550 raft_consensus.cc:3060] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.922158  4550 raft_consensus.cc:1275] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Refusing update from remote peer 117be35051554b3b837c42a5dfcfa38a: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:41.941731  5866 consensus_queue.cc:1048] T 5a9d0edf26f04c6eaef32e7dcc96eec6 P 117be35051554b3b837c42a5dfcfa38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:15:41.947126  4619 raft_consensus.cc:1275] T a4d49f8bbce5438badd7576f93fa30fd P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:41.950002  5631 consensus_queue.cc:1048] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:41.964046  4620 raft_consensus.cc:1275] T e7c8faa6496c4c3dbd2e25d9617b0ffc P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:41.969980  5631 consensus_queue.cc:1048] T e7c8faa6496c4c3dbd2e25d9617b0ffc P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
    @     0x7f59cac55a6b google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260709 13:15:42.005024  4698 raft_consensus.cc:3060] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.024273  4698 raft_consensus.cc:1275] T a4d49f8bbce5438badd7576f93fa30fd P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:15:42.026381  5769 consensus_queue.cc:1048] T a4d49f8bbce5438badd7576f93fa30fd P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
    @     0x7f59d4c47a9e kudu::TestWorkload::Setup() at ??:0
    @           0x365578 kudu::master::LeaderRebalancerTest::CreateWorkloadTable() at /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:112
    @           0x35a423 kudu::master::LeaderRebalancerTest_RestartTserver_Test::TestBody() at /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:412
I20260709 13:15:42.196168  5898 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:42.196784  5898 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.199622  5898 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:42.201948  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:42.202349  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:42.203132  4620 raft_consensus.cc:2393] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 5: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:42.203187  4550 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 5.
I20260709 13:15:42.208114  4740 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:42.209579  5898 raft_consensus.cc:2804] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:42.209997  5898 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:42.210415  5898 raft_consensus.cc:3060] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:42.233683  5922 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:42.234349  5922 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.236845  5922 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:42.239058  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:42.239858  4550 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 6.
I20260709 13:15:42.241725  4740 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:42.242195  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:42.242730  5922 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:42.243120  4620 raft_consensus.cc:2393] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 6: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:42.243253  5922 raft_consensus.cc:493] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:42.243863  5922 raft_consensus.cc:3060] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:42.252295  5898 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.265920  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:42.272976  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:42.273949  4620 raft_consensus.cc:2393] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 5: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:42.275949  5898 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
    @     0x7f59cb7bae10 testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
I20260709 13:15:42.329370  5922 raft_consensus.cc:515] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.334079  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:42.334847  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4693f763e2c548f8a49643afa022a5f5" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:42.335803  4622 raft_consensus.cc:2393] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 in current term 6: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:42.343345  4552 raft_consensus.cc:2468] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 6.
I20260709 13:15:42.344936  4550 raft_consensus.cc:2468] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 5.
I20260709 13:15:42.345896  4740 leader_election.cc:304] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:42.346959  5890 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:42.347306  4740 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:42.347623  5890 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:42.348668  5898 raft_consensus.cc:2804] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:42.349308  5898 raft_consensus.cc:697] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [term 5 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:42.350391  5898 consensus_queue.cc:237] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.351418  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:42.351612  5890 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:42.352327  4622 raft_consensus.cc:2393] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 4: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:42.352833  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:42.353564  4697 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:42.332485  5922 leader_election.cc:290] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 6 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:42.347630  5769 raft_consensus.cc:2804] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:42.356913  5769 raft_consensus.cc:697] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [term 6 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:42.357224  4518 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:42.358036  5769 consensus_queue.cc:237] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.359921  5890 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:42.360440  5890 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:42.360896  5890 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Advancing to term 4
    @     0x7f59cb798a12 testing::Test::Run() at ??:0
I20260709 13:15:42.407531  4451 catalog_manager.cc:5697] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 4 to 5, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 5 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.410477  5890 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:42.574792  5945 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:42.604194  5945 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:42.608783  5945 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:42.666936  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:42.667382  5890 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Requested vote from peers fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:42.667802  4700 raft_consensus.cc:2468] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
W20260709 13:15:42.673246  5958 log.cc:927] Time spent T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:15:42.677738  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 135643us
I20260709 13:15:42.680640  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:42.681679  4551 raft_consensus.cc:2393] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
I20260709 13:15:42.691844  4622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:42.692943  4622 raft_consensus.cc:2393] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 978db1e31e16405182e65e88f3b9bebc in current term 4: Already voted for candidate fb69c26178814cacb4f3abd0750d4b82 in this term.
I20260709 13:15:42.698326  4593 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:42.708683  5965 raft_consensus.cc:2804] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:42.709164  5965 raft_consensus.cc:493] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 117be35051554b3b837c42a5dfcfa38a)
I20260709 13:15:42.709590  5965 raft_consensus.cc:3060] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:42.733079  5945 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:42.733762  5945 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.740695  5945 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:42.743294  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:42.744508  4775 raft_consensus.cc:2393] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 4: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
W20260709 13:15:42.774809  4592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.193:36159, user_credentials={real_user=slave}} blocked reactor thread for 73560us
I20260709 13:15:42.787916  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:42.788839  4547 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 4.
I20260709 13:15:42.801131  4698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "978db1e31e16405182e65e88f3b9bebc" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:42.802096  4592 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:42.803794  5945 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:15:42.804246  5945 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:42.804705  5945 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.812384  5945 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.848116  5982 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.848662  5982 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:42.850244  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:42.851011  5982 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:42.851240  4771 raft_consensus.cc:2393] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 4: Already voted for candidate 098b8827550f475e8c8fa8a20de8f512 in this term.
I20260709 13:15:42.853926  4550 raft_consensus.cc:1275] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:42.857127  4624 raft_consensus.cc:1275] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:15:42.858426  4547 raft_consensus.cc:1275] T 4693f763e2c548f8a49643afa022a5f5 P 978db1e31e16405182e65e88f3b9bebc [term 6 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:42.873209  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:42.874125  4771 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 4.
I20260709 13:15:42.877907  4594 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:42.879067  5980 raft_consensus.cc:2804] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:42.879572  5980 raft_consensus.cc:493] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.879928  5980 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:42.886354  5965 raft_consensus.cc:515] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:42.889525  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:42.897728  4547 raft_consensus.cc:2393] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 in current term 2: Already voted for candidate 978db1e31e16405182e65e88f3b9bebc in this term.
W20260709 13:15:42.888931  4741 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.195:42995, user_credentials={real_user=slave}} blocked reactor thread for 44821.6us
I20260709 13:15:42.895267  5969 consensus_queue.cc:1048] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:15:42.888273  5980 raft_consensus.cc:515] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:42.889895  4697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:42.901343  4697 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 5.
I20260709 13:15:42.897746  4698 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978db1e31e16405182e65e88f3b9bebc in term 4.
I20260709 13:15:42.913713  5981 raft_consensus.cc:493] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.914446  5980 leader_election.cc:290] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 6 election: Requested vote from peers 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995), 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999)
I20260709 13:15:42.914335  5981 raft_consensus.cc:515] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.914351  4699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:42.916287  4699 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:42.916659  5981 leader_election.cc:290] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:42.917137  5975 consensus_queue.cc:1048] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:42.917505  5945 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:42.919822  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:42.920378  4518 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, 978db1e31e16405182e65e88f3b9bebc; no voters: fb69c26178814cacb4f3abd0750d4b82
I20260709 13:15:42.920668  4547 raft_consensus.cc:2410] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:15:42.921357  5847 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:42.921815  4550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:42.921854  5847 raft_consensus.cc:697] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Becoming Leader. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:42.925746  5847 consensus_queue.cc:237] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:42.929337  4701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be07b1d9e584cabb47a7e6a71d9cd44" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:42.931417  4771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746d9c187500488796f0879f81bf74f3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:42.931496  4775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f4fb667ee0546d7b1b1a8a6ca7519a6" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:42.932226  4771 raft_consensus.cc:3060] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:15:42.934639  5842 consensus_queue.cc:1048] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:42.935213  4594 leader_election.cc:304] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:42.946180  4701 raft_consensus.cc:2468] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:42.949061  4593 leader_election.cc:304] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:42.953624  4550 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 4.
I20260709 13:15:42.956180  4592 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:42.956307  5945 raft_consensus.cc:2804] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:42.957113  5945 raft_consensus.cc:697] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [term 2 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:42.958042  5945 consensus_queue.cc:237] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:42.958392  5980 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:42.958992  5980 raft_consensus.cc:697] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 4 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:42.959951  5980 consensus_queue.cc:237] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.947353  5981 raft_consensus.cc:2749] T 6f4fb667ee0546d7b1b1a8a6ca7519a6 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:15:42.935653  5965 leader_election.cc:290] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:42.979269  5868 raft_consensus.cc:493] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.984328  5868 raft_consensus.cc:515] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:42.986734  4620 raft_consensus.cc:1275] T 4693f763e2c548f8a49643afa022a5f5 P fb69c26178814cacb4f3abd0750d4b82 [term 6 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:42.989435  5967 consensus_queue.cc:1048] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:42.994751  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512" is_pre_election: true
I20260709 13:15:43.001260  5868 leader_election.cc:290] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:43.002341  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:43.003026  4552 raft_consensus.cc:2468] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 2.
I20260709 13:15:43.004280  4592 leader_election.cc:304] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 098b8827550f475e8c8fa8a20de8f512
I20260709 13:15:43.005434  5868 raft_consensus.cc:2804] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:15:43.005837  5868 raft_consensus.cc:493] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.006170  5868 raft_consensus.cc:3060] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:43.007843  4771 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 6.
I20260709 13:15:43.003314  4699 raft_consensus.cc:2468] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 6.
I20260709 13:15:43.000455  4450 catalog_manager.cc:5697] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc reported cstate change: term changed from 3 to 4, leader changed from 117be35051554b3b837c42a5dfcfa38a (127.26.83.195) to 978db1e31e16405182e65e88f3b9bebc (127.26.83.193). New cstate: current_term: 4 leader_uuid: "978db1e31e16405182e65e88f3b9bebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.026873  4594 leader_election.cc:304] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:43.038493  5769 raft_consensus.cc:493] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:43.046711  5769 raft_consensus.cc:515] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:43.049060  5769 leader_election.cc:290] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
    @     0x7f59cb799d29 testing::TestInfo::Run() at ??:0
I20260709 13:15:43.051548  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:43.074023  4552 raft_consensus.cc:2468] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 2.
I20260709 13:15:43.075361  4740 leader_election.cc:304] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
W20260709 13:15:43.068109  5985 log.cc:927] Time spent T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:15:43.076865  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:43.037063  5982 raft_consensus.cc:2804] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:43.077834  5982 raft_consensus.cc:697] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [term 6 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:43.078737  5982 consensus_queue.cc:237] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } }
I20260709 13:15:43.081276  5842 raft_consensus.cc:2804] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:15:43.081724  5842 raft_consensus.cc:493] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 978db1e31e16405182e65e88f3b9bebc)
I20260709 13:15:43.089998  4451 catalog_manager.cc:5697] T 4693f763e2c548f8a49643afa022a5f5 P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 5 to 6, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 6 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.082098  5842 raft_consensus.cc:3060] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:43.098078  4449 catalog_manager.cc:5697] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 2, leader changed from 117be35051554b3b837c42a5dfcfa38a (127.26.83.195) to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 2 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.123220  5868 raft_consensus.cc:515] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:43.124462  4620 raft_consensus.cc:2368] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3.
I20260709 13:15:43.128294  4774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "098b8827550f475e8c8fa8a20de8f512"
I20260709 13:15:43.128684  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:43.124298  4449 catalog_manager.cc:5697] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 3 to 4, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 4 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:43.129366  4552 raft_consensus.cc:3060] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:43.132851  5868 leader_election.cc:290] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Requested vote from peers 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196:36999), 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159)
I20260709 13:15:43.150604  4740 leader_election.cc:400] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 pre-election: Vote denied by peer fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607) with higher term. Message: Invalid argument: T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3.
I20260709 13:15:43.153980  4552 raft_consensus.cc:2468] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
I20260709 13:15:43.155246  4592 leader_election.cc:304] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978db1e31e16405182e65e88f3b9bebc, fb69c26178814cacb4f3abd0750d4b82; no voters: 
I20260709 13:15:43.150740  5969 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.158288  5969 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:43.159567  5945 raft_consensus.cc:2804] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:43.160171  5945 raft_consensus.cc:697] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [term 3 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:43.160403  5969 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:43.160995  5945 consensus_queue.cc:237] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:43.164891  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:43.165781  4552 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 4.
I20260709 13:15:43.170878  4740 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:43.173467  5981 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.174096  5981 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:43.165712  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82" is_pre_election: true
I20260709 13:15:43.177627  5631 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:43.177700  4551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc" is_pre_election: true
I20260709 13:15:43.178145  5631 raft_consensus.cc:493] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.178507  5631 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.228271  5981 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:43.229081  5842 raft_consensus.cc:515] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:43.235445  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a" is_pre_election: true
I20260709 13:15:43.236258  4700 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 4.
I20260709 13:15:43.237494  4593 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:43.239650  4774 raft_consensus.cc:3060] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:43.260090  5868 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:43.269161  5868 raft_consensus.cc:493] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.269551  5868 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.270455  5631 raft_consensus.cc:515] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:43.312856  5631 leader_election.cc:290] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:43.334810  4774 raft_consensus.cc:2468] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 3.
W20260709 13:15:43.336872  4740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.83.194:35607, user_credentials={real_user=slave}} blocked reactor thread for 47218.5us
I20260709 13:15:43.338418  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:43.338977  4547 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.340549  4620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e2e6b7b0af4d39b62cb855c6bfd81a" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:43.361052  5842 leader_election.cc:290] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194:35607)
I20260709 13:15:43.370121  4547 raft_consensus.cc:2468] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 098b8827550f475e8c8fa8a20de8f512 in term 5.
    @     0x7f59cb79ad05 testing::TestSuite::Run() at ??:0
I20260709 13:15:43.381855  5868 raft_consensus.cc:515] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:43.389324  4622 raft_consensus.cc:1240] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Rejecting Update request from peer 978db1e31e16405182e65e88f3b9bebc for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:43.391979  5890 consensus_queue.cc:1059] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:43.392889  5890 raft_consensus.cc:3055] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:43.393258  5890 raft_consensus.cc:740] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 978db1e31e16405182e65e88f3b9bebc, State: Running, Role: LEADER
I20260709 13:15:43.386384  4449 catalog_manager.cc:5697] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 4 to 6, leader changed from 117be35051554b3b837c42a5dfcfa38a (127.26.83.195) to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 6 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.394304  5890 consensus_queue.cc:260] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:43.397888  4740 leader_election.cc:304] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098b8827550f475e8c8fa8a20de8f512, 978db1e31e16405182e65e88f3b9bebc; no voters: 
I20260709 13:15:43.395622  4449 catalog_manager.cc:5697] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 1 to 3, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 3 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:43.400705  6001 raft_consensus.cc:2804] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:43.405368  6001 raft_consensus.cc:697] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 5 LEADER]: Becoming Leader. State: Replica: 098b8827550f475e8c8fa8a20de8f512, State: Running, Role: LEADER
I20260709 13:15:43.406306  6001 consensus_queue.cc:237] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:43.401799  5868 leader_election.cc:290] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 election: Requested vote from peers 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159), 117be35051554b3b837c42a5dfcfa38a (127.26.83.195:42995)
I20260709 13:15:43.410907  4547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:43.395588  5890 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.415380  4625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fb69c26178814cacb4f3abd0750d4b82"
I20260709 13:15:43.417057  4552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc2914707b4e1586aff19b228e92c3" candidate_uuid: "098b8827550f475e8c8fa8a20de8f512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "978db1e31e16405182e65e88f3b9bebc"
I20260709 13:15:43.417884  4552 raft_consensus.cc:2368] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3.
I20260709 13:15:43.418803  4700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e34aff2d174a4f3c8092c452a40c94a3" candidate_uuid: "fb69c26178814cacb4f3abd0750d4b82" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "117be35051554b3b837c42a5dfcfa38a"
I20260709 13:15:43.419447  4700 raft_consensus.cc:3060] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.441831  4699 raft_consensus.cc:1275] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 117be35051554b3b837c42a5dfcfa38a [term 2 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:43.457271  4740 leader_election.cc:400] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3.
I20260709 13:15:43.462188  4740 leader_election.cc:403] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:15:43.462877  5970 raft_consensus.cc:2749] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 978db1e31e16405182e65e88f3b9bebc (127.26.83.193:36159) with higher term. Message: Invalid argument: T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 098b8827550f475e8c8fa8a20de8f512 for earlier term 2. Current term is 3.
I20260709 13:15:43.467908  5982 consensus_queue.cc:1048] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:43.479091  4699 raft_consensus.cc:1275] T 746d9c187500488796f0879f81bf74f3 P 117be35051554b3b837c42a5dfcfa38a [term 6 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:43.483286  4547 raft_consensus.cc:2410] T e34aff2d174a4f3c8092c452a40c94a3 P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate fb69c26178814cacb4f3abd0750d4b82 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:15:43.499902  5982 consensus_queue.cc:1048] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:43.512400  4552 raft_consensus.cc:1275] T 0be07b1d9e584cabb47a7e6a71d9cd44 P 978db1e31e16405182e65e88f3b9bebc [term 2 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:43.514657  5945 consensus_queue.cc:1048] T 0be07b1d9e584cabb47a7e6a71d9cd44 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:43.526810  4700 raft_consensus.cc:2468] T e34aff2d174a4f3c8092c452a40c94a3 P 117be35051554b3b837c42a5dfcfa38a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb69c26178814cacb4f3abd0750d4b82 in term 5.
I20260709 13:15:43.538617  4593 leader_election.cc:304] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 117be35051554b3b837c42a5dfcfa38a, fb69c26178814cacb4f3abd0750d4b82; no voters: 978db1e31e16405182e65e88f3b9bebc
I20260709 13:15:43.541849  4771 raft_consensus.cc:1240] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [term 5 LEADER]: Rejecting Update request from peer fb69c26178814cacb4f3abd0750d4b82 for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:43.547277  5981 raft_consensus.cc:2804] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:43.548147  5965 consensus_queue.cc:1059] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:43.548898  5965 consensus_queue.cc:1243] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Peer 098b8827550f475e8c8fa8a20de8f512 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:15:43.549501  5965 raft_consensus.cc:3055] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:43.549836  5965 raft_consensus.cc:740] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:43.550633  5965 consensus_queue.cc:260] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } }
I20260709 13:15:43.551891  5965 raft_consensus.cc:3060] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:43.552551  5981 raft_consensus.cc:697] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [term 5 LEADER]: Becoming Leader. State: Replica: fb69c26178814cacb4f3abd0750d4b82, State: Running, Role: LEADER
I20260709 13:15:43.574637  5981 consensus_queue.cc:237] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } }
I20260709 13:15:43.578290  4771 raft_consensus.cc:1275] T 746d9c187500488796f0879f81bf74f3 P 098b8827550f475e8c8fa8a20de8f512 [term 6 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:43.603998  5982 consensus_queue.cc:1048] T 746d9c187500488796f0879f81bf74f3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:15:43.646327  4547 raft_consensus.cc:1275] T 74cc2914707b4e1586aff19b228e92c3 P 978db1e31e16405182e65e88f3b9bebc [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:43.651834  5868 consensus_queue.cc:1048] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:43.637199  4449 catalog_manager.cc:5697] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 reported cstate change: term changed from 4 to 5, leader changed from fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194) to 098b8827550f475e8c8fa8a20de8f512 (127.26.83.196). New cstate: current_term: 5 leader_uuid: "098b8827550f475e8c8fa8a20de8f512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:43.697975  4771 raft_consensus.cc:1275] T 74cc2914707b4e1586aff19b228e92c3 P 098b8827550f475e8c8fa8a20de8f512 [term 3 FOLLOWER]: Refusing update from remote peer fb69c26178814cacb4f3abd0750d4b82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:15:43.705744  5980 consensus_queue.cc:1048] T 74cc2914707b4e1586aff19b228e92c3 P fb69c26178814cacb4f3abd0750d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098b8827550f475e8c8fa8a20de8f512" member_type: VOTER last_known_addr { host: "127.26.83.196" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:43.765157  4449 catalog_manager.cc:5697] T e34aff2d174a4f3c8092c452a40c94a3 P fb69c26178814cacb4f3abd0750d4b82 reported cstate change: term changed from 4 to 5, leader changed from 978db1e31e16405182e65e88f3b9bebc (127.26.83.193) to fb69c26178814cacb4f3abd0750d4b82 (127.26.83.194). New cstate: current_term: 5 leader_uuid: "fb69c26178814cacb4f3abd0750d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb69c26178814cacb4f3abd0750d4b82" member_type: VOTER last_known_addr { host: "127.26.83.194" port: 35607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117be35051554b3b837c42a5dfcfa38a" member_type: VOTER last_known_addr { host: "127.26.83.195" port: 42995 } health_report { overall_health: UNKNOWN } } }
    @     0x7f59cb7af7f5 testing::internal::UnitTestImpl::RunAllTests() at ??:0
I20260709 13:15:43.927132  4552 raft_consensus.cc:1275] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 978db1e31e16405182e65e88f3b9bebc [term 5 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
    @     0x7f59cb7bbcd0 testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
I20260709 13:15:43.931818  5922 consensus_queue.cc:1048] T b5e2e6b7b0af4d39b62cb855c6bfd81a P 098b8827550f475e8c8fa8a20de8f512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978db1e31e16405182e65e88f3b9bebc" member_type: VOTER last_known_addr { host: "127.26.83.193" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
    @     0x7f59cb7aedad testing::UnitTest::Run() at ??:0
    @           0x38f0bc RUN_ALL_TESTS() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:?
    @           0x38dcbd main at /apache/dev/git/kudu/src/kudu/util/test_main.cc:?
I20260709 13:15:43.981868  4625 raft_consensus.cc:1275] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: Refusing update from remote peer 098b8827550f475e8c8fa8a20de8f512: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:15:43.982265  4625 pending_rounds.cc:85] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82: Aborting all ops after (but not including) 2
I20260709 13:15:43.982477  4625 pending_rounds.cc:107] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:15:43.982713  4625 raft_consensus.cc:2889] T b5e2e6b7b0af4d39b62cb855c6bfd81a P fb69c26178814cacb4f3abd0750d4b82 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
    @     0x7f59c7e60c87 __libc_start_main at ??:0
    @           0x2a338a _start at ??:?
